This is a not great quality picture of it, but I recently finished a painting inspired by the head of Hygeia in the National Archaelogical Museum in Athens.

There’s an underlying drawing I did of the subject from memory based on other studies I did by hand. I took this and projected the photo of the drawing onto my canvas:

It’s not an incredible drawing but was good enough to get the shapes in. But It didn’t end up seeming right and I painted over it a good bit, and sort of fragmented the features in a pseudo-neo cubist inspired thing. But didn’t end up liking that enough, so took a gamble on another projected image, one made by Dalle while trying to get it to reproduce something in the style of the original Hygeia museum piece.

This is my second time using AI in conjunction with a physical painting. But the last one was a reference image that I just based my own drawing off of freehand. This is the first time trying to overlay directly onto the canvas via tracing off a projection.

It’s gotten me thinking lots of crazy thoughts about how to potentially integrate AI tools into actual physical painting (and other art-making). Like what if I could hook my projector up to a generative AI which is being periodically fed webcam images of the latest state of my IRL canvas. And then it could automatically or when I ask it, suggest next brushstrokes or propose visual alterations, styles, directions I might explore. And these would be projected as overlaid suggestions on my canvas, which I could then “accept” or reject or whatever by painting them in myself in my own way.

It would be something like a camera lucida with an AI in the loop:

This train of thought sent me down a deep deep rabbit hole about early and possibly secret use of optics my artists going back to the 1400’s possibly, according to the Hockney-Falco thesis. You can see a great BBC documentary on Hockney’s theories, and he’s got a book about it I haven’t read called Secret Knowledge. Here’s a Youtube embed:

Another related one that’s worth watching is Tim’s Vermeer which goes on a similar track of trying to prove Vermeer used a similar method with mirrors and something something. Pretty fascinating all around.

I found there’s a business trying to market the “Vermirror,” or as others have called it more generically, the comparator mirror. From what I can tell, the trick is basically getting a first surface or front surface mirror, which is brighter and reflects on its front face instead of its back face. And it’s held by some kind of rigid arm to keep it in the correct position relative to your work surface and subject. This video of art students using them gives a good idea of how it works:

I’ve currently got on order some bits and bobs that I will finagle together into being something like this. The advantage to using the mirror is that unlike when you project directly onto your canvas, you can still see the colors you’re mixing. They aren’t washed out or modified by the projected image.

Anyway I tried to get Dalle to generate a patent-style drawing of what a contraption like this might look like, where a combination projector & web cam capture images of a canvas, send it for processing, and send back suggested next strokes. It’s a little wonky and disconnected, but it shows some fun relationships nonetheless, and it lets the mind wander a bit looking at it, thinking about the possibilities.

I actually tried getting Dalle to do this task:

  • Take my painting image as input
  • Reproduce the image with suggested modifications

But it doesn’t seem able to reproduce an image, let alone modify it.

So then I had the idea of like okay:

  • From the input image, make a verbal description of what you would change.
  • It was able to do that no problem.
  • Then I said, okay take the changes you would make based on your verbal description, and draw them on a new image which can be used as overlay to guide painting in changes on the original image

But nope, that didn’t work. It got stuck in a loop of thinking it was helping me and linking to non-existent Imgur images. So that’s a little disappointing, but I think there’s something here. I just have to figure out the right pathway with the right tools to get to it. And Dalle is probably not the one for this, currently.

I did manage to get one image set out of this when I was asking for overlays for how it would modify the original. It gave me this:

It’s not really what I had in mind by overlay and suggesting brush strokes at a micro-level that is in line with the style of the original input image. But I did notice that when I scaled it and rotated it a little, it’s actually fairly close to many of the major facial landmarks:

Obviously the neck and ear arrangements are different, but there’s a lot of match here, even though I don’t agree at all with the stylistic direction. But also that’s kind of the point of a collaboration too, that you both bring in completely different ideas and references and impulses. So maybe there is value to exploring in this part of the liminal latent/real image space…

One final thought on that note, from a Guardian 2014 piece criticizing the Vermeer documentary, I think quite wrongly:

The film is a depressing attempt to reduce genius to a trick. […]

The technology Jenison relies on can replicate art, but it does so synthetically, with no understanding of art’s inner life. The “Vermeer” it spits out is a stillborn simulacrum.

Love it or leave it, but we live in a world where the simulation in many cases is more “real” in some aspects than the so-called Real. And this notion that studying how this might work is somehow “cheating” or a “trick” itself to me completely misses the boat on what art is: it’s artifice upon artifice, and somehow through that, finding authenticity despite everything.

Related writing & research