PAINTING WITH MATHEMATICA: A TALK WITH JOHN MUELLERLEILE

When I saw John's artwork, I decided to try my hand at interviewing. Despite being fairly abstract, his work has a unique, surrealist feal. The complexity of his images bears the tell-tale sign of algorithmic artwork. And he makes a lot of it, too, which means it's getting really good really quickly.

John and I share some common background. We've both worked on open-source, non-relational, distributed databases. (I work on Apache Cassandra, he worked on Riak.) However, his artistic work is very different from my own. Wanting to know more about how he operated, I called him up over Skype.

Tyler. Hi John, nice to talk to you!

John. Nice to talk to you as well!

T. To start out, do you have any background in traditional painting or photography?

J. No, I did a year of film school, but it was largely uneventful. It didn't really work for me. I was more interested in the technology at the time than the actual art of making films.

T. Did you do film school before Computer Science?

J. In the middle of. So, I did three years of CS and then I went to film school for a year and kind of scratched that itch, and then I came back to CS.

T. When did you get into your current style of artwork?

J. I've been a life-long doodler, but I've never taken it seriously. I've always liked art class and all that (like everybody else, I guess), but over the last year or so I made the switch from music as my main hobby to visual arts. I quit spending time almost completely on making any kind of music and tried to cultivate some sort of style and technique. It's been about a year, I guess.

T. What tools are you using for your artwork. I know that you mentioned that you're using, uh —

J. Mathematica

T. Right, Mathematica, as well as some other more usual image processing tools. Can you walk me through your normal tech stack?

J. Sure. The overall process is that I'll usually generate some kind of seed image. I'll either generate or take a photo or capture it somewhere. And then from there I do an expansion in Mathematica, usually using inpainting to generate a lot of the pattern-based stuff, and then I take that and bring it in to Photoshop and do a ton of hand editing (these days, anyway). I spend a lot of time hand-editing and patching different areas to get the effects that I want. And I use a suite of sharpening and color tools called the Nik collection from Google.

T. You said you using something in Mathematica called inpainting?

J. Yeah, there's a function in Mathematica called inpaint and it's got a handful of different settings. The output you get from your input can vary wildly. You just kind of learn to feed it weird things after a while to get what you want. So, the process is really centered around there and in post. The general issue is that I'm trying to look for some interesting pattern or some interesting illusion that just happens by chance. So I'll take a photo of a maze or something with built-in patterns and I'll crop it a little off to force the inpaint to generate something a little irregular, or something that just kind of trainwrecks into a new pattern — if I can get it to spit that out.

T. That's something interesting about your work. There's obviously some sort of pattern, but it's always very irregular and it's kind of hard to tell exactly how the program arrived at that pattern.

J. Inject a little chaos in there. When you can see the perfect pattern — unless that's the point of it — I sort of filter it out. I can tune it out. I want to continually create these irregular things with the hope that they start to resemble something that your eye might latch onto and create an illusion.

T. How do you pick out your source images? A lot of them look like photography that you've taken yourself of San Francisco. Do you choose them randomly, or do you tend to seek out images?

J. At first it was random, but these days I will either look for some kind of photo that I specifically remember taking or I'll go and take specific photos for this stuff. I need certain angles, or certain sections of rooftops, or whatever, that I want to capture to try to exploit the patterns. These days it's much more specific than it used to be. I would take like a thousand pictures and run them all through a process to see what happened. From there I would generate certain rules in my head, like, "okay, these kinds of colors and these kinds of patterns" and just go from there to get the feel of it.

T. Would you say that you generally have a decent idea of what your work is going to look like before you've started? Or does it tend to surprise you when you see the output from the first run of the program?

J. Almost universally I'm surprised. That's almost the point of what my process is currently: I'm looking to be surprised. I'm looking to see something in whatever I'm editing. I'll keep editing and coming up with ideas. It usually starts with removing blemishes, or what I perceive to be blemishes, regardless of what the overall form is. And then from there I start to latch onto some idea. Maybe I see waves form, and an island form, and then I just start accentuating that to solidify the idea. So it's almost always a surprise.

T. How would you say that this sort of work is different from, say, programming a database?

J. Oh wow. Completely different. There's not much programming going on here. It's maybe ten lines of Mathematica, total. I mean, Mathematica's got some pretty big functionality baked into small function names. Nonetheless, there's not much code to it.

Although, I will say that the one parallel between those two things is that, I guess, the way I think about the pixels — the actual data in the photo, or the image that I'm working on — you can think about that as sort of like a database. You can go in and select by color ranges if you want, or you can replace colors, or take regions which are just rows and columns. There are parallels there at a very, very low level, inasmuch as they're both organized data. And you can manipulate them in the same ways, which is powerful and makes these tools work the way they do. Beyond that, they're not even both fruit.

T. I can't remember which abstract expressionist you mentioned recently…

J. Still, probably.

T. Right, Clyfford Still. Would you say that the abstract expressionists are one of your biggest influences?

J. For sure. I kind of learned this after the fact. I don't have any background in art, so to speak, so I didn't really know what I was looking at or what I was making, I guess. I saw Clyfford Still's work at the Museum of Modern Art here in San Francisco and I just loved it. They were huge and really moved me. I kind of figured out that that's what I was going toward.

In retrospect, when I would take panoramic images of San Francisco and really warp the terrain and create these weird structures so sometimes they weren't even buildings anymore, I kind of thought I was trying to do something abstract just using the city as the paint. I didn't have any other way to do it. I've been trying to refine the process and capture those elements more than relying on the city.

I also like the Cubist stuff where there's a sort of echo throughout the painting, or multiple perspectives over time or through space. I try to play a lot with that theme, so you'll see several copies of something throughout one of the pieces. The idea there is that whatever that object represents is changing over time.

T. Do you have a particular kind of quality that you're looking for in your finished pieces? Do you have a goal in mind?

J. It really is "I'll know it when I see it". It starts to converge pretty quickly. What I'm really looking for is some kind of illusion. If I can find some way of looking at what I've created in a way that's coherent but still has the quality of disorganization and asymmetry and … it's a little bit glitchy, I guess, but it's more pattern oriented. If I can keep that fundamental quality and still tell a story of some kind, that's the true test of "done".

T. Do you have any personal goals for your artwork?

J. When I get old, I want a timeline of stuff that I've made. This definitely fulfills that. When I'm retired, I want to have an archive of things that I've created. I will keep this up; it's just part of how I operate.

T. You're pretty profilic in the amount of work you put out.

J. Every Saturday is my art day. Usually throughout the week I'll run batch jobs to generate the starting material. So I prepare during the week just by letting the computer be my painting assistant. Come Saturday, I go through all the work of my assistant and cherry-pick the ones that have promise. I'll take five or ten that are my favorites and get through one to three, rarely more than that.

T. When you're looking through the images that have been generated by the batch jobs, how many images are we talking about?

J. It varies pretty widely. It depends on the theme. If I'm doing mazes I might do 20 different ones. If it's forests, maybe less, becase there's not as much variation.

T. What's the most difficult part of your artistic process. What's the most challening?

J. One kind of funny aspect is that I'm red-green color blind. So yellows and greens and reds and browns can become convoluted, depending on their context. For a long time I used glitch panoramas because I couldn't screw up the color. I wanted them to look right, and if I chose them, there was a high probability that I would get it wrong and it would look very awkard to other people. And I wasn't very into just making art for other color blind people. I did that for a long time. Now the hardest part is that with a lot of these compositions I have to be really careful with the color to make sure that I don't screw it up. Or I just lean into it as hard as I want and it just becomes something else.

T. How would you recommend that other programmers get started with this sort of artwork?

J. Find the entry point, how you create the original content. If you like to draw, photograph those or scan them in and then start. Find some kind of "instrument" to play to start the process. The rest should be a chain reaction if you still like it. The other thing is, be shameless. Just throw your pride in the trashcan. Get it out there and that will make you take your quality level seriously.

T. Anything else you want to mention?

J. You had mentioned programming a few times. I really push the other way, I think it's all about the tools. People like you and I and our peers have the ability to make those tools, and so that's why I like that there's more and more people starting to do the kind of stuff that we do. Good tools are going to pop out the other side and those tools are going to define some new stuff. New takes on old stuff. And that's really interesting to me.

T. I think tools like Processing are good for artists who don't have a programming background to get started, but …

J. I think Processing is a great step, but it's really… I don't know. I've been programming for a long time, and I've been interested in different artistic pursuits in semi-serious ways for a long time, and I've tried coding sound in Supercollider or whatever, and it just breaks my creative process. I automatically go into work mode and I'm doing the accounting of logic, and resource management, and that just turns off the other side — the wildly free associating that I'm trying to do when I actually create the image.

T. Alright, well, anything else, John?

J. No, but thanks for asking me to do this, it's been fun talking about it. I appreciate you thinking of me.

T. Absolutely.


If you like John's artwork, you can follow him on Instragram at @jrecursive, where he usually posts cropped images, or on Twitter at @jrecursive, where he posts uncropped images (and links to high resolution images, which is particularly nice).

John also recently made prints of his artwork available for sale.