One minor success of our summer garden has been the pepper plants. They haven’t produced a lot of peppers, but the ones they’ve made have been crispy and super sweet. Apologies for the shaky camera – it was hard to keep a steady viewpoint with the phone poised on a head of fast-growing lettuce. (Memo to self: maybe use a monopod next time?)
This season I got a head start on the garden, and also started branching out with some interesting-sounding seed packets from local growers. One herb I absolutely love, but have only ever seen in Japanese food, is shiso leaf. I wanted to see if we could grow it here, and what else we could do with it besides roll it in sushi. So far it’s been a raging success: a ridiculous number of sprouts have come up, seemingly twice as many as the seeds I planted. The leaves are gorgeous and super aromatic. Really looking forward to seeing what we can make out of this later in the year!
If you’re any kind of graphics geek, you’re probably familiar with the outstanding YouTube channel, Two Minute Papers. If not, you’re in for a treat! In this series, Károly Zsolnai-Fehér picks papers from the latest computer graphics and vision conferences, edits their videos and adds commentary and context to highlight the most interesting bits of the research. But what really makes the series great is his delivery: he is so genuinely excited about the fast pace of graphics research, it’s pretty much impossible to come away without catching some of that excitement yourself.
What an honor to have that firehose of enthusiasm pointed at our work for two minutes!
This past January I had the incredible good fortune to fall sideways into a wonderful graphics research project. How it came about is pure serendipity: I had coffee with my advisor from UW, who’d recently started at Google. He asked if I’d like to test out some experimental sketch-based animation software one of his summer interns was developing. I said sure, thinking I might spend an hour or two… but the software was so much fun to use, I couldn’t stop playing with it. One thing led to another, and now we have a paper in SIGGRAPH Asia 2020!
Have you ever wished you could just jot down a 3D character and animate it super quick, without all that tedious modeling, rigging and keyframing? That’s what Monster Mash is for: casual 3D animation. Here’s how it works:
When life gives you gherkins, you make bread-and-butter pickles. At least, that’s what I’ve been doing. I started with this recipe, but as usual, had to modify it based on what we happened to have in our spice rack. I made a few rookie moves, like using the mandoline bare-handed (and let me tell you, that’s a mistake you’ll only make once. Those things are vicious!) But the pickles are so worth it. Sweet and spicy, great with a sandwich, or just straight out of the jar in your pyjamas (not that I’d ever do that, no sir.)
Our new hand-built garden enclosure seems to be doing its job perfectly: we used wire mesh (or hardware cloth as the pros call it) with 1/2″ holes, too small for rats and mice to crawl through, but still plenty of room for the bees that have been happily pollinating our cucumber flowers.
Vegetable gardening and timelapse photography turn out to be an amazingly good match, because they both seem to make me pay attention to tiny details that would otherwise escape my notice. I never thought much about male and female flowers before, but on this gherkin plant it’s really obvious which ones are which: the males have pointy petals, and the females come equipped with a proto-fruit, ready for seed. Much less obvious is how they behave after pollination: some fruits grow, some shrivel up immediately, and others grow for a while, and then seem to give up halfway and start shrinking again. (I’ve read that this last case is what happens when there are some fertilized seeds, but not enough to fill the entire fruit.)
Here’s that big gherkin from the timelapse above. It was delicious.
I’m blown away by how fast the gherkin plant has grown. In just a few weeks’ time it exploded to ten times its original size, and it hasn’t stopped. On a hot day it can grow 2 inches taller. So far the tree rats have left it alone– our lettuce was not so lucky– but with fruit like this on the vine, I don’t know how long they’ll be able to resist it.
Some dear friends gifted us a garden starter kit with tons of herbs and vegetables to grow from seed. The gherkins are growing like crazy! Here’s a super rough timelapse of about two weeks of growth.
It’s an interesting time to be an artist. As machine learning becomes part of the toolkit, in different ways for different people, new ideas are shaking loose, and I feel compelled to write about them as a way of wrapping my head around the whole thing.
The most recent headquake hit me by way of the ML-assisted album Chain Tripping by post-punk-pop band YACHT. Here’s a great Google I/O talk by bandleader Claire Evans that describes just how they made it. (Tl;dr: no, the machines are not coming for your jobs, songwriters! Using ML actually made the process slower: it took them three years to finish the album.) This case is interesting for what it tells us about not just the limitations of current AI techniques, but also the creative process, and what makes people enjoy music.
In music there’s this idea that enjoyment comes from a combination of the familiar and the unexpected. For example, a familiar arrangement of instruments, a familiar playing style, with a surprising melody or bass line. Maybe it works like visual indeterminacy: it keeps you interested by keeping you guessing.
As genres go, pop music is particularly information-sparse. What I take from YACHT’s example is that low level noise— nearly random arrangements of words and notes— can produce occasional bursts of semi-intelligible stuff. By manually curating the best of that stuff and arranging it, they pushed the surprise factor well above the threshold of enjoyability for a pop song. And then they provided the familiarity piece by playing their instruments and singing in their own recognizable style. The result: it’s pretty damn catchy.
So if you like the album, what is it exactly that you like? It sounds to me like what you’re enjoying is not so much the ML algorithm’s copious output of melodies and lyrics, but YACHT’s taste in selecting the gems from within it. So far, so good. But there’s another piece of this puzzle that makes me question whether this analysis is going deep enough.
The first time I watched the video for SCATTERHEAD, one lyric fragment jumped out at me: “palm of your eye”. I’m not alone: NPR Music’s review calls it out specifically as a “lovely phrase … which pins the lilting chorus into place”. But it jumped out at me for a rather different reason: I’d heard those exact words before. I immediately recognized them from Joanna Newsom’s 2004 song Peach, Plum, Pear.
At the time, not knowing anything about YACHT’s process, I assumed they were making an overt, knowing reference to Newsom’s song. But then I learned how they generated their lyrics: they trained the ML model on the lyrics of their own back catalog plus the entire discography of all of the artists that influenced them. This opens up another plausible explanation: it could be that Newsom was among those influencers, the model lifted her lyric whole cloth, and YACHT simply failed to recognize it. If that’s the case, it would mean the ML model performed a sort of money-laundering operation on authorship. YACHT gets plausible deniability. Everyone wins.
This sounds like a scathing indictment of YACHT or of ML, but I honestly don’t mean it that way. It really isn’t that different from what happens in the creative process normally. Humans are notoriously bad at remembering where their own ideas come from. It’s all too common for two people to walk away from a shared conversation, each thinking he came up with a key idea. For example: witness the recent kerfuffle about the Ganbreeder images, created by one artist using software developed by another artist, unknowingly appropriated by a third artist who thought he had “discovered” it in latent space, and then exhibited and sold in a gallery. So, great, now we have yet one more way that ML can cloud questions of authorship in art.
But maybe authorship isn’t actually as important as we think it is. Growing up in our modern capitalist society, we’ve been trained to value the idea of intellectual property. It’s baked into how working artists earn their living, and it permeates all kinds of conversations around art and technology. We assume that coming up with an original idea means you own that idea (dot dot dot, profit!) But capitalism is a pretty recent invention, and for most of human history this is not how culture worked. Good ideas take hold in a culture by being shared, repeated, modded and remixed. Maybe there’s a way forward from here, to a world where culture can be culture, and artists can survive and even thrive, without the need to cordon off and commodify every little thing they do. It’s a nice dream, at any rate.
At some level this is just me, sticking a toe in the water, as I get ready to add ML to my own toolkit. (It’s taken me this long to get over my initial discomfort at the very thought of it…) When I do jump in, we’ll see how long I can keep my eyes open.
I’m heading to LA this coming weekend to do a retrospective talk about Spotlight Stories. It’s part of an ASIFA-organized “Animation Day” event at Infinity Festival Hollywood. starting Saturday, November 9th at 10am. (Our friends from Baobab will also be doing a talk about some of their latest work, so it should be a really interesting morning!) Angelenos, swing by and say hello!