Toward Believable Acting for Autonomous Animated Characters

Last month I had the pleasure of presenting some of my team’s recent research at MIG ’22. It’s our first publication, on a topic I care deeply about: acting for autonomous animated characters. Why do NPCs in video games seem so far behind, in terms of acting ability, compared to their counterparts in animated movies? How might we make their acting more believable? This is one of those hard, fascinating problems that are just starting to become tractable thanks to recent advances in machine learning. I’ll have more to say about it soon, but for now, here’s a short video that explains the first small steps we’ve taken in that direction:

You can find the above video, our paper, and also a recording of our 25-minute talk on our new site for the project: https://believable-acting.github.io/

Branching sand patterns

Here in Sargí, Brazil, when it isn’t raining, we get to take a lot of long walks on the beach. One feature we noticed right away were these unusual patterns just on top of the surface: little clusters of wiggly lines made of light sand that contrasted sharply against the dark, damp, compact sand beneath. Some were small and isolated, while others formed dense networks. We wondered out loud: what were these shapes, and where did they come from? Were they the trails of some tiny worm or crustacean? Detritus tossed up from the digging of underground warrens? That was our best guess on the first day. But the shapes were so tiny— barely wider than a few grains of sand— and we never saw any evidence of whatever life we imagined was creating them.

Later in the week, the weather changed, and the shapes changed too. The lines got longer, and they seemed to favor certain directions more than others. In particular, there was a strong breeze blowing up the coast, and the lines were oriented in the direction of the wind. Also there was something vaguely familiar about the way the shapes branched out and meandered, but I couldn’t put my finger on it.

It took a few more miles of walking, staring, and spacing out before it hit me. I knew where I’d seen shapes like this before: senior year of high school, on the screen of my Amiga 1000.

Like a lot of kids at that particular time, I was into fractals. I’d coded up renderers for Mandelbrot sets (in BASIC, super slow!) and other forms of emergent weirdness. And for one class project, I picked diffusion-limited aggregation: a simulation of fractal growth based on randomly meandering particles that stick together when they touch, creating shapes that look like lightning bolts, branching trees or coral fans. (This technique has since been used to great effect by digital artists and creative coders of all sorts.)

Looking down at the sand, I realized what I was looking at was, literally, exactly the process I had simulated (oh so slowly!) on that home computer: an accretion of individual grains of sand, propelled by the wind until they hit an obstacle, at which point they stick firmly in place.

Here are a few more photos of these patterns. Having an idea of how they’re formed doesn’t make them any less fascinating—in fact, it only raises more questions, like: could you “read” the history of wind since the last high tide by analyzing these shapes? Just how much information is encoded in their twisty branches?

I wish I had more time to spend on this (not to mention, shoot some timelapse!) but we leave Sargí tomorrow. As we’ve said many times this trip, “deixa pra próxima.”

The return of Flash?

December 31, 2020 marked the official demise of Adobe (nee MacroMedia) Flash. On that day, a number of web-native interactive artworks (Graffiti Archaeology among them) disappeared from public view, seemingly forever. This felt like a classic case of “why we can’t have nice things”, and it made me sad, but of course we all have had much bigger things to worry about the past couple of years, so I tried to just let it go and move on.

Imagine my delight, then, when I learned about Ruffle, a Flash Player emulator written in Rust. I’ve been tinkering with it the past couple of days, and while it isn’t perfect, it’s really quite impressive! Super easy to install, and when it works, it works everywhere, no fuss.

The grafarc UI reveals a number of ways in which Ruffle seems to differ from the original Flash Player, which makes the user experience a lot jankier than we originally intended. (You can install the Ruffle browser extension if you want to see for yourself.) So, I won’t be switching that on by default just yet– I’ll see if I can work with the developers to fix those glitches first.

But I also tried it on my old synaesthesia visualization applet, and there it works perfectly! (On desktop, at least– because the applet requires you to type words on a keyboard, it won’t work on mobile devices just yet.) So I’m happy to report that with one line of code, an old interactive artwork that I believed long gone has returned, unscathed, from the abyss.

Long live Flash! And three cheers to the devs of Ruffle!

Peppers popping

Time-lapse of two small peppers ripening from green to bright orange.
The peppers ripen amazingly fast once they get started.

One minor success of our summer garden has been the pepper plants. They haven’t produced a lot of peppers, but the ones they’ve made have been crispy and super sweet. Apologies for the shaky camera – it was hard to keep a steady viewpoint with the phone poised on a head of fast-growing lettuce. (Memo to self: maybe use a monopod next time?)

Growing Shiso

The first month of our shiso plants. Left: green shiso; right: red shiso.

This season I got a head start on the garden, and also started branching out with some interesting-sounding seed packets from local growers. One herb I absolutely love, but have only ever seen in Japanese food, is shiso leaf. I wanted to see if we could grow it here, and what else we could do with it besides roll it in sushi. So far it’s been a raging success: a ridiculous number of sprouts have come up, seemingly twice as many as the seeds I planted. The leaves are gorgeous and super aromatic. Really looking forward to seeing what we can make out of this later in the year!

It’s also really fantastic on avocado toast. Left: with pepitas, ancho chile flakes, and cilantro from the garden. Right: with sesame and red shiso.

Monster Mash in Two Minute Papers!

If you’re any kind of graphics geek, you’re probably familiar with the outstanding YouTube channel, Two Minute Papers. If not, you’re in for a treat! In this series, Károly Zsolnai-Fehér picks papers from the latest computer graphics and vision conferences, edits their videos and adds commentary and context to highlight the most interesting bits of the research. But what really makes the series great is his delivery: he is so genuinely excited about the fast pace of graphics research, it’s pretty much impossible to come away without catching some of that excitement yourself.

What an honor to have that firehose of enthusiasm pointed at our work for two minutes!

Monster Mash

This past January I had the incredible good fortune to fall sideways into a wonderful graphics research project. How it came about is pure serendipity: I had coffee with my advisor from UW, who’d recently started at Google. He asked if I’d like to test out some experimental sketch-based animation software one of his summer interns was developing. I said sure, thinking I might spend an hour or two… but the software was so much fun to use, I couldn’t stop playing with it. One thing led to another, and now we have a paper in SIGGRAPH Asia 2020!

Have you ever wished you could just jot down a 3D character and animate it super quick, without all that tedious modeling, rigging and keyframing? That’s what Monster Mash is for: casual 3D animation. Here’s how it works:

Bread and Butter Pickles

When life gives you gherkins, you make bread-and-butter pickles. At least, that’s what I’ve been doing. I started with this recipe, but as usual, had to modify it based on what we happened to have in our spice rack. I made a few rookie moves, like using the mandoline bare-handed (and let me tell you, that’s a mistake you’ll only make once. Those things are vicious!) But the pickles are so worth it. Sweet and spicy, great with a sandwich, or just straight out of the jar in your pyjamas (not that I’d ever do that, no sir.)

Four gherkins picked fresh from the vine.
These fresh gherkins have an amazing flavor, almost sweet, even the weird-looking yellow ones.
Sliced gherkins in a bowl.
Finally got an excuse to use the zigzag blade on the mandoline!
Spices and vinegar in the saucepan.
Cinnamon, turmeric, clove, cardamom, mustard seed, white peppercorns, chili flakes.
Gherkins and onions simmering in the pickling syrup.
3 pint jars of bread-and-butter pickles.
Two 5-inch gherkins, with 1/2 cup of pickling syrup, will just about fill one pint jar.

Growing and shrinking

Some gherkins grow, others shrink…

Our new hand-built garden enclosure seems to be doing its job perfectly: we used wire mesh (or hardware cloth as the pros call it) with 1/2″ holes, too small for rats and mice to crawl through, but still plenty of room for the bees that have been happily pollinating our cucumber flowers.

Vegetable gardening and timelapse photography turn out to be an amazingly good match, because they both seem to make me pay attention to tiny details that would otherwise escape my notice. I never thought much about male and female flowers before, but on this gherkin plant it’s really obvious which ones are which: the males have pointy petals, and the females come equipped with a proto-fruit, ready for seed. Much less obvious is how they behave after pollination: some fruits grow, some shrivel up immediately, and others grow for a while, and then seem to give up halfway and start shrinking again. (I’ve read that this last case is what happens when there are some fertilized seeds, but not enough to fill the entire fruit.)

Here’s that big gherkin from the timelapse above. It was delicious.

Gherkins Growing

I’m blown away by how fast the gherkin plant has grown. In just a few weeks’ time it exploded to ten times its original size, and it hasn’t stopped. On a hot day it can grow 2 inches taller. So far the tree rats have left it alone– our lettuce was not so lucky– but with fruit like this on the vine, I don’t know how long they’ll be able to resist it.

Cassidy Curtis's splendid display of colorful things.