Category Archives: perception hacking

What would 120fps mean for animation?

Following up on yesterday’s post about higher frame rates in movies, there’s another question looming. If high frame rates catch on industry-wide, what will it mean for animators?

We won’t really know for sure until the movies start coming out. But we can guess. There are televisions on the market that will play movies at 120fps regardless of how the movie was shot. They do this by creating the inbetween frames automatically, in real time. (How exactly it’s done, I’m not sure, but it’s probably some sort of optical flow technique, like Twixtor.) When you see this done to a live action movie shot at 24fps, the effect is impressive: movement really does feel incredibly smooth, and the strobing/juddering problem is minimal. But if you watch an animated movie on one of these TVs, the results are not good. Timing that felt snappy at 24fps feels mushy at 120. Eyes look bizarre during blinks. And don’t get me started on smear frames.

Of course, this is just a machine trying its best to interpolate frames according to some fixed set of rules. Animators will be able to make more intelligent choices, which of course it’s our job to do. But that’s where it gets interesting. How many frames should a blink take at 120fps? What’s the upper limit on how snappy a move can be, if you can potentially get from one pose to another in a mere 8 milliseconds? It could open up new creative possibilities too. Take staggers for example: at 24fps, if you want something to vibrate really quickly, your only option is to do it “on ones” (that is, alternating single frames). But at 120fps, you could potentially have staggers vibrating on anything from ones to fives. How will those different speeds feel to the audience?

One thing seems pretty certain: animating at 120fps would be a lot more work. For animators who agonize over every frame, it will mean five times the agony. It will certainly mean more reliance on computer assistance: more spline interpolation, fewer hand-crafted inbetweens, and forget about hand-drawing every frame! I look forward to hearing animators’ stories from the trenches on Hobbit. Will they find 48fps twice as hard, or more, or less? What tricks will they have to invent to make their job manageable?

Frames per second

Flickr photo courtesy of purplemattfish
Flickr photo courtesy of purplemattfish

There’s been some discussion brewing among certain filmmakers about the impact of making movies that play faster than the current standard of 24 frames per second. Peter Jackson is shooting The Hobbit at 48fps, and others are reportedly experimenting with rates like 60 or even 120.

Mixed into the discussion are some really deep misconceptions about how vision and perception actually work. People are saying things like “the human eye perceives at 60fps”. This is simply not true. You can’t quantify the “frame rate” of the human eye, because perception doesn’t work that way. It’s not discrete, it’s continuous. There is, literally, no frame rate that is fast enough to match our experience of reality. All we can do, in frame-by-frame media, is to try to get closer.

The problem is that our eyes, unlike cameras, don’t stay put. They’re active, not passive. They move around in direct response to what they are seeing. If you watch an object moving past you, your eyes will rotate smoothly to match the speed of the thing you’re looking at. If what you’re looking at is real, you will perceive a sharp, clear image of that thing. But if it’s a movie made of a series of discrete frames, you will perceive a stuttering, ghosted mess. This is because, while your eyes move smoothly, each frame of what you’re watching is standing still, leaving a blurry streak across your retina as your eyes move past it, which is then replaced by another blurry streak in a slightly different spot, and so on. This vibrating effect is known as “strobing” or “judder”.

Applying camera-based effects like motion blur only makes the mess look worse. Now, your stuttering ghosted multiple image becomes a stuttering, ghosted blurry multiple image. (The emphasis on motion blur is particularly bad in VFX-heavy action movies, which is why I try to sit near the back.)


Click the image to see a demonstration of the "judder" effect. This is what your eyes actually see when you watch an object moving back and forth on a movie screen. Even with motion blur, you can see that there's a distracting sawtooth vibration to the ball that can be reduced, but not eliminated, by increasing the frame rate.

Filmmakers tend to work around this problem by using the camera itself as a surrogate for our wandering eye: tracking what’s important so that it effectively stays put (and therefore sharp) in screen space. But you can’t track everything at once, and a movie where nothing ever moves would be very dull indeed.

I am pretty sure there is no frame rate fast enough to completely solve this problem. However, the faster the frame rate, the less blurring and strobing you’ll experience as your eyes track moving objects across the screen. So I am extremely curious to see what Jackson’s Hobbit looks like at 48fps.

There’s a second problem here, which is cultural. My entire generation was raised on high quality feature films at 24 frames, and poorer-quality television (soap operas, local news) at 60 fields per second. As a result, we tend to associate the slower frame rate with quality. Commercial and TV directors caught on to this decades ago, and started shooting at 24fps to try to get the “film look”. How will we perceive a movie that’s shot at 48fps? Will it still feel “cheap” to us? And what about the next generation, raised on video games that play at much higher frame rates? What cultural baggage will they bring into the theater with them?

Christian Faur’s crayon synesthesia

christianfaur_450

Christian Faur is an artist who makes these wonderful image-sculptures out of custom-cast crayons. The work itself is very Rungy Chungy and appealing to the eye. And Faur apparently has a thing for certain mathematicians. But it also turns out that he’s encoded synesthetic messages into several of his pieces:

Further, I have developed a mapping system that translates the English alphabet into twenty six discrete colors and I use these crayon “fonts” to add words and language to each of the pieces in the show… The direct representation of language in each piece further imbues the works with meaning and brings an aspect of color into each composition reminiscent of DNA coding. The alphabetic key at the lower left of each panel allows the viewer to interpret the individual words written throughout the various panels.

I gather that Faur does not actually have synesthesia himself, but this system makes perfect sense to me. If only he’d chosen his colors differently, I might be able to read the text straight up without the key! (Although it’s pleasing to note that his C, E, and Z are all perfect matches for my own.)

(via Drawn)

 

Head-mounted telestereoscope!

melinda_photo.jpgmelinda_app.jpg

For the past few months I’ve been corresponding with Melinda Green about a project of hers, to build a portable stereo-enhancing device: essentially, a head-mounted version of the Telestereoscope. Her goal? Aerial combat with radio-controlled gliders! She just finished her first prototype, and it looks amazingly lightweight, and I dare say almost comfortable. I can’t wait to hear her first battle stories. Also be sure to check out the cool applet she wrote to visualize all the angles and proportions. Rock on, Melinda!