Category Archives: interactive

Vote for “Pearl” for the FoST Prize!

screen-shot-2016-09-16-at-8-40-04-am

Pearl is one of twenty finalists for the Future of Storytelling Prize! Now’s your chance to vote for Pearl!

kaleidovr

In other news, we’ll also be showing Pearl at the Kaleidoscope VR Summer Showcase, which travels around the world to London, Seoul, Berlin, New York, San Francisco and Los Angeles. I’ll be at the SF event on September 30th.

screen-shot-2016-09-16-at-8-51-01-amPearl has also been nominated in three categories (Narrative, Mobile, and Original Score) for the Virtual Reality Foundation’s third annual Proto Awards, coming up on October 8th. The nominees all look amazing. I can’t wait to meet them!

Atlas of Emotions

For a few weeks last spring I had the tremendous pleasure of working with my dear friend Eric Rodenbeck on an amazing project: an Atlas of Emotions. Commissioned by the Dalai Lama, and based on decades of scientific research by Paul Ekman and his colleagues, the project aims to help people find a path through the complex landscape of their feelings toward a state of calm and happiness.

This was such a fresh and exciting experience. First, because Stamen is an absolutely lovely place to spend time for any reason. (Seriously: pineapple plants and bubble machines!) Second, because it forced a connection between parts of my brain that had never met before: emotion brain, meet design brain. Well, hello! My time on the project was brief and my contribution very small, but will that stop me from kvelling? No it will not! The rest of my feelings can be found right here.

Something New

Here’s something different: I have a new job! Today was my first day at Google Spotlight Stories. I’ll be working with some amazing filmmakers and technologists who are busy inventing a new kind of narrative visual storytelling uniquely suited to handheld mobile devices. If that sounds crazy, that’s because it is. It’s my kind of crazy. It’s exactly the kind of wild, inventive, “let’s try this and see what happens” attitude that got me interested in computer graphics in the first place, all those years ago. I couldn’t be more excited.

The video above really does a great job of capturing the delight of experiencing one of these stories for the first time. It’s almost impossible not to grin like a ninny. There’s not much more I can say about it right now, but there’s been some terrific press about the projects they’ve created so far. I’ll share more when I can!

Mapping the brain’s semantic space

A color-coded map of four dimensions of the semantic space. The branches show "is-a" relationships.
A color-coded map of four dimensions of the semantic space. The branches show “is-a” relationships.

This is one of those moments in science that makes me think I’m so lucky to be alive right now. A team of scientists at UC Berkeley have found a way to map the brain’s representations of objects into a shared semantic space— a multidimensional space in which related things are nearer than unrelated ones. And there’s reason to believe this might be not just a semantic space, but the semantic space: they ran their test on five different people, and found that the first four dimensions of this semantic space were the same for all five subjects–dimensions easily labeled with ideas like moving/stationary, man-made/natural, animate/inanimate, and so on. In other words, the brain’s way of relating different objects might be something we all share at much more than a superficial level. This alone is pretty mind-blowing to me.

As if that weren’t enough, they’ve also created a very cool interactive visualization that shows how all of this plays out on the surface of an actual brain. (That page requires WebGL and a lot of memory, so if you’re reading this on an older device, you might want to just watch the video instead. Actually, you should watch the video anyway, because it’s really well done!)

Go science!

Exquisite corpse fold-in pixel art!

I’ve been a fan of Ed Stastny’s work since the early days of the web, when SITO.org clued me in to the awesome possibilities of massively collaborative, software-guided web art projects. Gridcosm was surprising, disturbing, inspiring, and highly addictive. It was also a big influence on me personally: without that shining example of the weird things a group of total strangers can do together, I may never have come up with the idea to put Graffiti Archaeology on the web.

Now Ed has a wonderful new project in the works, and he needs your help to make it happen. Yono mashes up 8-bit pixel art, the exquisite corpse surrealist parlor game and Mad Magazine’s fold-ins. It picks up where Gridcosm left off, but it takes full advantage of the magical powers of today’s Internet: it will come with its own pixel art painting tool, and will be designed to work on pretty much any device, so the barriers to participation will be far lower, meaning everyone can play.

He’s offering some very cool thank-you gifts on his Kickstarter, and there are only 9 days left! So if this is something you’d like to support, jump in and help, will you?

Stamen’s watercolor maps

My friends at Stamen keep doing wonderful new things to maps. Today’s innovation: a globe’s worth of map tiles rendered automatically in the style of a watercolor painting.

This is the kind of project I could only dream of, back when I was doing this kind of research. So much has happened since then! Stamen’s tiles combine scanned watercolor textures with vector map data from Open Street Maps, and various image-based filters for effects like edge darkening. The results are organic, seamless and beautiful. Even though I know the textures will repeat themselves eventually, I can’t help scrolling out into the ocean, just to see what’s there.

Update: Zach has posted a really nice step-by-step breakdown of how they did this. (Gaussian blurs and Perlin noise are your friends here.) I particularly love the way they use small multiples and animated gifs to explore the parameter space:

Update 2: And now Geraldine has put up her post about painting the watercolor background textures. A process of discovery and exploration, with much trial and error. As a bonus, she’s put up a set of lovely texture samples on Flickr.

Flickr set of watercolor samples

The Fourth Dimension (for your phone)

The third eye is for seeing the fnords.

If you look closely at my friend Drew Olbrich, you might notice something strange about him. He doesn’t like to talk about it, but: the part of Drew you can see is just an infinitesimal slice of a much higher-dimensional being.

To give the rest of us a sense of what it’s like to be him, Drew’s written a nifty little app (for your iPhone or iPad) called The Fourth Dimension. Give it a spin, and see if it doesn’t thicken your mind up just a little bit.

Frames per second

Flickr photo courtesy of purplemattfish
Flickr photo courtesy of purplemattfish

There’s been some discussion brewing among certain filmmakers about the impact of making movies that play faster than the current standard of 24 frames per second. Peter Jackson is shooting The Hobbit at 48fps, and others are reportedly experimenting with rates like 60 or even 120.

Mixed into the discussion are some really deep misconceptions about how vision and perception actually work. People are saying things like “the human eye perceives at 60fps”. This is simply not true. You can’t quantify the “frame rate” of the human eye, because perception doesn’t work that way. It’s not discrete, it’s continuous. There is, literally, no frame rate that is fast enough to match our experience of reality. All we can do, in frame-by-frame media, is to try to get closer.

The problem is that our eyes, unlike cameras, don’t stay put. They’re active, not passive. They move around in direct response to what they are seeing. If you watch an object moving past you, your eyes will rotate smoothly to match the speed of the thing you’re looking at. If what you’re looking at is real, you will perceive a sharp, clear image of that thing. But if it’s a movie made of a series of discrete frames, you will perceive a stuttering, ghosted mess. This is because, while your eyes move smoothly, each frame of what you’re watching is standing still, leaving a blurry streak across your retina as your eyes move past it, which is then replaced by another blurry streak in a slightly different spot, and so on. This vibrating effect is known as “strobing” or “judder”.

Applying camera-based effects like motion blur only makes the mess look worse. Now, your stuttering ghosted multiple image becomes a stuttering, ghosted blurry multiple image. (The emphasis on motion blur is particularly bad in VFX-heavy action movies, which is why I try to sit near the back.)


Click the image to see a demonstration of the "judder" effect. This is what your eyes actually see when you watch an object moving back and forth on a movie screen. Even with motion blur, you can see that there's a distracting sawtooth vibration to the ball that can be reduced, but not eliminated, by increasing the frame rate.

Filmmakers tend to work around this problem by using the camera itself as a surrogate for our wandering eye: tracking what’s important so that it effectively stays put (and therefore sharp) in screen space. But you can’t track everything at once, and a movie where nothing ever moves would be very dull indeed.

I am pretty sure there is no frame rate fast enough to completely solve this problem. However, the faster the frame rate, the less blurring and strobing you’ll experience as your eyes track moving objects across the screen. So I am extremely curious to see what Jackson’s Hobbit looks like at 48fps.

There’s a second problem here, which is cultural. My entire generation was raised on high quality feature films at 24 frames, and poorer-quality television (soap operas, local news) at 60 fields per second. As a result, we tend to associate the slower frame rate with quality. Commercial and TV directors caught on to this decades ago, and started shooting at 24fps to try to get the “film look”. How will we perceive a movie that’s shot at 48fps? Will it still feel “cheap” to us? And what about the next generation, raised on video games that play at much higher frame rates? What cultural baggage will they bring into the theater with them?