Now, in 2017, machine learning comes on the scene and opens entirely new frontiers for nonsense-lovers. Witness this attempt to train a neural network to generate paint color names based on the contents of the Sherwin-Williams catalog (producing some names that could have come straight out of LiarTown, USA):
I wasn’t sure about machine learning before, but it’s growing on me.
Links via @golan, via @annaleen, and via the great glorious Internet, long may she reign.
The theatrical print of “Pearl” will be screened next week at the San Francisco International Film Festival, in a collection of family-friendly animated shorts. I’ll be there at the 10am Sunday screening at the Castro Theater, along with some of the other filmmakers, for Q&A. Stop by and say hi if you’re in the area!
At the VR Storytelling Meetup last night, an interesting conversation with the other panelists got me thinking about frame rates again. Apparently, for filmmakers shooting live-action 360º video, the high frame rate required for playback in a VR device can be a bit of an obstacle. Not just technically, but psychologically: it’s a turnoff for the audience.
I felt that emotional turnoff when I finally saw Peter Jackson’s first Hobbit movie at 48 frames per second. It was astonishing and beautiful in the sweeping exterior shots. But when it was just characters sitting and talking, it felt… fake. I found myself scrutinizing the makeup, looking for flaws and finding them. At the time I attributed it to a cultural bias: because I grew up in an era when high quality entertainment came in the form of 24p films, and cheesy soap operas were 60i video, I must subconsciously associate high frame rates with low quality.
But what if there’s more to it than that?
In a recent interview about Pearl, Patrick Osborne pointed out that simplifying the visual style, removing texture and detail, leaves room for the audience to put themselves into the characters. It lowers a barrier to empathy. Scott McCloud said as much in Understanding Comics. This is why I’ve always preferred non-photorealism over realism. It’s what you leave out that counts.
What if a similar mechanic is at work around the question of frame rates? The secondhand report from the live action VR filmmaker was that at 60fps, it felt too obvious that the people were actors. You could look at a background character and tell instantly that they were pretending. Leaving aside the possibility that it may have just been bad acting: is it possible that the high frame rate itself lets you see through the ruse? Could it be the density of information you’re receiving that pushes your perceptiveness over some threshold, and makes you a sharper lie detector?
And if that turns out to be the case: how should filmmakers respond?
I’ll try to answer that and other questions, alongside a really interesting mix of writers, artists and technologists on a panel hosted by the VR Filmmakers Meetup this Thursday, February 16 in San Francisco. Tickets available here. Hope to see you there!
Pearl has also been nominated in three categories (Narrative, Mobile, and Original Score) for the Virtual Reality Foundation’s third annual Proto Awards, coming up on October 8th. The nominees all look amazing. I can’t wait to meet them!
SIGGRAPH attendees are a sophisticated audience, so demoing Pearl in the VR Village last week led to some really interesting conversations. One thing I heard more than once was this idea that to do storytelling in VR, we have to throw out all the rules of traditional cinema. While I appreciate the swashbuckling spirit of that sentiment, I don’t think it’s actually true.
I had a life drawing instructor in college who used to teach us rules like “highlights are circular, and shadows are triangular.” As a math major, this really bothered me at the time, because taken literally it was provably false– just give me a flashlight and a grapefruit and I’ll show you! But that was missing his point. The human body is made of smooth, convex masses, and the highlights on them do indeed tend to be round. And when one limb casts a shadow on another, the contour of the shadow’s edge wraps around and hits the silhouette at an angle, forming a sharp point. In other words, “triangular”. So my teacher’s rule, within the context of human figure drawing, was totally valid and actually pretty insightful. But it wasn’t a law of nature, it was something he invented. And to construct it, he had to synthesize knowledge from human anatomy, physics, geometry, and visual perception.
The rules of filmmaking seem atomic and universal to us, but they’re not. Like the “triangular/circular” rule, they’re chimaeras, hybrid creatures assembled from bits of wisdom from different disciplines. They’re not real the way math and biology are real, we’re just so used to them that we mistake them for reality.
For example, take film’s 180º rule. That’s the rule that says if you’re shooting a conversation between two characters, there’s an imaginary line connecting them, and when you cut from shot to shot, you always have to keep the camera on the same side of it. Cross that line, and you risk confusing your audience. This rule has elements of geometry (projecting 3D space to a plane), perception (how humans construct mental models of 3D space) and psychology (how we organize those models based on relationships between people). That’s a lot of moving parts! Now imagine trying to apply this rule to a VR experience where you can walk around the scene. Some of those elements change (the flat screen becomes a volume) but the perception and psychology parts are still there. So the question is not whether to keep the 180º rule or throw it away. The question to ask is which parts do we keep, and what else do we add into the mix, to construct a new rule that works for VR?
For VR storytelling, we shouldn’t have to throw out the rules of the mediums we know and love. But we can unpack them, dismantle them into their component parts, and analyze them at a deeper level than we’re used to doing. And that’s going to be a fun way to spend the next few years, for all of us.
Pearl will have a big presence at SIGGRAPH this year! We’re doing our making-of presentation in a Production Session on Sunday, July 24th from 10:45-12:15, and showing it on the Vive in the VR Village all day from Sunday through Thursday. Pearl will also be shown at the Appy Hour event on Wednesday, July 27th from 5-7 pm.
I’ll only be there Sunday-Tuesday, but I’m sure looking forward to it!
Cassidy Curtis's splendid display of colorful things.