Rolling with the shutter

A couple of weeks ago, I posted an article about a very weird video effect I saw when I was in a small airplane: The propeller looked like it was in several pieces, with parts of it apparently hovering off the plane. This is obviously not something physically happening to the propeller, but is instead an artifact, an effect occurring inside the camera.

In my explanation, I said it was due to two effects: shutter roll and aliasing. Shutter roll has to do with how the digital detector rapidly scans the scene row by row, which can cause weird warped distortions in rapidly moving objects. Aliasing is when the video frame rate of the camera beats, or resonates, with a cyclic motion in the scene (like a wheel spinning). Although I don’t say so explicitly in the article, I wound up implying that aliasing was the bigger of the two effects.

Here’s the video:

In the video, I actually didn’t mention shutter roll for the simple reason that at the time it slipped my mind! Mea culpa. That’s one reason I wrote the article, so I could add that in.

But my friend and fellow science communicator Destin, who makes the fantastic “Smarter Every Day” video series, has (with the help of another friend, Henry Reich of “Minute Physics”) just put out a new video that explains rolling shutter extremely well. I mean, like very very well. The footage is simply stunning, and you really should watch this whole thing, because it’s so cool:

How about that? I’ve seen a lot of these effects before, but the guitar string and coin spin were new to me. Henry’s animations really bring home how the scanning of the shutter stretches out or compresses the motions of objects in cameras.

They also put together a behind-the-scenes video with more technical details for those of you who, like me, love to dig into the bits (haha) of digital imaging:

So the weird distortion is due to rolling shutter, and the multiple dissociated propeller blades are due in part to aliasing (note how when he changes the scan rate you see a different number of phantom blades).

At one point, near the beginning of that video, Destin says —quite rightly— that Henry is a wizard. He really drives home how this works.

I was surprised to feel a strong pang of nostalgia watching the second video. After I got my degree, I worked at NASA’s Goddard Space Flight Center helping to calibrate a camera that was being built to go onboard the Hubble Space Telescope. Called STIS, for the Space Telescope Imaging Spectrograph, it was an incredibly advanced machine, with three separate detectors and a vast array of filters and spectroscopic settings. My job was to understand its performance: Literally, photons go in one end, and data (brightness, color info and more) come out the other. What happens in between? If you want to fully understand what you’re seeing in the images and spectra, you have to know what’s happening inside the camera.

I used software (IDL, for those of you fluent in ancient languages) to do this analysis, and many times those of us working on this had to dream up odd ways of taking the data and manipulating it so we could understand it better. Watching Henry work reminded me strongly of that, and I’ll admit it made me smile. The first idea I came up with to show the rolling shutter effect would have worked, but would’ve also been inefficient. Henry’s method using a temporal gradient mask is way more efficient. Even as I write these words a part of my brain is chewing over how I’d do this in IDL.

You can take the programmer away from Hubble, but you can’t take the programmer out of the brain.

So I apologize for my first article not being more clear on how this works, and I’m delighted to be able to showcase Destin’s and Henry’s work here. And the point I made in the article remains the same: Seeing is not believing, and what you see is never, ever what you really get. Cameras change what’s really happening, inevitably, and if you don’t understand how, you’ll be at the mercy of those who are trying to fool you when they say, “The camera doesn’t lie.”

Because oh my, yes it does.