Cinematography is the art of camera placement in filmmaking. It's an important feature in the language of film. A camera placed overhead can give the viewer the impression of being very tall, and can make a character look small or inferior. A camera that frames its subject to the side can suggest that some action is about to take place in the negative space. And if it sneaks up behind a character, they can build suspense. But how can we take these well-established tools of live-action filmmaking and use them to educate the public with data? One of the major arguments for doing scientific visualization in a cinematic style is that we can take advantage of audiences' familiarity with filmmaking techniques. Techniques that were pioneered with physical movie-making equipment like film cameras, fresnel lights, and all the limitations of the physical world. When we're trying to design a visualization that immerses our audiences and suspends their disbelief, we're the most successful when we follow the rules Hollywood filmmakers have established for themselves. Or at least follow the same rules as much as we can considering we're touring data environments that humans and cameras could never go. One of these informal rules of cinematography is called the rule of thirds. It suggests that people find images more pleasing when the subject is off center. And to compose a shot you might place the subject, in this case, a 3D scan of a mammoth skeleton, along the lines that divide the image in thirds. It's good to keep this rule in mind when framing moving imagery as well. Often, cinematographers will start a shot with their focal object in one third of the screen and end the shot with it in the opposite third. Another informal rule is that audiences feel more immersed in a shot if the camera is kept at the height of an average human. In this more didactic visualization, we are flying high off the ground and feel disconnected from the storm. But in this visualization, with the camera closer to a human eye line, we feel like we're on the ground being threatened by the storm. Of course, this rule is harder to follow when you're floating in outer space or between atoms. Playing with the perceived height of the viewer is a good way to convey the scale of your subject matter. By placing the viewer high above or down below the subject, you can make the subject seem huge or tiny. Framing your subject is another important part of cinematography. Filmmakers have named different framings of their human subjects. A close-up shot generally means the character's face fills the image. A medium shot shows more of their upper body and some of the background, and a wide shot fits their whole body and a lot of the background. In visualization, we can create analogous framings of our data subjects. A wide shot of your data can establish a sense of location and environment, and maybe even which direction is up, while they're floating through space. Typically, scenes in films begin with a wide shot, also called the establishing shot. And the close-up shot of your data can focus the viewer and help drive the narrative forward. Keep in mind that the edges of your image are not the only frame you can play with, either. Often, you can frame your subject with other features in the shot. These features might draw attention to your subject by surrounding it, pointing toward it, or getting brighter around it. You can see in this visualization of the Sun that the rolling waves of plasma are framed by a textured spherical shell, and a transparent window gives us a view to the inside of the Sun. And in this biological visualization, you get the sense that we're about to dive inside one of these spherical molecules because of the way its neighbors frame it. When working with data, it's easy to fall into the trap of looking at the data from a distance. You need to remember, traditional cinematography in the physical world mostly limits cameras to stay near the ground. Viewers feel much more involved in the shot if you get as close as possible to the data. For instance, in this visualization, we skim over the surface of Venus on our way to visit a Venusian volcano. It would have been easy to watch this volcano from a distant helicopter perspective for the whole shot. But by getting close to the surface, we get a sense of a journey, and the small details in the terrain rushing by us give us something visually interesting to look at. Common advice you will hear from animators is that the motion of your camera needs to feel motivated. In other words, you shouldn't move your camera before an event happens in your data, but rather, you should react to it, only moving the camera after you've noticed it happening. This mimics the way documentary camera operators record the real world, and it adds a sense of naturalistic motion to a digital environment. You can take that idea of camera motivation even further. In this sequence, we're surprised by a moon as it enters the frame, heightening the sense of surprise. Of course, there's nothing accidental about passing this closely to Jupiter's moons, but it still seems like it caught the camera operator off guard. You can take that sense of motivation even further by adding some camera shake to really make it feel like a close call. It also helps to immerse the camera in the logic of the scene. You might imagine that your camera is mounted to an aircraft to give it a sense of physicality and momentum. You can see in this shot how the camera gets swept away with the winds of the hurricane. It's not flying against the wind, but with it, and the same goes for this spinning proto-planetary disc. The camera is traveling with the spin of the disc, being pulled in by its gravity. One crucial detail to this technique is that you don't want the camera to move at the same pace as the subject matter, or it will appear that the subject matter is standing still. You can see how the gas and dust in this scene is still traveling faster than the camera. The speed the camera travels at has another significant effect as well. It affects our sense of scale of the subject. The slower the camera moves, the more we imply that the object is distant and large. This environment can feel large if we take our time flying by it, or it can feel small if we fly by it at an unreasonably high speed. But why do we have this association with camera speed and the size of the environment? Can't you just zoom in on something at a faster speed than any camera movement? Zooming in on large objects is technically another way that real world cameras can do a fast push in, but it's actually a visually distinct effect. A camera zooms stretches the field of view, which is how audiences recognize that the camera is not moving. This is actually the source of the famous dolly zoom effect that Alfred Hitchcock made famous. One of the early discoveries about making films with computer graphics was that there were two things missing from the crisp digital images that were almost universally found in photography, motion blur and depth of field. Motion blur is the effect in a photograph where something moving faster than the shutter of the camera leaves a streak across the image. And depth of field is a camera lens effect that blurs foreground and background elements based on the focal distance of the lens. When attempting to imitate photography, you will always want to at least consider using motion blur and depth of field. These two effects can have a negative consequence of blurring out your beautiful data, but the magnitude of these effects can be dialed very carefully. Interestingly, these have artistic implications as well. Motion blur limits how fast you will want to rotate the camera, because it covers the image in blurriness. And we can use depth of field to draw viewers' attention. An extremely shallow depth of field can also help give the sensation that something is extremely small. Another design consideration is what size of screen you're designing for. It's an inconvenient fact that camera design that works on a cell phone will not play well in a movie theater, and vice versa. This might mean you will have to redesign your camera treatment if you want to produce your visualization for multiple screens. A good guideline for designing a camera treatment for a theater space is to pretend you're piloting a spaceship with the whole audience inside it. Doing a fast roll with a theater-sized ship is going to make everyone puke, so I would not recommend that. In smaller screens that don't cover your peripheral vision, fast motion isn't disorienting, and might actually be more engaging. You'll notice that television shows tend to have a lot more shaky handheld camera shots than theatrical movies for this reason. Camera motion can translate to action, even if very little is actually happening in the scene. Another consideration for screen size is how large you frame your subject. For a small mobile device, you will want to fill the screen with the subject, but in a theatrical environment, you can fill more of the screen with the surrounding environment. After all, you'll never be able to depict a life-size human on an eight-inch tablet, but a life-size human is a very small portion of a typical theater screen. Designing a camera path, sometimes called camera choreography, is a process that isn't always similar between Hollywood and scientific visualization. In Hollywood movies, fast cuts among different camera angles keep the audience engaged. But Hollywood filmmakers can usually avoid confusing the fewer sense of location within a scene with recognizable shapes, like actors, props, and even the ground beneath them. Things that are often not available to visualization designers. In visualization design, long, continuous camera paths usually seem to work better to keep the audience oriented. These unbroken paths have other uses as well. They help audiences appreciate dramatic changes of scale, as we change from the size of an organism to the size of an organelle to the size of an atom. Or from the size of a nebula to the size of a planetary system to the size of a planet. When designing a long camera movement, it's still helpful to think of the path having distinct moments, which might be associated with data events. These moments give your camera different still compositions to target as your path progresses. And moving from moment to moment, you'll want to be careful to track the motion of your background as well as your foreground to make sure it's not moving so fast that it steals the focus. Especially in immersive spaces, avoiding cuts helps the audience maintain the illusion that they are physically in a space without being suddenly teleported from one camera location to another. But camera cuts are necessary at some level. Sometimes you need to hide a deficiency in your data, and eventually the shot will come to an end because you've run out of data. A cut is a useful way to get more value out of your data by replaying an evolving simulation from different angles. When designing cuts, cinematographers often try to maintain some visual flow from shot to shot. Perhaps this means keeping the focal object in the same part of the screen, or perhaps it means moving the camera in the same direction from shot to shot. Typically, filmmakers will want to keep familiar objects in the shot between cuts to maintain the viewers' sense of location. But they will make the camera angle noticeably different, usually more than 30 degrees, so it doesn't feel like the camera just jumped forward. However, part of the art of filmmaking is breaking traditional rules once you understand them. Cutting to the same camera angle is called a jump cut, and has entered film language as a way to jump forward in time. Another rule of filmmaking is the 180 degree rule. The idea is to keep all of your camera angles within 180 degrees of each other and to not cross the invisible axis in the scene. This line connects two characters in conversation, two teams on a football field, or points the direction of an object's motion. Cutting to a view on the other side of the line flips the positions of features in the scene, and audiences are easily lost. A Hollywood filmmaker will tell you that when two characters are in conversation, you can give the audience a fairly objective point of view by placing the camera evenly between the characters. >> You can suggest the audience take a side in the discussion by placing the camera over the shoulder of the sympathetic character. >> Or you can fully immerse them in an argument by using a point of view shot. When working with visualizations, we often think of scientific objects as our characters. In this sequence, we see two galaxies heading toward each other in a straight line. This is our axis of motion, that 180 degree line. As the two galaxies interact, we keep the camera centered between them, facing perpendicular to the axis between them. In this scene, we feel like scientists objectively observing a balanced phenomenon. But in this sequence, we are watching an onslaught of destructive magnetic plasma hurtling toward the Earth. The camera choice here is essentially an over-the-shoulder shot, emphasizing with the audience that we should sympathize with Earth. We're no longer being objective in our viewpoint, because we are earthlings and we care about our planet. Actually using computer graphics tools to deliberately create camera choreography can be a technical hurdle, especially if you're not using the right tools. Most software intended for scientific visualization provides simple camera design options, scripting a camera motion along a straight line or perhaps a sequence of straight lines. Some software doesn't acknowledge the concept of a camera at all, though, instead relying on the designer to reposition the objects in the scene in front of a static camera. While these approaches are sufficient for research visualization, they're generally unsatisfactory when it comes to outreach visualization. Animation software tends to have better tools for camera choreography. A designer can create a curved line and then animate the camera along that line. Vertices along the line determine how quickly the camera moves from point to point. And a sense of physical weight and inertia can be created by flattening the ends of your animation curve, which gives the camera a slower start and a more gradual end. The most advanced camera choreography tools actually allow designers to perform the choreography, similar to how a live-action cinematographer uses a film camera. These tools use virtual reality representations of data sets in real-time interactive environments, and they progressively build camera path geometry as time goes by. These paths can then be edited in animation software. This kind of choreography tends to feel the most organic and natural, but it is an art that requires practice. Cinematography dramatically influences the quality of a scientific visualization, but it really elevates a visualization to a cinematic experience. By understanding how Hollywood has defined the language of moving imagery and adapting their techniques for data-driven environments, we can share science with broad audiences in ways they've never engaged with it before. [MUSIC] Fun fact, virtual cinematography was pioneered in virtual reality rooms called caves in the 1990s. These systems required electronic stereo glasses and a magnetically tracked wand. Members of our team at the University of Illinois a patented an early system for interactive data exploration called Virtual Director.
