The Rule Book
The rise of 360 degree video has taken the established filmmaking rulebook – crafted over more than a century of experimentation, trial and error – and thrown it to the wind.
Over the last 120 years of film, a rule book has been written, a contract between film-makers and audiences. Negotiated through generations of collaboration and finely honed by the successes and disasters of the ages, this unwritten contract, which we agree on when we set out to make a film, guides us through the whole process. From Michael Bay to David Lynch, there are underlying principles that directors follow. Underlying principles which don’t work with the advent of 360 video.
The DoP’s Challenge
As a DoP, my responsibility is ensure that the directors vision is brought to life in the most appropriate way through the use of lighting, framing and camera movement. 360 video has taken all of these elements and turned them on their heads.
360 Video Technical Specifications:
Due to the technical differences presented by 360 video, the current rule book falls apart in most key areas. With the camera having a 360 degree field of view, traditional framing is now obsolete. The all-seeing camera also makes established lighting techniques impossible as the camera will see all the studio lighting.
Previously in film, the audience was very passive. But with the ability to look around the filmscape it now takes on a much more active role and as such rule book camera framing and position can now create deeply uncomfortable reactions in the viewer. For example, if the camera is low they will feel like they are a dog and that their legs are now 5 feet below the floor.
One new key area has now emerged and that is the “Stitch”. This is the area where different cameras meet and have been ‘Stitched’ together in post to create the 360 environment. The stitch is essential in selling the illusion of 360 video. A bad stitch will result in glitches in the matrix, like double vision in areas, walls not lining up and actors crossing bad stitch lines ending up looking like monsters from ‘The Thing’.
360 Video vs Traditional video
When we’re portraying emotion, traditional video would likely go for a close up so we can see the fear, love, anger or laughter on the actor’s face. The problem with 360 and this trope is that if you’re a viewer it gives the impression that your nose is pressed against the actor’s face, doubly so if viewed on a VR headset, and no one wants to be cheek to cheek with Gary Busey.
So how then do we portray emotion in a 360 environment? Well, a better way to convey sadness is now to have the character further away from the viewer so that they appear small and vulnerable in the scene. This creates a much more visceral response in the viewer and provokes empathy towards the crestfallen actor.
This does however open up a whole new can of immersive worms. With the actor small on the screen and with the audience’s newly gained omniscient state, how do we ensure that viewers are even looking at sad Gary Busey?
For the answer to this, I think we need to take a step back in time to before 360 video – or any kind of video for that matter. In a full circle we arrive back at the storytelling ways of the Renaissance artists. More specifically, I’m going to be referring to Titian. Due to his careful composition, the stories in his paintings can be followed in a designed order. His paintings break down into rough shapes which take advantage of the way our eyes naturally scan across an image. Using his painting ‘The Entombment’ as an example, we can see how its composition dictates how we look at it.
Here we can see how, with the use of the characters’ eyelines combined with the shapes and tone used in the image, we are led around in an oval, always returning to the central character in this piece. The viewer is encouraged to wander round but is always guided back to the deceased Jesus. As humans we are naturally drawn to look where others are looking and this fact has been used to great effect here. As well as an oval composition there is also an overall ‘V’ shape to the composition as can be seen highlighted. This further reinforces the centre point as the focus of this artwork.
360 Video Composition
So how does this help a 360 DoP make sure the audience are looking where they are supposed to be looking? Well, the same psychological rules still apply. Viewers will still follow people’s eye lines and can be led by well crafted compositions, lighting, blocking and framing. Which brings me onto a tricky subject: framing.
360 Degree and Lenses
With 360 video being…well, 360 degrees, it doesn’t leave much scope for lens selection. The concept of picking the lens for the framing you want has become rather redundant. The lens which produces the best ‘stitch’ is now the priority. I think it’s also worth mentioning that, although you want to ensure your viewers are focusing on the action you want them to see, it’s also important to reward people’s curiosity. Taking cues from computer game design, if people are exploring their surroundings then there should be something for them to find. Nothing too distracting from the frame you’ve created but enough so that the viewer doesn’t feel ‘cheated’ in their exploration efforts.
With lens choice out the window it dawns on us that we also can’t use traditional ideas of framing such as long, mid and close.
So as DoPs, what can we do? There are a few options open to us now. We can move the camera closer or further away. This might seem like an obvious point to make but with the change of film language that 360 video brings and the fact that moving the camera towards something also now moves the rear view away from something else, there are knock on effects to take into consideration. One aspect I’m excited about is how we can use the environment to create frames within frames. This is nothing new but I think it’s a framing technique that will really come into its own in the 360 era.
We can see the use of this technique in this screen grab from ‘The Grand Budapest Hotel’. If we imagine that this is happening in a 360 environment, we would still have the same mid 2 shot framing of the Lobby Boy and Hotel manager. It’s also worth noting the tonal difference in the frame that draws our attention to the centre. The frame is quite busy overall, but not distracting due to the use of colour and tone, and our eyes are drawn to where the story is happening. This is a good example of how framing and composition can be used in 360 film-making to overcome previously mentioned framing obstacles. It’s also a good indicator of how important lighting is when we are shooting 360 video, which leads me onto the problems with lighting in a 360 environment.
The obvious difficulty with lighting a scene with regards to 360 video is that the camera can see everything around it. It doesn’t take a huge leap of imagination to see why this is going to be a substantial problem with traditional lighting techniques. Lights now need to be hidden from the view of the camera completely, with the option of framing them out now substantially diminished. There are various ways this can be done. Light could be spilling in from doorways and windows in sets. Locations could be designed or actors positioned in such a way that they can catch light whilst keeping the source hidden from the all-seeing camera. Lighting design needs to adapt to having the majority of studio lights only able to back and side light.
The need arises for closer collaboration between the art and lighting departments, for example “Can I have that wall white so I can use it as a reflector?” 360 video also opens up landscape for a very natural lighting approach with scenes being fully lit by practical lighting. We may need a front key light for an actor facing the camera in a scene shot in a house. Using an octodome isn’t an option now. “Why does this character have a large studio light in the middle of his house?” the audience will surely ask. One option would be to have a light mounted where you would expect to see a ceiling light with a china lantern. It wouldn’t look out of place if the viewer looked up but it might pose some problems with regards to hot spots, so the whole scene would have to be lit and exposed to control for that. This brings me to my final point.
Cameras and Post Production
The arrival of 360 film-making is an exciting step forward in visual technology but it is suffering from some birthing pains. Proving itself to be more than just a gimmick is going to be a challenge. For me, one of the major hurdles is the cameras themselves. The current generation don’t provide the control I want to have over a camera, with them being mostly in the vein of GoPros with auto settings. There is also a divide in how cameras deal with ‘stitching’ video together into a 360 image.
The current most popular set up is a franken-rig of 6 or more GoPros which does a job but with 6 or more cameras to stitch together the post production time is drastically increased and there are problems with distortions when action crosses a stitch line. There are other options such as Kodak’s sp360 4k cameras which work with 2 cameras back to back. These suffer from the same problems of automation that affect GoPros but with only 2 cameras there are only 2 stitch lines to worry about crossing and it opens up creative opportunities to shoot one camera at one location and one camera at a separate location, and then stitch the 2 separate videos together. This opens up the option of shooting against an infinity curve or a full set split in two lit with traditional lighting, as you can shoot 2 separate 180 videos and stitch those together. But the images these cameras produce isn’t very high quality and certainly isn’t acceptable for professional use.
There are companies using 6 Reds rigged together to overcome the lack of control over ISO, shutter speed, aperture etc. This provides very high quality video but it’s a very expensive, inelegant and I suspect, temporary solution. The other option is to use cameras that stitch ‘in camera’ such as the Nokia OZO. This solves the post production headache but then also removes a lot of creative opportunities that the dual or more camera rigs can offer.
Summary – Where does this leave us?
Where does all this leave us in our quest to go where no video has gone before?
Practically speaking, DoP-ing 360 is challenging. Lighting a scene in which the camera can see everything is a bit of a problem. Lights now need to be completely out of sight, relegated to lighting backgrounds, moodily spilling out from door frames or peering in through windows. Practical lights have now taken centre stage when it comes to lighting the scene. More thought now needs to go into dressing a set so light sources can be positioned where they need to be in a way that makes sense in the scene. This means that interior design and set design take a step closer together. We need to take a trip down to the galleries and look at some Titians and Michelangelos and figure out how to block, frame and direct attention in a 360 environment. Finally, we need to wait and see what professional camera options become available, and we need to prove that 360 video is more than just a gimmick, it is a viable film format moving forward.
This is a once in a generation opportunity to write a whole new film rule book, and it’s an exciting ‘Mad Max’ dash to the vanguard of what is now possible.
About Tom Watts
Tom Watts, Director of Photography
Tom Watts is an artist and designer in the film world. He holds a Film Production degree from UWE but has always known that the real education is found in the work. With a background across all elements of the production process Tom looks for new challenges where he can experiment and innovate. Clients include SONY Music, where he is currently collaborating on 360 productions as well as more conventional work with Red Bull, Discovery Channel and of course Camberwell Studios, where he has DOP’d for a wide range of our clients.