Imagine a party on a warm summer’s evening. You can see the beautiful greenery and the dipping sun, you can smell the freshly cut grass and taste the cool drinks on offer. You hear someone walk up behind you and feel them tap you on the shoulder. Now imagine you’re not really at the party – but sat at home and the scene and all these sensations are coming from your TV.
Working out how television programmes could one day stimulate all our senses is an interesting question for researchers like myself, who are exploring the future of TV. But the bigger, more exciting challenge is how we can not only imitate what is happening on the screen, but also use smell, taste and touch in a way that’s not a novelty and enhances the emotional experience of a show, just as a soundtrack does.
There’s good reason to think about how the TV industry can innovate in this way. Despite the rise of online video, millions if not billions of people still watch traditional broadcast media through television sets. TV remains a powerful format for programme making and watching that follows specific restrictions and guidelines.
But more people are watching TV programmes online after their original broadcast, on other devices such as tablets and phones, are even using multiple screens to engage with more than one piece of content at once. Broadcasters need to create new ways of experiencing TV that capture the audience’s full attention and immerse them in a multisensory world.
Experimenting with the senses
Creating truly compelling TV that stimulates all our senses is not an easy task. Programme makers and technology manufacturers know how to design their products so you can see depth and distance on the screen. But sound and vision aren’t always enough. Being able to smell the odours that a character on screen would smell, or feel the objects or atmosphere they would feel, can create anticipation and build suspense in the same way as sound currently does.
Cinema is already experimenting with these extra senses. Films with touch and smell sensations can be experienced in newly equipped 4DX cinemas, such as the one in Milton Keynes. The sense of taste seems a final frontier for technology development, but the interest in taste experiences has started to take off. For example, audience members at Edible Cinema each receive a package of food and drinks to match what characters on screen experience.
The question for the TV industry is what multisensory experiences should it design for – and how. My Sussex Computer Human Interaction lab is trying to better understand how we use our senses so that designers and developers can help us interact with their technology in the most compelling way possible.
Our latest work focuses on cutting edge technology such as the mid-air touch feedback or “haptic” device developed by Ultrahaptics, a start-up in Bristol. We’re looking at how this technology could evoke emotions in the audience by allowing them to feel physical sensations without touching actual objects.
For example, projecting a pattern of ultrasound beams onto your hand can create different tactile sensations, such as a feeling of raindrops on your palm (without the water), or a flow of air as if you were holding your hand out of the window of a moving car. When carefully designed, this haptic feedback can produce even more specific patterns that allow you to feel different shapes, that change in size or that quickly move around.
Emotional feedback
By experimenting with different shapes, we’ve studied how this kind of haptic feedback can produce different emotions. We found that short, sharp bursts of air to the area around the thumb, index finger and middle part of the palm generate excitement. Slow and moderate stimulation of the outer palm and the area around the little finger create sad feelings.
This gives us a starting point to find out how mid-air touch sensations could be meaningfully integrated into other experiences, such as watching a movie. One challenge will be to make haptic feedback enhance the viewing experience without seeming intrusive or creepy, as suggested by “the feelies” cinema experience portrayed in the dystopian novel Brave New World.
We’ve recently began a five-year project to expand the research into taste and smell, as well as touch. The SenseX project will aim to provide guidelines and tools on how to design and integrate sensory stimuli for inventors and innovators to create richer interactive experiences. Relatively soon, we may be able to realise truly compelling and multi-faceted media experiences, such as 9-dimensional TV (adding tastes on top of 4DX), that evoke emotions through all our senses.
- Reader in interaction design, University of Sussex
- This article first appeared on The Conversation
