How personalized technology could turn exercise pain into pleasure

Virtual reality (VR) video games that combine screen time with physical exercise are a great way to get fit, but game designers face a major challenge: As with regular exercise, adherence to “exergames” is low, with most users dropping out once they start to feel uncomfortable or bored.

Computer scientists at the University of Bath believe they have found a solution: creating exercise games that use sensors to continuously measure a person’s emotional state while they exercise, then modify the game (for example, by making it easier or harder) to keep the user engaged.

Dr Dominic Potts, lead author of a new study into harnessing advanced sensor technology to motivate exercisers, said: “When it comes to exercise in any form, motivation and adherence to exercise are huge issues. With exergaming, we can solve this problem and maximise an individual’s enjoyment and performance by tailoring the level of challenge to suit the user’s abilities and mood.

“Fully adaptive exercise games will sense a person’s emotions and give them more ‘rewards’ when they’re struggling and more obstacles when they’re ready for a new challenge.”

The results are published in Proceedings of the CHI Conference on Human Factors in Computing Systems.

Game designers have long aspired to develop more personalized exercise games—programs that adapt to a person as they exercise, adjusting to their difficulty and ambitions for the course of the game. However, finding a reliable method to measure changes in a user’s emotional state has proven difficult.

The Bath team has made a breakthrough by using a new range of sensors – which could be integrated into virtual reality headsets and wearable devices such as smartwatches – to track the physical changes an athlete undergoes as they train.

The team’s paper received an honorable mention at the CHI Conference on Human Factors in Computing Systems.

The researchers hope their findings will be adopted by game designers to create immersive programs that can keep a person cycling, running or lifting weights long after they would normally choose to stop.

Background noise

So far, sensors, which can be extremely effective at tracking emotional states when a person is sedentary, have proven unreliable at measuring the emotional landscape of someone engaged in physical activity, making it difficult to recognize whether a person is experiencing happiness, stress or boredom.

“Traditionally, these sensing devices have been integrated into virtual reality headsets to track blinks and pupil dilation, but they are typically very sensitive to physiological and background noise,” Dr. Potts explained.

Unwanted “noise” is generated in two ways: by a person moving unpredictably (as is often the case when exercising) and by the user’s reaction to the virtual environment (VE) in which they are immersed. So, for example, a sensor that shows a person’s pupils dilating while they exercise may reflect changes in brightness in the VE rather than the user’s changing emotional state.

Reliable tracking

In the new study, 72 participants took part in a static bike ride in virtual reality while the Bath scientists used a specific combination of sensors to measure pupil size, facial expressions, heart rate, sweat levels, skin inflammation and electrodermal activity (which measures the skin’s ability to conduct electricity, reflecting stress levels).

Data was collected through the sensors while the runners trained in four distinct virtual environments, where each environment was designed to induce a specific emotion (joy, sadness, stress, and calm). Participants moved through these virtual environments at three different exercise intensities (low, medium, and high).

For each training session, the researchers were able to build an accurate picture of the user’s emotional state, matching the game’s difficulty level and the nature of the VE with the physiological changes the user felt, as detected by their sensors.

From this research, eight guidelines were formulated for creators of virtual reality exercise games, aimed at enhancing users’ emotional engagement. These recommendations include the following:

• Pupil detection sensors must be designed to correct for brightness changes in the virtual environment.
• A user’s pre-existing sweating levels should be taken into account to predict nervous system stress and arousal.
• Sensor data must be cleaned before and during a practice game (i.e., removing signals unrelated to emotional changes) to measure interpersonal differences and allow the game to be tailored to each user.
• Multiple physiological sensors should be used to improve predictions of a user’s emotional state.

Dr Christof Lutteroth, who leads the exercise gaming research as director of the REVEAL research centre and is a co-investigator at CAMERA, both based at the University of Bath, said: “In the long term, our goal is to make virtual reality exercise emotionally intelligent.

“We expect virtual reality physical activities to experience explosive popularity in the coming years. Schoolchildren are already using them as part of their exercise programs and they are also used in rehabilitation and sports science. It is therefore important to focus on creating technology that is emotionally intelligent and adaptable to user differences.”

The University of Bath research team for this study included Dr Dominic Potts, Masters student Zoe Broad, psychology student Tarini Sehgal, Joseph Hartley, Professor Eamonn O’Neill, Dr Crescent Jicol, Dr Christopher Clarke and Dr Christof Lutteroth.