Monitoring the Body to Spy on the Brain - Part 1: Why?

Remember when we were talking about brain waves, and I started talking about the practical applications of using EEG to tell how engaged people might be? Well, that's still true, but it's not the only method though which we can attempt to track mental states by monitoring physiological changes, nor is observation the only possible application of these technologies.

Biofeedback, by the way, is the use of monitoring equipment to monitor certain physiological activities. That information can presented back to the person being monitored so that they eventually can learn to consciously control the function in question. Biofeedback can be used to  (e.g. Adding physiological data to the in-game performance data that a video game AI might use to control game behavior) or as an  (e.g. Using biofeedback devices in lieu of traditional input devices such as keyboards or game controllers).

For the purposes of today's discussion, however, we will focus on the application of physiological monitoring methods for the purposes of user testing, specifically video game user testing.


Why would we want to track physiological data?
Let's say you're a developer and have just finished the alpha version of your game. During the user-testing stage, you would want to be able to collect information about the players' abilities to actually complete the level,  as well as get some idea of how engaged they were while they were playing (and whether or not they were actually having any fun).

That first part is fairly straightforward. You can capture all the information you need about in-game activity just by recording the game and tracking in-game metrics like the amount of time taken to complete the level and the number and nature of any bugs encountered during those trial runs. It's the latter bit where things get tricky.

Traditional Methods
Traditionally, if you wanted to get an idea of what people thought about something you made, your options for getting that information were primarily based on asking questions, writing down the answers you received, and praying your testers were competent, honest humans with good memories and decent language skills.

Examples include:

• Surveys - Allow you to ask a lot of questions about a lot of different things, but it's all after-the-fact. People may forget to bring up things they noticed during gameplay, and you have to deal with things like confirmation and recall biases, which can mess up the accuracy of your data.
• Direct observation - Which is great for seeing how people interact with the game, but you're physically around, and that's going to affect how people play, and what they do.
• Verbal reports -  Can be done before, during, and after gameplay; and usually take the form of interviews (either structured or unstructured), or by asking people to narrate what they're doing as they play. Again, though, you may end up with people leaving out valuable information.

All of these are fantastic methods of gathering information about gameplay and user experience, I'm not going to debate that, but I will say that traditional user-testing methods suffer from two major limitations.

1. They are not objective. Any time you ask someone a question, the answer you receive will be based entirely upon that person's subjective experience, and as such, cannot be reliably compared to that of any other person. What's more, that same answer may be corrupted by some kind of cognitive bias, or it may be a complete fabrication.

2. They are not quantifiable. Yes, it is possible to perform statistical analysis on survey results, and yes, you could totally say "9 out of 10 players said they had fun", but there is no way a talk-aloud trial, for example, could tell you a player was definitively more aroused at point A than point B, nor would direct observation be able to tell you if that one guy who kept turning right for an hour and sending himself in circles was getting more or less agitated over time.

To get this information, we have to turn to biology, and this is where the biometrics come in.

Integrating Physiological Data Into User Testing Experiments
We'll get into specific techniques and devices in part two, but first, some notes on experimental design. When talking about video game user testing, most of the experiments to date (of which I'm aware, anyway) have fallen into two categories.

1. Experiments in which physiological data is collected in addition to traditional data.
In these experiments, physiological data is treated as an additional source of information about subjects' mental states. Ideally, this means that at the end of your experiment, you will be able to say things like "The majority of players showed signs of heightened arousal during gameplay. This is further supported by information collected during post-gameplay interviews. As such, we can confidently assume that this particular level is engaging and fun for users." (e.g. )

2. Experiments in which physiological data is used to shape traditional data-collection.
In contrast to the "Traditional plus physiological data" experiments described above, these "shaping" experiments are a three-phase process in which data is collected, mapped to a timeline or recording of the users' test run, and then used to guide post-testing data collecting by identifying critical moments/events during gameplay. E.g. "Your heart rate increased dramatically at this point. Do you remember what you were thinking at that time?"
(See also: )

In either case, physiological data is valuable for user testing in that it can serve not only as an additional source of information about users' psychological states, but also because these same records can be used to guide and modify traditional methods so as to produce data that is both more reliable, and more comprehensive, than traditional methods alone.


[Continued in Part 2]