The valley has enjoyed widespread discussion in relation to the appearance of CGI humans over the past few years, and while thinking about it recently, the very worst moments of my social life flashed in front of me interlocked with thoughts of some of the best game AI. I now think that the uncanny valley applies to behavior too.

 

There's a small minority of people who are consistently strange in particular ways. You've probably met a few of them. Human though they are, interaction with them doesn't follow the usual dance of eye contact, facial expressions, intonations, gestures, conversational beats, and so forth. For most, it can be disconcerting to interact with such people. Often, it's not their fault, but even so the most extreme of them can seem spooky, and are sometimes half jokingly referred to as monstrous or robotic.

 

I don't mean to pick on them as a group; nearly all of us dip into such behavior sometimes, perhaps when we're upset, out of sorts, or drunk. Relative and variable as our social skills are, AI is nowhere near such a sophisticated level of interactive ability. It is, however, robotic. Monstrous and sometimes unintentionally comedic; the intersection of broken AI and spooky people is coming.

 

The problem is compounded by the fact that there's no way to abstract behavior or make it "cute". Cuteness is visual, so by rendering it as a cartoon even the repellent appearance of an ichor-dripping elder god can be offset. In a similar way, by its visual characteristics a Tickle Me Elmo doll pushes a lot of our "cute" buttons. However, when it's set on fire and continues to giggle, kick it's feet and shout "Stop! Stop! It tickles!" while it burns into a puddle of fuming goo, it seems horrific, profane and hilarious by turns.

 

That’s programmed behavior pushed out of context, and the highly specialized fragments of AI currently integrated into video games easily break in the same way when they stray from their intended stages.

 

Strange or sick behavior can't be abstracted into a cuter, more appealing version of itself unless it's made burlesque, naive, or consequence free, and of course this would have drastic narrative effects. While a story can be told through any number of sensory aesthetics, behavior itself works through time, its meaning often independent of representation. That's extremely important for interactive media.

 

There are lots of things across all media that can already fool us. The crucial question, though, is how well do they do it? Distance and brevity obscure all manner of flaws, but at some point in a game, the player can always get closer or look for longer.

 

This applies to absolutely every aspect of simulation, but the aspects centered on other humans are critical. We're a very social species, and as a result large amounts of our cognitive resources are thrown into the assessment of other human beings. For instance, we show extraordinary specialization in recognizing, processing and categorizing the faces of other humans. We're acutely aware of whether or not other people are looking at us. We spend every second of interaction inferring the emotional state, values, and likely actions of others.

 

Of all the sensory data we deal with, other people are among the most relevant to our existence, so of course we have some highly specialized capacities to deal with it. Speech, movement, body language, behavior, and consistency of actions are all things we're well accustomed to.

 

That means people are much more difficult to simulate than rocks and trees, not just because of relative complexity, but because we're more wired to scrutinize our fellow humans. In film and real-time rendering alike, the plastic sheen of 90's CGI has given way to environments my unconscious mind doesn't balk at and just accepts even if not quite photoreal, but simulated people continue to pop out of them as fake.

 

Whether or not something is "realistic" is largely a red herring. The more important test is whether or not it's convincing, and I suspect behavior will prove to be a much bigger challenge than appearance.

 

Simulated appearance can be constructed from various elements that we are presently mastering. behavior is a complex, dynamic, context sensitive system that, in addition to dealing with immediate situations, can also operate informed by elaborate historical contexts and long term aims. Where actions and physicality are based on syntax, the behaviors underlying the vast scope of human actions, along with the limited repertoires imparted to AI, are often about meaning and have a rich undercurrent of semantic relations.

 

Real human behavior, for the most part, seamlessly elicits my empathy, and also tells me that, in turn, others understand and empathise with me. It also tends to demonstrate consistency, and at some point can generally be expected to explain any inconsistencies.

 

At best, such dynamics exist in a fragmented fashion if at all in game AI, which generally follows a very predictable cycle no matter how good it is: When it's new it may surprise me a few times with various tricks, and will tend to elicit empathy too, but every time a human seeming art asset or piece of behavior is instanced or recurs, my empathy diminishes. This continues until eventually I can let my Id go to town on NPCs without feeling bad. The greater the degree to which AI repeats itself, the more likely this result is.

 

Beyond patchy AI, the emotional engagement of a game is in the motivation I have to achieve goals, which are nothing but syntax. Games can and do rise above this. At present, there seem to be two ways in which they can use NPC behavior to drive emotionally engaging narrative and social interaction.

 

 

 

The first is traditional,