We Live in the Uncanny Valley Now

Making virtual worlds can help us understand the real one, but they’re not places we can live permanently.

As a parent, I’ve had the chance to read back over many books I loved as a child and have been fascinated to realize just how many of them are about building little worlds. It seems that at the very beginning of our time inhabiting it, the way we seek to understand the world is to build models of it.

Any fiction, of course, is a work of world-building to some degree or another. Science fiction and fantasy are typically the genres where world-building is central to a story’s success, but even the simplest, human stories rely upon a recognizable world to work. If Jess’s small, rural Virginian town didn’t feel real, then Terebithia, the imaginary world he and his friend Leslie create in the woods would have no meaning, no power, and no purpose. Theirs is a story about real children, in the real world, struggling with the common challenges of growing up. But a constructed reality is their sensemaking tool. They understand the world by rebuilding it in part.

Little worlds are created to be partially inhabited — visited, hidden in, for a time — but not permanent homes. Max visits Where the Wild Things Are, but cannot stay. Alice goes through the looking glass to Wonderland but returns home. Wendy outgrows Neverland.

Though we may outgrow the worlds we build as children, we do not outgrow world-building. However, I continually find myself critical of unnecessary world-building. It’s not the utility of it that I question, but the intent. The space between an imaginary world created to better understand the real one and an imaginary world made to take its place is vast. It’s one that, again, so many of our stories attempt to describe. Parables chiding us for our pathological preference for representations over the real are everywhere in culture.

The Matrix, though a weaponized simulation, is ultimately preferred by at least one self-aware character who is willing to return to it — at the cost of a mind-wiping annihilation — rather than suffer a moment more of the real world. In Synechdoche, New York, a theater director constructs a copy of the world so vast and detailed that he loses track of which is which, and his sanity. The Good Place, essentially a pop-culture Trojan Horse for philosophy, proposes that, ultimately, annihilation would be better than a perpetual paradise. It seems that, though we may desire it more than the truth, our psyches cannot tolerate illusion.

Whenever I think of virtual realities, I cannot help but recall the usefully abysmal version depicted by The Lawnmower Man, which Vincent Canby described in his 1992 New York Times review as “very loud but, after a while…induce[ing] a torpor that is quite soothing.” This is an extremely gentle note for a story about a man driven psychotic by immersion in a virtual world. Torpor, it seems to me, is the best-case scenario. Existential dread made endemic by the uncanny’s penetration of our porous minds is the worst.

The Uncanny Valley

Uncanny is a word with a time-worn edge. Originally from the Scottish lexicon, it meant maliciously occult. It was otherworldly, and not in a good way. Today, it is still a word we use to describe a less-than positive experience — an unsettling feeling, say — but it’s typically focused upon our reaction to something rather than that something’s intent. Pareidolia, for instance, can create an uncanny feeling, but it’s not the tree’s fault we see a face staring back at us from its knotted bark. Nor can it be Ameca’s fault that we find its programmed animation so unsettling; it’s just the latest branch of a deep family tree of creepy humanoid robots who simply cannot be malicious because they have no real intent of their own.

But this is all semantics. Uncanniness is simply a feeling we get when something is off enough to turn intended affinity into repulsion. That was the key insight of Masahiro Mori, the robotics expert who coined the phrase uncanny valley. In paper of the same name, Mori wrote:

“An example of a [mathematical] function that does not increase continuously is climbing a mountain…owing to the intervening hills and valleys. I have noticed that, in climbing toward the goal of making robots appear like a human, our affinity for them increases until we come to a valley, which I call the uncanny valley…One might say that the prosthetic hand has achieved a degree of resemblance to the human form, perhaps on par with false teeth. However, once we realize that the hand that looked real at first sight is actually artificial, we experience an eerie sensation. For example, we could be startled during a handshake by its limp boneless grip together with its texture and coldness. When this happens, we lose our sense of affinity, and the hand becomes uncanny. In mathematical terms, this can be represented by a negative value. Therefore, in this case, the appearance of the prosthetic hand is quite humanlike, but the level of affinity is negative, thus placing the hand near the bottom of the valley.”

It is this reactionary experience — the abrupt falling off of the cliff of affinity into the uncanny valley, as it were — that his original phrase, bukimi no tani genshō (uncanny valley), was meant to summarize.

In theory, technology that induces uncanniness can be improved — balancing its synthetic sophistication with our organic expectations — in order to preserve affinity. Sure, every single android we seem determined to create is upsetting and grotesque in its own way, but perhaps one day we’ll make one that can pass for human. One we might actually accept, desire, or love. But I’m not sure that would flatten every one of Mori’s valleys. After all, Mori highlights the role that knowledge plays in creating the uncanny experience. A passing robot could still push a human observer into that valley, once the human understood its true nature.

Nevertheless, the pursuit of fidelity and seamlessness in technology seem motivated to fill every uncanny valley, but I suspect that the opposite is what is actually happening.

The Valley Inexorably Widens

The Uncanny Valley was once a “place” we found ourselves in periodically — pulled in by isolated, unpleasant encounters with the likes of mannequins, Tickle-Me-Elmo, Second Life pandas, and Clippy — but if we follow the notion Mori had in mind, it’s a place we inhabit now almost continually.

Digital technology has provided us with a flexible armature for an infinite variety of immersive uncanniness that not only easily ingratiates itself in our lives, but persuades us of its necessity. “The Internet” is a conceptual structure almost entirely composed of uncanniness made normal by its ubiquity. Memes, GIFs, ambient and ASMR videos, deepfakes, autotune, MMORPGs, generative art, chatbots, digital assistants, and on and on and on comprise an uncanny landscape without boundary. We so frequently create, encounter, and employ false renderings of the real that they have earned standing of their own. And though you may still find any number of the previous list unsettling — I know I do - there are plenty of truly uncanny conventions you use every day that likely no longer register as such.

A perfect example of the way ubiquity covers for uncanniness is the avatar. Avatars are an accidental Stepford-wiving of us all; motivated by presenting our best, we represent ourselves with a fixed expression, permanently taking stage with confidence, optimism, and a million-dollar smile. No one, of course, is like that all the time; few are like that most or even some of the time. And yet, it’s normal to see one another as if we are. It’s normal to repeatedly project the same image over and over again, with every text message, tweet, and status update — smile, Smile!, SMILE, S M I L E — regardless of the reality of our fluctuating day-to-day and hour-to-our affect. Places like Slack and iMessage, where the most recent thoughts from the minds of people I know reach me, present to me a strange memorial of frozen faces, ever more out of sync with the words they accompany.

I sometimes think it would be interesting if our avatars updated in real time, showing us to the world as we really are, right now…until I consider the tyranny of how that might be accomplished. Meanwhile, I will always prefer a neutral expression or something else altogether over a smiling avatar, because if it’s going to be uncanny, I’d rather it be so on the lowest end.

It’s odd to me — uncanny, even — that the culture is moving with such momentum in a different direction. That floating through a cartoon world Pixar would be right to sue over is advertised as the best place to express ourselves is an indication of just that. Horizon, Facebook’s name for their metaverse, carries the implication that virtual reality is the means by which we will evolve and transcend the many limitations of life in tangible terms. Forget that no one visits Horizon without strapping a screen to their face or that there is no Horizon at all without hundreds of thousands of real objects that consume real things from the Earth and while leaving behind real waste. Why worry about that when you’re expressing yourself as the legless, anime-eyed, gummy-torso that you really are? If that isn’t uncanny, I don’t know what is.

We Cannot Live in the Valley

Uncanniness is a diagnosis that fundamentally lacks precision. It’s one of those you-know-it-when-you-see-it things. Most people wouldn’t find avatars uncanny, and though I cannot unsee their uncanniness now that I’ve seen it, I wouldn’t expect others to be persuaded by my point. Uncanniness is subjective because what repulses and what draws affinity is different for everyone.

The way that a thing becomes uncanny is when the artificial stands in for the real. That’s all it takes. In that way, any and every symbol could be potentially uncanny. But there’s a simple reason why we don’t find common symbols like chat bubbles and emoji to be unsettling — they bring intangible ideas into reality, not the other way around. A symbol materializes the immaterial to make it easier to understand one another and communicate. It creates affinity without needing to be the object of it.

A virtual world could do the same thing, if it was designed to do so. But what happens when a virtual world accumulates itself into existence? Sure, the Second Lifes and the Horizons are easy objects of critique — their hubris and beside-the-pointness unmistakeable — but the worlds that comprise our digital culture is an uncanny valley in which we have been permanently pulled. Especially today, “locked-down” as we are, living the remote Zoom-life, so much of what we depend upon to function and connect is so thoroughly uncanny. I cannot help but wonder whether we can design our way out somehow.

You may be reading this from thousands of miles away, and that I can put my words in front of you instantly is truly a modern miracle. And though it may take the modern equivalent of sticks and stones compared with the technology underlying the metaverse, its effect is to create intimacy in this world, not synthesize it in another. At the root of the digital world is something as healthy — as necessary — as the construction of Terebithia. If only we can find a way to come and go.

𓊍