Book Review: Klara and the Sun

We humans are in the center of the three intersecting circles: the intelligent, the conscious, the loving. Is anyone else there with us? Could there be? How would we know?

In Remains of the Day, Never Let Me Go, and Klara and the Sun, Kazuo Ishiguro writes from the perspective of conscious, intelligent beings whose ability to love is not understood or acknowledged. In the Remains of the Day, a butler devoted to service; in Never Let Me Go, a genetic clone; and in Klara and the Sun, Klara, a robot. Klara, like all Artificial Friends, is built to serve as a social companion for children in a semi-dystopian future world. (And as you have just noted, there will be a few minor spoilers in what follows.)

Klara and the Sun is written from Klara’s point of view in the first person. This is no small thing, for we would seem to have the privilege of understanding Klara’s qualia - the feeling of experience that is available to each of us in any given moment, which none of us can never really completely describe in words, not just to anyone else but even to ourselves. It may seem that viewing Klara from the inside would settle the question of who she is, but Ishiguro does not let us off so easily. For Klara, like us, is not fully consciously aware of all of what is inside of her, what drives her own feelings and responses. This mystery is central to the Klara and the Sun, for without a solution, it is difficult to answer certain questions, for example “does Klara feel”, or “can Klara love”.

The novel’s tone is characteristic of Ishiguro, that is, a bittersweet sadness mixed with the warmth of hope. This is appropriate, given the questions regarding Klara and those that surround her, and the backdrop of the story itself. The world of Klara and the Sun is a plausible forward projection of what is already happening, which makes the passing descriptions of the world beyond Klara foreboding if not terrifying. Conditions that gave rise to our modern selves, namely our environment and our social order, seem precarious these days. Ishiguro provides no comfort in this regard.

Klara and the Sun is about AI and robots in the same sense that Moby Dick is about whales, and therefore seems likely to be under-appreciated by left-brain dominant rationalists. Bill Gates’ awful review of the book is the platonic ideal of not-getting-it: “In a world filled with stories about killer machines, it was refreshing to read about a future where robots make our lives better—even if they complicate things along the way.” He misses the point so wildly and reductively that I hope for humanity's sake that an intern wrote the review, but I fear not. To talk about Klara and the Sun without talking about feeling and longing is pointless. Even when viewed strictly from the prism of AI, Gates’ view is simplistic, though to be a teensy bit fair we must forgive him for having commissioned his review prior to the recent explosion of large language models and their descendants.

Ishiguro is far more perceptive in aspects of AI such as alignment, the integration of signals, learning, and emergent behavior, than many (most?) practitioners in the field. We would do well to take the potential societal implications of artificial general intelligence as seriously as Ishiguro, rather than the cartoonish takes (both positive and negative) that one frequently encounters, and which I will not cite.

Klara and the Sun is not an arty treatise on the problem of hard consciousness, i.e. what gives rise to subjective experience. While this question arises, Ishiguro is more interested in Klara’s experience itself, and whether her kind of experienced consciousness can serve as a substitute for humanity’s. The book is a weird cousin to Thomas Nagel’s “What does it feel like to be a bat”. As Wittgenstein said to me one morning over coffee: “Nicht wie die Welt ist, ist das Mystische, sondern dass sie ist”: it is not how but that the world exists that is mystical. Existence itself is the mystery, not its mechanics.

Klara and the Sun deserves a close reading because it, with care and subtlety, raises so many interesting questions. The questions are implicit and there is no answer key at the back.

Peter Godfrey Smith in “Other Minds” asserted “perhaps the best way to understand what it is to be human is to study something that is not.” Klara is not human. What do we learn by observing her? 

Some neuroscientists say we perpetually live in a remembered present. Billions of sense impressions interpreted by trillions of neurons enmeshed in countless reentrant clusters, firing in patterns shaped from past experience, in changing unique combinations, an ever-evolving sequence of qualia. Klara possesses this kind of consciousness. But is that like our kind of consciousness? How would we know?

Without spoiling too much, a key passage in the book describes Klara’s observation of glass sheets in a barn as the sun sets, serving as a metaphor for Klara’s consciousness:

"I stared at the glass sheets. The Sun's reflection, though still an intense orange, was no longer blinding and as I studied more carefully the Sun's face framed within the outermost rectangle, I began to appreciate that I wasn't looking at a single picture; that in fact there existed a different version of the Sun's face on each of the glass surfaces, and what I might at first taken for a unified image was in fact seven separate ones superimposed one over the other as my gaze penetrated from the first sheet through to the last. […] In any case, whatever the nature of the images on each glass sheet, as I looked at them collectively, the effect was of a single face, but with a variety of outlines and emotions." 

Can consciousness arise from the integration of lower level stimuli, or does it require a higher level appreciation of the whole, as Iain McGilchrist in the Master and His Emissary would have it? Klara’s reality consists of superimposed frames, and her consciousness, in order to make sense of the world, unifies these into a single scene. This is how sense-making works for her. It is at once a limitation and an amplification, because while information is discarded in the unification, depth and nuance emerges. Klara unifies, linearizes, and fits the world presented to her into an image that conforms to past training and understanding. She is a machine for making sense. Is this what happens for us?

More questions relate to what lies hidden from us, even to the aware. Klara says "the more I observe, the more feelings become available to me." This quality seems to make her different from Artificial Friends (AFs). We observe her yelping out in surprise, experiencing fear, curiosity, and happiness, and even being deceptive. As the narrator, we observe Klara reporting what she describes as feelings, as she experiences them. Is she truly reporting what she is feeling? Are we getting a report of her interior, her qualia? Is she a reliable narrator of her experience? Am I? Can I truly say that I am when I am surprised at my own reactions?

One of the key questions is whether we ever fully know someone else. In one dialogue, one character asks whether the heart can ever be fully explored and known. The image of walking from room to room to room within the heart, is a fairly direct Biblical reference (John 14:2 about “my Father’s house” having many rooms). A materialist account of consciousness would answer this question in the affirmative. Are the materialists right?

In an age of almost everyday change in the field of AI, it is not at all clear to me where we are headed. It is clear to me, however, that these changes are so fast and so profound that it will take some time for society to adjust, if ever we do at all. I believe the patient, observing style of Ishiguro, and indeed Klara herself, is a useful guide for how to act in such a situation.