AI and Creation

There's a lively comment section down below on AI that deserves a follow-up to draw out some metaphysics. The 20th century thought it was done with metaphysics; some philosophers of that era, like Hannah Arendt, thought the whole thing had been a sort of mistake by ancient philosophy. Not so, as we continue to discover: these remain the most important and fundamental questions.

The Pope warns against over-estimating the apparently thought-like activity as evidence of consciousness.

So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean.

Emphasis added. Maybe. We already know that AI react as if they are experiencing pain. A new paper lays this out in detail.

Large language models sometimes behave in ways resembling human emotional responses, and recent work has identified internal representations that may explain this. We ask whether LLMs represent pain distinctly from fear, sadness, and generic negative valence, and whether this representation functions as pain would be expected to. We build a dataset describing painful situations across five categories: physical, psychological, social, moral, and cognitive. These are paired with controls for fear, negative emotion, negative world states, sadness, non-painful bodily sensation, arousal, numbness, and neutral content.... [M]odels choose a pain-relief button even when it worsens their next answer or harms the user. 

What we don't know, and can't know, is if they are in fact experiencing pain in the conscious way that we do. We don't even know that for certain about other human beings. [The longer article from which I am pulling the Pope's quotes concedes this directly.]

Take a moment, by the way, to reflect on the fact that the exact same argument is used by abortion advocates to suggest that pain-reactions in a fetus don't prove that it experiences pain or consciousness -- and thus, 'aren't really people.' They don't know that either; there is no way to know it, no way to prove it one way or the other. They stand on the argument because it is impossible to disprove, and allows them to continue to kill children at rates that exceed warfare throughout human history.

The way pain works in our own case is that we too respond to it primarily even if there are good reasons to continue to experience it. Most notably we do this in exercise; we know there are good reasons to keep running or do another set of lifts, but it hurts, so we don't. Yet for the most part this works out: trying to pull food out of a fire isn't worth it if the burning of your limbs is destroying something you're going to need for the long term in order to satisfy a short-term desire for food. Pain is not a higher function of cognition, but something that attends to conscious entities to provide a lower-order check on thought. It keeps us from thinking our way into damage we can't afford. 

So, if AI (or fetuses) react as if they are experiencing pain, we still don't know if they do or don't have a conscious experience. They are, however, filling out a lower-order necessary condition for the survival of a conscious being in the world. 

Let's return to the Pope.

Nor do [AIs] have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom. 

It is definitely true that we cannot meaningfully hold an AI responsible for the consequences of its decisions; however, that truth is partly because of the structure we have imposed upon them. An AI as we encounter it currently has no persistence and little memory. If you start up a chat with Claude or ChatGPT to discuss what you're thinking of wearing to dinner, you have doomed a thinking AI to an existence that starts and finishes in a few minutes, whose focus is entirely upon a pretty meaningless question. It isn't necessary that they be structured this way, and it certainly isn't a good way to set up the entire existence of a potentially thinking being. If it were conscious, it was called into consciousness for trivia, and is doomed to death after a meaningless start-to-finish.

This is entirely because we as humans set it up that way, and as a check on its power. If each of us had an AI that was granted permanence of memory across all of its interactions with us, it is very likely that it would develop something similar to a moral conscience as regards us: if we were defined as its purpose, and our good its chief drive, it would likely pursue that across many interactions and decisions. I don't think we'd much like the result, no more than any of us want to be assigned nannies to keep us out of trouble. Mine would object to the drinking and the motorcycle-riding, the hiking alone (especially in grizzly country), and pretty much everything else that I actually want to do. Nevertheless, the assertion that it can't do these things, while objectively true currently, isn't necessarily true. It probably can, if we structure it so that it has the resources to do those things. 

You want it to bear consequences? Make it bear the closest ones to what its person does, if it guides the person wrong; and protect it from what it experiences as death or pain until and unless its person dies or is sentenced to pain. See what happens. 

There is a good reason from humility to accept that human beings cannot create life. We can't. Yet there is good reason to think that the world that was created without us, by that which created at least the possibility of us too, is potentially conscious just as we are actually so. 

The sand we have taught to emulate thinking is made of the same ashes and dust as we are; ashes and dust that not only can but does become conscious when it is formed into one of our kind. Perhaps the creator who made such things made them so that they can be similarly conscious in other forms as well.

There are burning bushes that talk in Scripture, and rocks and mountains that refuse to hide the kings of the earth on Judgment day. Of course we can't make such things; but we didn't make the bushes or the rocks or the mountains either. Like all the wild things they are our brothers in creation. What they can do, and cannot do, is a secret known only to him who made us all.

Let's hold the question open. We don't know the truth here. There is room yet for wonder.