Interested in discussing this article in your classroom, parish, reading group, or Commonweal Local Community? Click here for a free discussion guide.

As an anesthesiologist, I once took care of a pregnant woman on the obstetrics unit who was very particular about what she wanted. With her husband at her side, she demanded that I get rid of most but not all of her contraction pain, as she said she needed to feel some pain to imagine herself a mother giving birth. At the same time, she didn’t want to feel the urinary catheter in her bladder. She also wanted to be able to move her legs, as the feeling of being paralyzed scared her. Other demands included no nausea and no shakes.

I told her that we stood a good chance of meeting all her demands, thinking she would forget half of them once the epidural was in and the television was on. Still, by suggesting that the perfect epidural was possible, both to her and her husband, I had made myself more vulnerable and doubled my exposure to potential dissatisfaction. Yet I also knew that for many patients, a world without dreams, without illusions—yes, without lies—is a terribly empty world. Patients must have hope.

Artificial intelligence (AI) cannot practice this fine art of lying to patients, because if ever given permission to lie, how would it know when to stop? To boost a patient’s spirit, and by extension, a patient’s health and well-being, doctors must sometimes stretch the truth a bit. Fortunately for patients, doctors are burdened with moral scruples, bound by notions of social responsibility, and stricken with self-judgment the moment they tell a fib, which keeps such behavior from getting out of hand. Because AI lacks these qualities, it cannot be given permission to lie. It is part of why AI cannot totally replace doctors.

The epidural was surprisingly hard to place. Her husband constantly pecked at me during the procedure, asking, “Why is this taking so long?” Afterward, I sheepishly dosed the catheter with local anesthetic. The woman felt better, but twenty minutes later she still complained of pain. I reminded her that she had wanted to feel some pain. “Yes, but not that much pain,” she insisted. I gave her more anesthetic, started a continuous drug infusion through the epidural, and left the room.

Over the next two hours I was called back every twenty minutes. Each time, something wasn’t quite right. The woman’s complaints had all sorts of sides to them: there was this, that, and the other, but when I took this and that into consideration, and changed the anesthetic dose, this became that and that became the other. Each time I was left guessing what to do, trading pain relief for muscle strength, her need to feel some pain for the risk of her feeling too much.

At this stage of the case, AI would have been even less successful. For while AI can calculate logically, it cannot empathize with a living being. It cannot grasp the arcane moment when “good” pain becomes “bad” pain, when the concern with urinary catheter pain trumps muscle weakness, or what the proper order of concerns should even be—especially when that order changes every fifteen minutes, as it did with my patient, who also happened to hate the television show she was watching, further lowering her threshold for dissatisfaction. True, AI can straightforwardly tell patients, “Medicine cannot meet your demands,” but given how often Americans complain that their doctors behave too much like scientists and lack the “common touch,” a health-care system organized along the lines of pure reason or “pure” AI would incite even more popular fury. 

Around the third hour, the nurse called to say my patient was in pain again. I walked toward the room tired and peevish. This time, the woman was screaming. Only the pain of her uterine contractions silenced her. 

I stared at the continuous infusion pump. The machine seemed to be working well, with no alarms ringing and with anesthesia flowing through the tubing. I felt instinctively that the epidural was fine, and had good reasons to do so: my patient’s hypersensitivity and unreasonable demands, along with the machine’s solid numbers. I reassured the couple and gave the woman an extra dose of anesthetic through the machine tubing, then left.

Ten minutes later, the nurse called me to say the patient’s pillow was wet. The epidural and machine tubing had become disconnected, with local anesthetic leaking onto the pillow all this time. I returned to the room, where I had to force myself to look the couple in the face. Having missed a problem so obvious, I knew the husband had the right to curse me all he liked. Under reproachful glances, I reconnected the epidural and quickly left. 

My mistake illustrates the supreme danger with AI in medicine, beyond its obvious deficiencies: that it cannot lie, empathize with people, understand or convey nuance, or even really think. In regard to this last deficiency, large language models generate text by focusing on a word, studying how that word is used in other contexts, and making a probabilistic calculation of what word should follow. By doing so, they can stitch together whole paragraphs of impressive language. But this is not thinking, which is why some scholars have called AI a “stochastic parrot,” generating convincing language it doesn’t really understand. It is why AI can “hallucinate” or nonsensically say to a patient, “Take two aspirin and ten brownies.” But even a stupid error such as this is not the supreme danger, which lies, rather, in people’s reactions to AI, especially doctors’ reactions. 

During my epidural case, I looked at the machine’s numbers, which suggested the epidural was working fine. I trusted the machine implicitly, certainly more than the complaints of my very human patient. As a consequence, my eyes were too drowsy to open wide and look around; my senses too numbed to be stirred by curiosity. My investigative spirit was paralyzed by the surety of the machine; I had no desire to know anything more. Even stranger, I knew that a machine pump and an epidural can get disconnected, and yet this knowledge was mysteriously forgotten. 

My deference to the machine would have been worse had AI been involved. For AI touches on the deepest longing of professionals for the “foolproof” machine. All professionals, no matter their field, secretly desire a career without mistakes, without tough decisions, even without responsibility. The desire slumbers in their souls, like a dark memory out of Paradise. They look to machines to accomplish this goal, thinking machines know best and will not deceive them or let them down. Because AI, more than any other machine, promises the fulfillment of this desire, it risks turning professionals into its supplicants. 

I use the metaphors “soul” and “Paradise” here on purpose, as religion has something important to say about this problem. Perhaps more than any other discipline, religion recognizes humanity’s inherent defects, which include people’s tendency to trust what should not be trusted. When, in January, the Catholic Church entered the debate over the risks posed by AI with a doctrinal note titled Antiqua et nova, I looked forward to the Church bringing its age-old wisdom to bear. Sadly, it disappointed me.

Many of the dangers Antiqua et nova addresses had already been described elsewhere, including misinformation, the lack of human oversight, and the threat of job replacement. The document’s highlight is its emphatic distinction between AI and human intelligence. God-given human intelligence, the document argues, goes beyond calculation to include “loving, choosing, and desiring”—things AI cannot do. Such emphasis primes the reader to think AI’s risk to society begins with its intellectual deficiency relative to humans, rather than in any human deficiency.

By pursuing this line of argument, the document rattles old chains and fits squarely within the tradition of Church reactions to other scientific and technological challenges. In each case, the goal has been to preserve the special position of the human being in the face of some new discovery. 

In 1616, the Church rebuked Galileo’s notion of heliocentrism, which said the earth travels around the sun rather than the other way around. Heliocentrism threatened the sanctity of Scripture, and, derivatively, of earth and humanity. By condemning Galileo, the Church hoped to reinforce humanity’s special position. 

In the nineteenth century, Darwin’s theory of evolution threatened humanity’s special position by arguing that humans and apes share a common ancestor. Unlike in the Galileo affair, this time the Church wisely resisted direct attack; instead, speaking more elliptically, it forbade Christians from defending scientific conclusions that were contrary to doctrines of faith. Over time, the Church grew more accepting of Darwin’s theory as a scientific explanation for the development of life, but it nevertheless insisted that God had infused humans with immortal souls that did not evolve. Again, the goal was to preserve the distinction between humanity and other beings.

Recently, the Church joined neuroscience’s mind-brain debate. It said the human mind is more than just an extension of the brain, as the mind arises from a more profound and complex union of physical body and immortal spiritual soul. Again, the message was that human beings are special. 

Antiqua et nova follows this pattern in the face of the AI juggernaut, defending human intelligence and calling for a “renewed appreciation of all that is human.” For example, human beings, it says, have a special attitude toward work; for them, it is a source of dignity and a necessary outlet for their creative impulses. By taking over most of the interesting tasks and leaving people with the repetitive, unthinking ones, AI risks turning human beings into little more than livestock—useful only for drudgery—and potentially igniting social rebellion. 

The document also identifies the risk posed by the “technocratic paradigm,” which sacrifices human dignity and fraternity in the name of efficiency. As it seeks to maximize economic output and minimize costs, AI ignores human relationships and the common good. The “non-quantifiable aspects of life are set aside and then forgotten, or even deemed irrelevant because they cannot be computed in formal terms,” the document declares. 

Antiqua et nova ventures into health care with the same modus operandi. AI threatens human dignity, it declares, by replacing the relationship between patients and health-care providers, and “leaving patients to interact with a machine rather than a human being.” It “risks worsening the loneliness that often accompanies illness,” exacerbating the ills of a culture where human beings are “no longer seen as a paramount value to be cared for and respected.” AI may also aggravate “real forms of social inequality”; determine who gets treatment based on impersonal “metrics of efficiency”; and reinforce a “medicine for the rich” model. All these warnings flow from the section’s basic premise, which is that human beings occupy a special position that must be respected and that is threatened by AI-run health care.

I agree. But I do not agree that these problems constitute AI’s greatest threat to patients. That threat originates not in AI but in ourselves, in our eternal impulse to hero worship. In the future, when doctors use AI, it will not be as masters attached to a particular tool; rather, the tool will attach itself to them with a thousand tiny claws, transforming them. Doctors will hear AI pronounce on a case and many of them will accept that pronouncement unquestioningly and follow AI’s lead. Already predisposed to accept a machine’s word as more accurate and credible than that of an error-prone human being, they will imagine AI pronouncing not just in its capacity as a machine, but as a kind of emissary from the world of science. They will look upon AI as the source of all medical knowledge, the expounder of medicine’s algorithms, and the arbiter of what constitutes medicine’s best practices. Doctors already tend to “treat the number” that appears on a machine’s screen rather than scrutinize matters with their own senses, as I did during my epidural case. But doctors will listen to AI with even greater deference. Worse, their respect for AI will be tempered with fear—if they deviate from AI’s recommendations and their case goes badly, the plaintiff’s attorney at their malpractice trial will argue, “AI, which is a thousand times smarter than you, told you what to do. So why did you act differently?”

Even doctors with the courage of their convictions, who might be tempted to use their own judgment over conflicting AI recommendations, will inevitably have second thoughts: “On which side lies the truth, or, to speak more carefully, the greatest probability?” While pondering this point, their knees will grow wobbly; face to face with the decisions of the AI wonder-machine, they will demure. Even the bravest among them will lose faith in themselves. 

It is by this route that the dangers associated with AI’s inability to “think” will penetrate health care. Lying at the root of the problem is not a defect of AI exactly, but a human defect that causes doctors to overlook AI’s limitations. 

Here is a more specific example of the trouble likely to occur. AI works with broad concepts, algorithms, and formulas that are abstracted from thousands of patient cases, which doctors then apply to individual cases. While doing so, doctors are supposed to use their judgment as well, which includes not just knowledge of concepts and algorithms but also a “feel” for the particular situation. This brings into play their instincts, intuition, suspicions, and presentiment—all qualities that AI lacks. My epidural case exemplified the kind of situation that demands judgment (although my judgment was a bit off). Managing elderly patients, especially, requires judgment. The broad concepts and algorithms derived from thousands of cases that illustrate physiology and pharmacology in ideal conditions rarely apply so nicely to older patients, whose impaired status and sensitivity to medicines leave little room for error. In addition, elderly patients often have multiple medical problems, meaning that treatment involves tradeoffs. Prescribing medicine to solve one problem may exacerbate another problem. Doctors mesmerized by AI will be less likely to use their judgment in these individual cases, relying instead on machines that lack judgment. Patients will get hurt.

In a recent essay in these pages, Georgetown University professor Thomas Banchoff criticized Antiqua et nova for paying too little attention to AI’s ability to simulate human intelligence. He correctly argued that the human tendency to anthropomorphize the nonhuman—for example, in the case of pets—may stretch to include AI companions, which people will bond with, leading to a new world of pseudo-intimacy with far-reaching complications. Still, his concern differs from mine. He fears people will look upon AI machines as human and bond with them. I fear doctors will look upon AI machines as gods and defer to them.

Much of medical care is straightforward. While working as an anesthesiologist, many of my decisions were made on autopilot. But decisive moments do come, when judgment and courage are required, and it is then that a doctor’s character is disclosed by his or her behavior. Crises bring to light hidden capacities, or, conversely, their absence. When the temperature is moderate, those overshadowed qualities lurk beyond the range of measurement, and do not even seem particularly important, but when it heats up, as during an emergency in the operating room, those qualities are made suddenly and unmistakably apparent.

Does the Church have an intellectual tradition with which to investigate the mystery of the human character as it relates to AI? Attention to St. Augustine’s writing may help the Church shift its focus away from humanity’s special position and toward humanity’s unique vulnerabilities in the face of AI.

Augustine probed the human will and our character defects with an intuitive rather than a systematic genius; his method was psychological empiricism more than rational deduction. People, Augustine observed, may know what is right, but they are often too weak-willed to act on that knowledge. Instead, they come up with all sorts of excuses for why doing the right thing is impossible. For example, we might project false powers onto another human being, or onto a nonhuman being, such as fate, which we feel obligated to listen to and powerless to resist. 

In Against the Academics, Augustine tangled with skeptics who touted the most surefire excuse for doubting oneself and obeying some higher authority. We cannot be certain of anything, the skeptics declared. Our senses are untrustworthy. Do not trust your judgment. Augustine admitted that our senses do sometimes betray us. But at least we can be sure that the world appears to us as we perceive it, he said in reply. In my epidural case, for example, I might have been wrong in judging my patient’s level of pain, but not that my patient appeared to be in pain. In this regard, my senses were indeed trustworthy. They reported things to me just as they should, even as my judgment strayed. Yet that is the point, Augustine explained. The judgment of truth is to be found in the mind and intellect. The problem for doctors is that, cowed by AI, they will lack the will to believe in themselves and their own judgment. 

The way my epidural case ended illustrates this point. After several hours, the epidural’s effect had dwindled, despite my giving additional anesthetic. I had thought about replacing it, but I didn’t want to announce to the couple that my epidural, which I had struggled to place, was a failure. Besides, I thought, if I replaced the epidural, I might have the same problem all over again—assuming I even got another one in. More importantly, the machine hooked up to the epidural said that anesthetic was still flowing easily through the tubing. There was no blockage. In the end, the machine’s numbers swayed me. 

But now my patient needed an urgent cesarean section. I was in a bind. The patient’s substandard epidural might fail during the operation, in which case I would have to induce general anesthesia mid-surgery, with all its attendant risks, including the fact that the patient was severely obese and had a difficult airway. Yet if I removed the epidural, I might not have enough time to replace it with a spinal anesthetic, given the case’s now-urgent nature and the patient’s tricky back anatomy. I would then have no epidural and be forced to use general anesthesia at the outset. 

I felt as if all my sins had found me out. Joined to my feeling of self-reproach was an indefinite reluctance to meet the horrid logic of the situation, especially when the machine kept telling me the epidural was working fine. If I replaced the epidural with a spinal, I would be a guilty man doing a guilty deed. The patient’s husband would say to me, “If the epidural wasn’t working, then why didn’t you replace it earlier?” And he would be right. 

But then something mysterious happened inside my mind. I suddenly felt the will to ignore the machine, and to act on my own judgment, imprecise as it was. Why this strength of will bubbled up inside me at that moment, I do not know—Augustine never explained why or how the will suddenly changes, other than to credit the change to God’s grace. Nevertheless, that strength helped gird me against what followed. 

I decided to remove the epidural and place a spinal anesthetic. I told the couple my plan. “Damn it! If the epidural wasn’t working, then why didn’t you replace it earlier!” shouted the husband. “These are the circumstances now,” I explained as we headed toward the operating room. After pulling the epidural from the patient’s back, I struggled to insert the spinal. The patient winced. Her husband shouted at me, “You’re hurting her!”

“I know, I’m sorry,” I said politely. Then I added, “Sometimes you have to go uphill before you can go downhill.” I was trying to be philosophical to defuse things, as well as to cement my courage inwardly. After a minute of poking her back with a small needle, I decided to use a larger needle to help me place the spinal faster. It worked. Afterward I politely told the patient, “You may have a headache because of the larger needle I used. It may last a couple of days.” “Damn you,” her husband murmured. 

We rolled the mother onto her back. She delivered a healthy baby five minutes later. The mother felt no pain. I dropped the patient off in the recovery room. Neither she nor her husband thanked me. I was left thinking how the slightest change in will can be responsible for the most overwhelming results.

How should doctors view AI? Not as the perfect machine that will help them barricade themselves to the last loophole against any invasion of professional stress or tension. Instead, they should look at AI as just another book on the shelf. Books are not like people, who sometimes impose themselves and burden us with their complaints. When you don’t call for books, they stay put. They offer their opinion only when asked, expressing their thoughts and arousing in us further thoughts. But they do not disturb us when we want quiet. They can provide information, but they lack judgment as it relates to our particular situation, and they do not seek to impose their opinions on us. How could they, when they lack a will? 

AI is not a perfect machine, nor a god, nor even a commanding person. It is a glorified book, a useful accessory. That is its proper role in medicine.