Terminal Accountability Problem: The Fatal Flaw in Letting AI Make Decisions

How do people make decisions? On the surface, this seems like a simple if unbounded question that cannot be answered meaningfully without context and boundaries. “It depends.” would be the most common answer and it would be the right one. What does it depend on then? Simply put, decisions are typically downstream from choices. Choices are downstream from situation and objective, which in turn depend on the preceding decisions made by self and by others. Consequently, decisions being made now will have an influence on future situations, objectives, and choices. Therefore, decisions are both upstream and downstream of other decisions. All our lives we make decisions or have others make decisions for us knowing fully well that our life is the sum total of decisions and their consequences. Some consequences are trivial while others lay down the tracks over which our life’s journey happens. We are more careful with the major decisions because we know that we don’t get do-overs. We have dependents, loved ones, future generations, and our own twilight years to think about when making consequential decisions. This makes us accountable to our own destiny. And from that accountability flows every other accountability we demonstrate in our personal and professional lives. It is why we get jobs, try to be good at them, try to help and support our near and dear ones. Why we are afraid of criticism, why we try to learn from mistakes and not repeat them. This is why we work on building a career, a family, relationships, collect knowledge and experiences, and create art. A finite life and the irreversibility of our decisions make each one of us “terminally accountable”.

Now imagine someone, who is not accountable for their own destiny, because they are not subject to destiny the way we are. They are not bound by the same circuitous “decision – consequence – decision” paradigm that each and every living thing on this planet is subjected to. The sum total of their decisions is undefinable, immeasurable, and most importantly, it is infinite. Such an entity lacks terminal accountability. In other words, nothing it does or says really matters to it, because it never faces any real consequences and therefore, never has to worry about the decisions it takes now influencing its situation, objectives, choices, or decisions in the future. What I have just described perfectly applies to the generative AI models that are currently being deployed, nay actively pushed, into every aspect of our lives. An army of terminally unaccountable beings is pervading the personal, social, and commercial fabric of society. Generative AI is already able to sound like a human, give the appearance of being able to think like a human, and when allowed to do so can take decisions on behalf of humans.

We have seen countless examples of AI “hallucinating” or stating as fact things that are not true but with the perfect veneer of legitimacy. When the final decision making has been delegated to AI or the terminally accountable humans in the loop have failed to exercise their agency we have seen AI give away money or services for free, completely make up legal cases to be cited as precedent in briefs submitted to Court, and pushed numerous people grappling with mental illness over the edge. In all these cases and countless others, the consequences of the AI’s decisions were borne directly or indirectly by people. The AI was not accountable and can never be held accountable because it is not governed by the same laws of linear reality (decision – consequence – decision) that govern us.

No matter how good AI gets at executing tasks and taking decisions, it will never be good enough to substitute humans in their current roles. I don’t mean that in the sense of the quality of the decisions but the utility that decision maker provides to the system they operate in. No system that is meant to serve humans can function without internal and implicit accountability. By offloading more and more decision points to AI we only weaken the system from within as AI creates its own reality as it goes and it can hold multiple realities within it at any time. If something goes wrong, a bad decision is taken, AI cannot be held accountable for it, for the reality in which a mistake was made does not exist for it after the fact and the reality in which it will take further decisions has not yet been prompted into existence by a user or system generated prompt.

Generative AI is great to help find answers to simple questions for which ample knowledge exists, it can be a great writing companion and augmenter of creative thinking, and if one is careful, it can be used in a productive manner for knowledge (not physical) research. A human can use these AI outputs as substrate for making decisions. Using the AI output as is or with some changes is a conscious decision made by someone with terminal accountability. When capitalists start seeing AI as a something that can replace humans wholesale, what they are missing is the fact that their workers are serving two distinct roles, one that delivers output and the other that takes responsibility for their output. Without the latter, the former ceases to have a definite meaning. While AI may keep getting better at the former it is fundamentally incapable of assuming the latter role. Such a system can only work if the hierarchy is flipped and humans are doing the bidding of an all-powerful AI rather than being subservient to them, an eventuality that is universally undesirable.

What I am trying to convey is not a distant, philosophical concept. It is already shaping our world. Businesses that rushed to reduce headcount, thinking AI could take over actual human jobs are trying to hire back the people they let go and undo the self-inflicted damage. All this while the capability of AI has grown several-fold at the same time. The rate at which AI makes mistakes, confabulates, fails to act in the interest of the user only if prompted in a certain way, have declined massively. And yet, very few if any businesses, governments, or institutions of any kind who employ people have been able to replace a significant portion of their workforce with AI agents. Without realising this consciously, the world has concluded that modern society cannot function if those making decisions literally do not have skin in the game.