The miserable sinner
— ah damn, I should’ve stolen the ginger.
Of all the things I have tried to make sense of, this one comes out at the top for being the hardest.
For background, I have written before about theoretical reasoning and practical reasoning. Theoretical reasoning is the mechanism by which we form beliefs, for example that the Earth is not flat. Practical reasoning is the mechanism by which we form intentions, for example that I should get a coffee. One deals with how things are, the other with what should be done, or how they should be. Practical reasoning concerns action. But what is the difference between a calculator “acting,” a coffee machine “acting,” and me acting? The question is: what necessitates practical reasoning?
In the last essay, I understood that the capacity to do otherwise is what necessitates practical reasoning: something needs to be done in the face of alternatives, and hence the question of what should be done is posed. However, there are alternatives in the case of a calculator as well. There is one world in which it calculates and one in which it does not. Similarly for the coffee machine: one world in which the coffee is made, and one in which it is not.
To make the problem more interesting, let’s add determinism into the mix. Determinism says the full state of the world at one moment, together with the laws, fixes every later state. That is, every action has a cause, and given the appropriate resources and information, every single action can be modelled, at least hypothetically.
To set up the problem, consider three cases:
- A calculator asked to make a certain calculation. There are two contingent worlds: one in which the calculation was made, and one in which it wasn’t.
- A coffee machine asked to make a coffee. Similarly, one world in which the coffee was made, and one in which it wasn’t.
- Me, before getting a coffee: do I get a coffee or not? One world in which I did get a coffee, and one in which I did not.
Determinism says that only one of these contingent worlds will actually come true, and the other will be ruled out. And this can be modelled from some prior state, from before the calculator was posed the question, before the coffee machine was switched on, and before I ever thought about coffee. Or one can say whatever world turns out among the two contingents, we all are a “cause” in a certain sense. So where is the difference?
And yet it seems clear that there is a distinction between the sense in which a calculator or a coffee machine is a cause, and the sense in which I am the cause. Or is the distinction only apparent?
To see the difference, look at what happens when each one fails. If the coffee machine does not make the coffee, we say it is defective. Same for the calculator: if it outputs $2 + 2 = 5$, it is broken. But if I fail to keep a promise, I am not broken. Apparently, I have violated something.
To see the difference, and to simplify a bit, consider the following computer programs.
- Code to add two numbers: $x + y$.
- A conditional code: if $A$: $x + y$, if $B$: $y + z$.
- A third case of the form where there are rules $\{A, B, C, D\}$, and then a situation $f(Z)$ where $Z \in \{A, B, C, D\}$.
- Third program from above, and a certain fixed rule to pick from $\{A, B, C, D\}$.
- Third program from above, but a randomisation rule to pick from $\{A, B, C, D\}$.
- Third program from above, plus the “agent” with a “second-order” ability to evaluate $A, B, C, D$. No mechanism is defined to select from $\{A, B, C, D\}$.
The first one and second one are completely specified programs. And the third one has an explicitly unspecified variable $Z$ that the output of the program depends on. Clearly, we need some mechanism to choose or pick (or decide) for $Z$. We can think of a fixed mechanism (another meta-program) that returns $Z$. A randomisation over $\{A, B, C, D\}$ could be one such mechanism. However, the sixth case is a case of an incompletely specified program, where no mechanism for $Z$ is explicitly specified. However, it has been left to the certain “agent” (if the program is being run on the machine, then to the machine) to decide and choose for $Z$.
Consider a machine in which these programs are to be run, then it is apparent that there is something distinct about the sixth case above, in which the machine will be the “cause” of what happens, vs the other programs. We can argue that a calculator and a coffee machine could fall into any of the first to fourth (or even fifth) programs. Not the sixth one. The sixth case asks for something more, i.e. deciding on $Z$ is also something the agent does, by utilising second-order evaluative standards. And not merely execute the program like the other cases.
This will lead to define how differently I’m the cause of the one of the contingent worlds that would turn out: one in which I got the coffee and another in which I did not. Simply by not merely executing a program or being a machine on which it is executed, but I’ve been given an incompletely specified program, where a part of the work is needed for me to use my second-order capacity to fill in for the incomplete information. I’ll next state what that capacity amounts to.
Normative Agency. The name of the capacity is normative agency. An agent has normative agency when it has the capacity to give itself principles, and to be answerable to them. However, it is clearly confusing, so below is an elaboration.
A principle is a rule, of the form “in situations of type $S$, do $X$,” that the agent holds as a rule. The agent gives itself a principle when it adopts the rule as a result of its own evaluation, rather than having it fixed from outside. A clarification: I can get a rule from outside. I can learn from a scripture or from an older person that lying is bad, and adopt it. But that is not yet giving myself the principle, unless I recognise and accept the rule under my own evaluation. The rule can be deliberately put, or deliberately kept. A dog trained to return the ball, or a child made not to bite other children, follows a rule. However, it is unclear whether a dog or a child have evaluated the rule under their own evaluation.
Then comes the ability to recognise the situations where a rule applies, and to navigate the situation as per the rules that apply to the case. For example, I might have given myself a rule to always say the truth, however, if I cannot recognise the situation where the truth needs to be said, then something is off. This part also contributes to the answerability word in the statement made above. That is the agent answers to the rule: it can recognise when it failed to apply it. Second, the rule answers the agent’s reasons: it can be revised when new reasons appear.
It is worth clarifying that normative agency, defined this way, is empty about whether the rule is good or bad. Consider someone who makes it a rule to steal ginger from the grocery store every week, for the reason that grocery stores are exploiting people. Every time this person is at the store, they can call the rule into being and steal the ginger. Say one time they fail to, because they forgot, or they already had ginger at home, or they were travelling. They are still answerable to the rule: “Ah damn, I should’ve stolen the ginger, I could have given it to friends.” And because the rule rests on their reason, it could also be revised, say if they learned this store was a worker-owned cooperative. This person is a normative agent. Their rule is not mechanically imposed.
The definition has two parts, and apparently that justifies the title of the essay. A normative agent can give itself rules as per its own evaluation, however the rule cannot be automated. For example, it cannot happen as the fourth program from above where there is a fixed mechanism to decide for $Z$ over $\{A, B, C, D\}$. Instead, a normative agent has rules that are, in a certain sense, exposed where while the agent can give itself rules, they have to be always brought into considerations, their evaluations always subject to reasons, and revising and updating. And even if revision is not happening, the agent has to navigate the situation recalling the rules that apply every single time. Another way of framing this is that the agent retains the capacity for failure, i.e. failing to apply the rule. A rule should not become automatic as the fourth program. However, that does not mean its application does not get easier.
This defines the problem of a miserable sinner. A calculator will be defective, however, a normative agent is handed an incompletely specified problem, and has to fill in the gap every time it acts. The possibility to fail is in the definition, and hence failing to apply the rule is also a failure of the agent, unlike a malfunction in the case of a calculator.
To think further, let’s contrast a surfer with a train. The train runs on rails, and the path is fixed by the track. However, the surfer stays upright only by constantly responding to the wave, and it could fall at any moment. Surfing requires the ability to read the waves, as well as to choose which waves to ride. Normative agency is like a wipeout waiting to happen: the agent is moving, and every once in a while the rules get challenged. Sometimes the situation challenges them, with a temptation or a messy case where it is hard to see which rule applies. Sometimes the rule itself gets challenged, when a reason appears that it might be wrong. However, a good surfer is the one who navigates the challenges well.
So morality is hard because it disallows the mechanical application of rules, while still requiring enough reasoning at each time to do the right thing. This does not mean reasoning from scratch every single time. A deeply honest person may tell the truth smoothly, without being tempted at all, the same way a skilled surfer rarely falls. What matters is that the rule never escapes from reasons: the light is off, and even if you remember the room, you have to be ready to stop when you bump into something new. You have to put in effort when the situation calls for it, and you can never know in advance when it will.
This also answers the question I set out to ask. What necessitates practical reason, or deliberation. The answer is normative agency. A normative agent is given an incompletely specified problem, and they have to be the cause of the contingent worlds that happens. To be that cause, it is demanded to deliberate what needs to be done. Christine Korsgaard uses the term authorship of the action, to author an action means to fully commit to it, and it requires doing one’s due diligence and deliberate to cause the action that one can stand behind.
Now I didn’t start thinking of normative agency out of nowhere. My work on deliberative prediction (that is not out yet still, however it deals somewhat with the theoretical vs practical reason as the name suggests), and it led me to normative agency in AI. So I’ll end the essay by entertaining that thought.
Notice nothing in my argument appeals to sentience, consciousness, or being alive. Normative agency is about being an author of your own action. That is the agent has been given an incompletely specified problem, with certain second-order capacity (either to set rules for oneself, or to have the capacity to set evaluation by itself over the specified rules), and the capacity to execute the task by its own evaluation, including evaluation of its own standards. The calculator and the coffee machine fall outside. However, sentience is not at play, but their problem is fully specified and their rules are locked. The infant falls outside, for now, because it has not yet formed rules and cannot reason about them.
In my viewpoint, normative agency is something closer to the shape of a problem than to a property of living things. It also means the line is not drawn in advance between machines and humans. A machine built so that its task was genuinely incompletely specified, and whose standards stayed open to its own review, would, by this argument, be on the same side as me. Whether any machine is actually like that is a separate question. But the answer would depend on how it is built, not on whether it feels anything.
So, to come back to the start: what necessitates practical reasoning is not the bare existence of alternatives, since the calculator has those too. It is being handed a problem that is not fully specified, being the one who has to fill it in, and being answerable for how one does. That is the burden of the miserable sinner, and, as far as I can tell, it is what it means to be the cause as well as the author of one’s own actions.