Stafford Beer’s cybernetics tells us that in these cases, while people’s opinions are important, the facts of the organisational outcomes are what we need to work with. In his most pithy formulation of the principle, he expands the black box principle to a rather more uncomfortable statement.

The purpose of a system is what it does.

So, from here on in, I will try to refer to ‘decision making systems’ rather than ‘artificial intelligences’. Corporations are systems, and the make decisions, so they’re decisions making systems. The question is whether they’re black boxes or not — whether we are able to attribute the actions of the corporation to individual human beings within it.

An organisation does things, and it systematically does some things rather than others. But that’s as far as it goes. Systems don’t make mistakes — if they do something, that’s their purpose. But it also works the other way around. Systems don’t have inner desires, so they don’t do things intentionally either. There’s just a network of cause and effect. We might think they’re conspiring, but they’re working within structures that made the outcome inevitable. Or we might see everything as a terrible cock-up, but we don’t understand that the outcome was the inevitable result of the way the system works.

This is the whole theory of the firm, according to the Friedman doctrine. There is no analysis of the company as a decision-making system, just individuals making decisions. While modern HR people might talk about ‘bringing your whole self to work’, Milton Friedman explicitly tells the (assumed to be male) modern executive that he has to divide his moral self into judgements ‘as a person in his own right’ and decisions ‘in his capacity as a businessman’.

One of its signals has been so amplified that it drowns out the others. The ‘profit motive’ isn’t something that can be ascribed to corporations — they don’t have motives. What they have is an imbalance between the two key higher-functions — here-and-now versus there-and-then. They aren’t capable of responding to signals from the long-term planning and intelligence function, because the short-term planning function has to operate under the constraints of the financial market disciplinary system. Either a corporation has a survival condition based on needing to make a monthly interest bill, or there’s an implicit threat from the financial environment that if it fails to behave in a particular way, it will be taken over by an outside entity.

If you take away that pressure, it’s quite likely that the natural equilibrium of corporate decision-making systems will be less hostile to human life. Viable systems fundamentally seek stability, not maximisation.