Stafford Beer’s cybernetics tells us that in these cases, while people’s opinions are important, the facts of the organisational outcomes are what we need to work with. In his most pithy formulation of the principle, he expands the black box principle to a rather more uncomfortable statement.

The purpose of a system is what it does.

So, from here on in, I will try to refer to ‘decision making systems’ rather than ‘artificial intelligences’. Corporations are systems, and the make decisions, so they’re decisions making systems. The question is whether they’re black boxes or not — whether we are able to attribute the actions of the corporation to individual human beings within it.

Trying to build things meant that everything had to be kept specific, but the theory of information couldn’t straightforwardly be applied to massively connected systems. You needed to find a way of describing things that was both rigorous and representative of reality. As economists will tell you, this isn’t an easy thing to do.

There’s one sense in which the recording engineers’ maxim is more applicable, though. These are problems of human organisation, and the overwhelming majority of environmental variety of created by humans. The knowledge that something is an organisational problem means that the source of its complexity is the human beings in the organisation. That, in turn, means that the human beings in the organisation have sufficient variety and capability to match the complexity of the problem; the problem can’t be bigger than them, because their own misdirected efforts created it. Only problems that aren’t cybernetic can genuinely be insoluble.*

*One implication of this proposition is that it gives us a rigorous deduction of what would constitute environmental Armageddon — it’s the singularity point at which the problem of climate change ceases to be soluble purely by changing human behaviour and organisation.

Corporations are decision-making systems, not ‘intelligences’. They have homeostatic forces which aim to maintain their equilibrium, and higher-order decision-making systems which mean they are able to reorganise themselves in order to respond to shocks beyond the scope of anything anticipated when they were designed. To attribute motivation to them is to make assumptions about the internal workings of the black box — the original intellectual sin of cybernetic analysis.