2. Stafford Beer

Later on, Beer would identify it as one of his fundamental axioms: ‘It is not necessary to enter the black box to understand the nature of the function it performs.’ The underlying idea is somewhat stronger than this: if the black box is a complex system, it’s likely to be pointless — or even dangerous — to try to understand its inner workings and use that understanding to manipulate a precise outcome. This is a matter of respecting the complexity of the problem — a genuinely complex system is one in which you cannot hope to get full or perfect information about the internal structure, and cannot have any acceptable degree of confidence that the bits of information you don’t have can be safely ignored. Rather that trying to use a mixture of partial information, preconceived theory and guesswork, you need to step back, accept that the system will keep its secrets, and observe its behaviour. And of course, this means that the different observers might have different opinions. The property of ‘being a black box’ isn’t an objective one — it’s a description of a decision taken by someone working on the system that they don’t have enough understanding to safely treat it in any other way. In some of Beer’s writings, the property of complexity itself is also dealt with in this way; rather than trying to count combinations of connections, you define what it is for something to be a complex system by saying that it’s one which has to be treated as a black box.*

*if you can control something in your factory like a machine, it’s a ‘clear box’ in this language; there are even ‘muddy boxes’, where more detailed analysis could help understand the relationship between inputs and outputs but the system keeps a few secrets because really detailed analysis would require disproportionate effort.

Stafford Beer’s cybernetics tells us that in these cases, while people’s opinions are important, the facts of the organisational outcomes are what we need to work with. In his most pithy formulation of the principle, he expands the black box principle to a rather more uncomfortable statement.

The purpose of a system is what it does.

Trying to build things meant that everything had to be kept specific, but the theory of information couldn’t straightforwardly be applied to massively connected systems. You needed to find a way of describing things that was both rigorous and representative of reality. As economists will tell you, this isn’t an easy thing to do.

This is how the cybernetic problems of the post-war industrial state have been shaped by economics. There are three major blind spots: the belief in markets as computing fabric, the flattening of time and uncertainty and the commitment to Ricardianism.

The belief in markets as computing fabric is such an obvious source of blind spots as to hardly require explanation. Faith in the power of free markets to solve problems is so great as to support the reverse inference of economists that if something appears to be a market equilibrium, then it must be an optimal solution.

If every maximising system has to have a higher-level system governing it (to make sure it doesn’t go bonkers), then that logically implies that the top level of any decision-making system that’s meant to operate autonomously can’t be a maximiser. And so, the governing philosophy of the overall economic system can’t be based on the constrained optimisation methodology that’s currently dominant in the subject of economics. Otherwise there’s a risk that the system will go bonkers, and that it will start pushing maximising objectives, oblivious to the danger that it’s on course for making human life impossible. Like it actually has done.