🔍
← Back

2. Stafford Beer

Later on, Beer would identify it as one of his fundamental axioms: ‘It is not necessary to enter the black box to understand the nature of the function it performs.’ The underlying idea is somewhat stronger than this: if the black box is a complex system, it’s likely to be pointless — or even dangerous — to try to understand its inner workings and use that understanding to manipulate a precise outcome. This is a matter of respecting the complexity of the problem — a genuinely complex system is one in which you cannot hope to get full or perfect information about the internal structure, and cannot have any acceptable degree of confidence that the bits of information you don’t have can be safely ignored. Rather that trying to use a mixture of partial information, preconceived theory and guesswork, you need to step back, accept that the system will keep its secrets, and observe its behaviour. And of course, this means that the different observers might have different opinions. The property of ‘being a black box’ isn’t an objective one — it’s a description of a decision taken by someone working on the system that they don’t have enough understanding to safely treat it in any other way. In some of Beer’s writings, the property of complexity itself is also dealt with in this way; rather than trying to count combinations of connections, you define what it is for something to be a complex system by saying that it’s one which has to be treated as a black box.*

*if you can control something in your factory like a machine, it’s a ‘clear box’ in this language; there are even ‘muddy boxes’, where more detailed analysis could help understand the relationship between inputs and outputs but the system keeps a few secrets because really detailed analysis would require disproportionate effort.

DaviesThe Unaccountability Machine
p.40-41

Related Quotes

Economists end up solving this problem by denying it (a mechanism that Stafford Beer looked at in detail, concluding that ignorance is a kind of information processing system of last resort). Where analysis fails, ideology steps in, and the solution the economists decided on was to fantasise an equivalent system for producers — a profit-maximising firm with an understanding of its market, a style production process and a manageable set of decisions. It ended up going disastrously wrong, but the original intellectual sin may have been the failure to respect the integrity of the black box.

DaviesThe Unaccountability Machine
p.44-45

Stafford Beer’s cybernetics tells us that in these cases, while people’s opinions are important, the facts of the organisational outcomes are what we need to work with. In his most pithy formulation of the principle, he expands the black box principle to a rather more uncomfortable statement.

The purpose of a system is what it does.

DaviesThe Unaccountability Machine
p.52

So, from here on in, I will try to refer to ‘decision making systems’ rather than ‘artificial intelligences’. Corporations are systems, and the make decisions, so they’re decisions making systems. The question is whether they’re black boxes or not — whether we are able to attribute the actions of the corporation to individual human beings within it.

DaviesThe Unaccountability Machine
p.68

There is a solution to this kind of problem in the viable system model — Beer emphasises it in his later work, particularly in the parts of the second edition of Brain of the Firm that were written after his experience in Chile. A truly viable system needs to have communication channels that link the operations to higher-level management functions and even to higher levels of recursion. Stafford Beer called these ‘algedonic signals’ — a kind of neurological metaphor, coined from the Greek words for pain and pleasure.

It might be easier, though, to understanding them as the kind of messages sent by the red handle in a train driver’s cab. As well as a physical check on the individual vehicle, the emergency brake sends an organisational message to the whole railway, informing it that a piece of track can’t be used.

DaviesThe Unaccountability Machine
p.131

Corporations are decision-making systems, not ‘intelligences’. They have homeostatic forces which aim to maintain their equilibrium, and higher-order decision-making systems which mean they are able to reorganise themselves in order to respond to shocks beyond the scope of anything anticipated when they were designed. To attribute motivation to them is to make assumptions about the internal workings of the black box — the original intellectual sin of cybernetic analysis.

DaviesThe Unaccountability Machine
p.267