Economists end up solving this problem by denying it (a mechanism that Stafford Beer looked at in detail, concluding that ignorance is a kind of information processing system of last resort). Where analysis fails, ideology steps in, and the solution the economists decided on was to fantasise an equivalent system for producers — a profit-maximising firm with an understanding of its market, a style production process and a manageable set of decisions. It ended up going disastrously wrong, but the original intellectual sin may have been the failure to respect the integrity of the black box.

There’s one sense in which the recording engineers’ maxim is more applicable, though. These are problems of human organisation, and the overwhelming majority of environmental variety of created by humans. The knowledge that something is an organisational problem means that the source of its complexity is the human beings in the organisation. That, in turn, means that the human beings in the organisation have sufficient variety and capability to match the complexity of the problem; the problem can’t be bigger than them, because their own misdirected efforts created it. Only problems that aren’t cybernetic can genuinely be insoluble.*

*One implication of this proposition is that it gives us a rigorous deduction of what would constitute environmental Armageddon — it’s the singularity point at which the problem of climate change ceases to be soluble purely by changing human behaviour and organisation.

This is the paradigm shift that might be required — that organisations and systems can be like people, having purposes without a single goal. An artist doesn’t have a successful career by maximising their art; they do it by repeatedly producing work that they are proud of.

That’s what the world could look like if we got rid of the blind spots. Business ought to be like artists, not paperclip maximisers. The economic concept of optimisation, and the institutions of management and government which enforce its use, effectively act as a brutal information reducing filter. By taking away the pressure to maximise a single metric (and therefore to throw away information that doesn’t relate to it), organisations could apply their decision-making capabilities much more effectively. They could innovate more, design more sustainable solutions and build less adversarial, longer-term relationships with their people.

Corporations are decision-making systems, not ‘intelligences’. They have homeostatic forces which aim to maintain their equilibrium, and higher-order decision-making systems which mean they are able to reorganise themselves in order to respond to shocks beyond the scope of anything anticipated when they were designed. To attribute motivation to them is to make assumptions about the internal workings of the black box — the original intellectual sin of cybernetic analysis.

If every maximising system has to have a higher-level system governing it (to make sure it doesn’t go bonkers), then that logically implies that the top level of any decision-making system that’s meant to operate autonomously can’t be a maximiser. And so, the governing philosophy of the overall economic system can’t be based on the constrained optimisation methodology that’s currently dominant in the subject of economics. Otherwise there’s a risk that the system will go bonkers, and that it will start pushing maximising objectives, oblivious to the danger that it’s on course for making human life impossible. Like it actually has done.