Cognitive bias: The elephant in every room

Across government offices, tech company board rooms and the homes of every individual, there is an elephant in every room. The English idiom is in this case a reference to the same “elephant” described in Jonathan Haidt’s social psychology work, where the conscious and subconscious human mind are described as the “elephant and the rider.” What this means is that in every room where a human is present, there is also an elephant: That human’s subconscious.

Although the conscious mind is very adept at ignoring the elephant, doing so only produces blindness to the biases being applied to every decision. The elephant is always there, and if it isn’t addressed, that blindness dominates the decision-making process.

For more than 200 years, economic systems were built on the blatant fallacy that humans behave rationally, ignoring the elephant and producing global systems that are prone to periodic severe failures, thanks in part to “theory-induced blindness.” Political pundits were also famously demonstrated to be less accurate in their predictions than “dart-throwing chimpanzees,” thanks to biases like “inattentional blindness” and the “Dunning-Kruger effect.”

The unscalable brain

Worse still, the elephant is growing, and sometimes that growth is exponential. This growth is the product of the trade-off between complexity and cognitive bias. Human cognition isn’t fundamentally scalable, as the human brain can’t just spin up new hardware to apply cognition to address greater complexity. Complexity also continues to increase, at an increasing rate, so not only is the trade-off growing more severe, but that growth is accelerating. Humanity also faces too many challenges to survival to suffer technological stagnation, as that too poses a myriad of different existential risks like climate change and antibiotic-resistant bacteria.

Narrow Artificial Intelligence (AI) and groups of humans can shift the damage caused by more than 200 different kinds of bias into diluted and obscured new forms, but neither offers the means to overcome the trade-off. Frequently, they make the problem worse by shifting the damage below thresholds of perception and into “unknown unknowns.”

Humanity is already suffering severely and comprehensively from this growing problem, in every domain and every city around the world. Even as the problem grows more severe, humanity’s ability to recognize it is diminished, promoting the growth and formation of information siloes in a world of “choice overload” where low-quality information is the viral norm.

The gates of every information silo are guarded by biases of expectation, rejecting information that doesn’t conform to the picture of the world that the silo paints. Information that agrees with, and is able to move across many such siloes, must thus be so agreeable as to be trivial, the very opposite of a “breakthrough.”

Reality immunity

Much as the concept of “herd immunity” shows that a virus can’t survive once it can no longer travel across a population, humanity is entering the advanced stages of a parallel concept, “reality immunity.” When meaningful discoveries and breakthroughs can no longer travel across a population, that population will reliably drift further away from reality over time.

The means of manipulating populations are well-studied under the domain of “marketing,” and that manipulation has been automated globally through narrow AI, greatly accelerating and reinforcing such information siloes.

In this way, narrow AI has been utilized to great effect, to accelerate the growth of the elephant in every room by making every room conform more robustly to the expectations of the elephant. Should this status quo continue over the coming years, the metaphorical exit to every room will be blocked by the elephant, and the elephant’s 200-plus cognitive biases will make every decision using data of low and decreasing quality.

No human is immune to this, nor can any human in a deeply interconnected world escape the consequences should this continue past that “event horizon,” beyond which human cognition can no longer escape.

Beating back subconscious bias

There is a way to overcome this impending existential threat. Not only that, if a single industry, major company, government or other source still able to transmit knowledge across the world’s information siloes applied the methods to overcome this challenge, a chain reaction could occur across humanity.

This is possible because the opportunity and advantage posed by overcoming this challenge is equal and opposite to the damage caused by the problem. When the impending risk is at the level of cognitive death and subsequent human extinction, the selection pressure applied to the first method of overcoming the challenge becomes the maximum value possible.

Humans can’t scale, and narrow AI can’t “think,” so neither is suitable for this challenge. However, scalable human-like intelligence can, and at humanity’s present level of scientific understanding, it is also the only thing that can. Since humanity’s further development is effectively gated, the adoption of such technology could be compared to the concept of “The Great Filter,” where all roads in the development of a species save for one result in extinction.

Climate change, war, supply chain collapse and a myriad of other great risks exist in the world today, but all of them are underpinned by the shared root cause of increasing complexity, and proportionately increasing reliance on cognitive bias. As humanity approaches the event horizon of cognitive death, beyond the point where meaningful information can travel across populations, the line we must never cross grows ever fuzzier.

There is no problem in the world that requires more immediate attention than the root cause shared by all of them. Once this root cause is addressed, what can be done with $100 million today may only require $1 million, and with far greater odds of success and more expediently than was previously possible. The elephant in every room can either be the cause of human extinction, or the greatest opportunity humanity has ever known.

Much as innovation contest hubs discovered that experts outside of a “hard problem’s” narrow domain most frequently solved these problems, the problems that those major companies still able to transmit information today consider hard are often, and frequently already have been, solved by others. The problems we face may not be as hard as you think, and they may even have already been solved.

Full disclosure: I’ve focused my life on addressing these challenges, because it would be hypocrisy to do otherwise, and extinction doesn’t agree with me. Over the past few years, I and others at AGI Laboratory have volunteered our time for this purpose.

Kyrtin Atreides is COO and a leading researcher at AGI Laboratory.