Data indicates that about 55 percent of Americans are invested in the stock market through direct ownership of shares, stock mutual fund holdings, or retirement accounts. That figure jumps to 88 percent for households with income greater than $75,000. When it comes to handling their investments, many people turn everything over to someone else, like a stock broker or financial advisor. What’s interesting is how that “someone” increasingly refers to an artificial intelligence system.

Already, some 1,360 hedge funds rely on computer models to trade stocks and other investments. These funds represent $197 billion dollars of investor money being directed by lines of computer code. Most of these funds represent traditional “quant” (quantitative) funds that use computer models to predict share price movements and determine trades.

But an increasing number of hedge funds are entirely directed by AI-powered trading engines. These funds are at the vanguard of AI in financial markets. And, like many markets where AI is transforming business as usual, these engines represent innovative new investment products while simultaneously raising new questions.

The growth in AI-directed investing could have radical consequences, especially in a scenario where a single investor or investment fund using proprietary AI is able to secure an unfair advantage over other market actors. Call it “stock market singularity.” And the groundwork for such an occurance has already been laid.

Financial neural networks resurrected

The investment world began looking at artificial intelligence in the 1990s. The focus then was on artificial neural networks (ANN), computer algorithms modeled after the connections that power the human brain. ANN can be thought of as a predecessor to today’s machine learning systems, computers that self-modify by learning from massive data sets. Neural networks were expected by some to transform trading in the 1990s, but the revolution never came. However, their legacy — programmatic trading — lives on.

Programmatic trading is computer-controlled investing using algorithms to perform the tasks of traditional investment professionals, like spotting opportunities, managing risk, and making lightning-fast trading decisions. This approach shifts a lot of decision-making onto computers, but the technology by itself hasn’t given one market actor an unfair gain over the others.

Today’s neural networks represent a significant increase in functionality over programmatic trading systems. The technology is already becoming mainstream as neural networks are used in voice-activated assistants and self-driving cars. Investment funds want to leverage these increasingly sophisticated systems to achieve faster, smarter trades and better yields.

To see why automated AI trading systems might generate a lot of unprecedented challenges, we need to walk through a few ideas.

On the path to an AI super-investor

Ever wonder why insider trading is illegal? It’s illegal because a single person with information not readily available to other investors has an unfair advantage. This is significant because share prices should, in theory, reflect all of the information available about a company. Having knowledge that other participants don’t have creates the opportunity to trade (buy or sell) shares in anticipation of the price change that happens when that information becomes widely known. If someone knows in advance that a company’s quarterly sales will be unusually strong, that person can buy shares at a price lower than they will trade when the information becomes public. Insider trading and other rules that restrict investment activities of investors with special access to information are one way to enforce fairness among market participants.

Getting back to automated AI systems, we next need to get theoretical. It’s possible to imagine an AI system that’s a perfect predictor of a single financial variable like, say, interest rates. Another system might develop infallible inflation predictions. A third might get really good at predicting earnings growth in a particular industry. And so on.

If these individual systems are possible, it’s within reason to imagine a single system made up of several of these specialist systems. And though it’s far outside the limits of today’s technology, it’s at least plausible that a single system could make use of countless specialist systems that are near-perfect predictors of all of the key factors that move markets. That single system would, by this logic, be virtually perfect at predicting price changes in any — or all — tradeable assets.

Now picture this AI system in the hands of a single investor.

The invention of the perfect AI super-investor is not exactly here yet. But this is the sort of consideration that rapid advances in artificial intelligence require. Because if there’s a chance such a perfect system could come into existence — for the benefit of one individual or group at the detriment of everyone else — it’s worth having a conversation about what to do about it. Luckily, the financial singularity conversation has already started.

Hedging bets on AI

There are, speaking generally, two responses to the financial singularity question. The first is that it’s not actually possible. The second is that it wouldn’t actually be that bad.

Representing the first group is Babak Hodjat, the founder of one AI trading fund, who is eager to see trading in the hands of AI. “It’s well documented we humans make mistakes,” he told Bloomberg. “For me, it’s scarier to be relying on those human-based intuitions and justifications than relying on purely what the data and statistics are telling you.”

Others dismiss the idea that one company will achieve such advances without competitors close on their heels. “If someone finds a trick that works, not only will other funds latch on to it but other investors will pour money into [it]. It’s really hard to envision a situation where it doesn’t just get arbitraged away,” author Ben Carlson told Wired.

There is also the idea that a financial singularity would be beneficial. By this logic, a market that operates purely on logic could reach perfect efficiency, where all assets are priced correctly with no need for human intervention. Computers would set prices based on optimized projections that include future profits, tech advancements, and demographic shifts, according to Robert J. Schiller, a Yale economics professor.

Schiller is skeptical that a financial singularity lies ahead. He argues that it would have to occur in a world where markets run according to rationality alone. But humans are irrational, and a successful AI would have to account for our unpredictable natures.

A future worth pondering

At present, trading algorithms can fake one another out to gain advantages, which the BBC notes is illegal but difficult to prove. They can also predict a slower program’s next moves and then trade accordingly. With firms competing aggressively to get faster trading times, a slower program could create massive functionality gaps. As algorithms become more intelligent and more powerful, the financial industry will require ever-smarter safeguards against exploitation and risk.

Then there are the potential glitches. In August 2012, a trading program at one fund “ran amok,” creating losses of $10 million a minute. It took nearly an hour for the human team to identify and solve the problem, and the firm lost $440 million in the process. Two years earlier, an algorithmic trade caused a “flash crash,” in which U.S. share and future indices dropped 10 percent within minutes.

Some say those incidents are telling preludes to disaster. A rogue algorithm at one of the country’s major banks, or a cascading failure in which multiple big banks are derailed by faulty programs, could lead to a catastrophic crash.

Whether the financial singularity will happen — and whether its impact would be positive or negative — remains to be seen. But we should all be paying attention because, as we witnessed in 2008 with the financial crisis, what happens in the market affects us all.

Additional article contributors: Mehdi Ghafourifar and Brian Walker

Alston Ghafourifar is the CEO and Cofounder of Entefy, an AI-communication technology company, introducing the first universal communicator.

The article originally appeared at Entefy.