Car companies say self-driving cars are coming soon, but that’s only half the story.

Automakers can build cars with semi-autonomous hardware, but they aren’t in the business of building infrastructure. From the millions of miles accumulated by semi-autonomous cars so far, one thing is clear: Our roads aren’t ready.

Drivers will pay for the hardware that lets their cars steer, accelerate, and stop themselves. But as voters, will they pay the massive bill to fix today’s roads and build a generation of highways ready for self-driving cars?

Levels of self-driving cars

Futurists predict that gas-powered, internal-combustion engine cars are in their sunset years. Within a couple of decades, some estimates say up to 25 percent of all miles driven by 2030 will happen in autonomous electric cars.

Engineers describe self-driving cars (more accurately, autonomous or semi-autonomous cars) in five levels. Level 0 means no car intervention in the act of driving, while Level 1 means the car can intervene for safety. At Level 2, the car can control direction and speed for some time before it requires driver input — like today’s Tesla Model S and X, and the Mercedes E-Class.

Level 3 autonomy puts the car in total control, but requires the driver to take over if the systems fail. It’s a step likely to be skipped over for Level 4, in which the car can control itself and come to a stop if its systems stop functioning. At Level 5 autonomy, the car can drive itself under any conditions. It doesn’t even need controls or a driver.

A few automakers have promised self-driving cars with Level 4 autonomy by 2021. Tesla’s Elon Musk says by the end of this year, his company will make a self-driving car run from Los Angeles to New York — without any driver input.

Road to nowhere

Automakers can build cars with self-driving hardware, but they can’t build the roads those cars need to work flawlessly.

Level 5 autonomy will require near-perfect roads with strong lane markings, extremely accurate maps updated instantaneously, and for now, no ice and snow. That’s the reason most self-driving car demonstrations have taken place in California and Nevada, where test vehicles and Level 2 cars like the E-Class and Model S have begun to accumulate millions of semi-autonomous miles.

In other regions, road quality will have to come dramatically up to speed. The City of Atlanta will christen a smart-traffic corridor along North Avenue later this year. The project will upgrade the street with better striping, better paving, and traffic-sensing cameras. While it grapples with the costs for just that mile of road, it estimates that blanketing the city with similar technology will require 50,000 environmental sensors, 20,000 pedestrian and mobility sensors, and 10,000 cameras.

Costs to repave roads can run more than $1 million a mile. The cameras and sensors could tack on another $50,000, likely more.

The problem? The state of Georgia, like most municipalities in America, is woefully behind on road maintenance. The state asked voters to increase taxes in 2015, by $900 million a year, so it could catch up on deferred maintenance, not including smart-street updates. Voters gave the nod, and now the state is spending $2.2 billion alone to resurface 2,500 miles of roads, repair and build new bridges, and improve intersections for smoother traffic flow.

Georgia isn’t alone. California legislators estimate more than two-thirds of the state’s roads are in poor to mediocre condition, and the repairs could total $130 billion. According to the Department of Transportation, a majority of states face big-ticket repairs on more than half of their roads.

A big bill coming

Automakers say they will deliver Level 4 cars in five years, but American roads can’t live up to their part of the deal.

Without a massive rebuild of our interstates and surface streets, huge swaths of roads will effectively be offline to self-driving cars: they may be under construction, under a blanket of bad weather, or simply in poor repair.

A solution has to come on a national level, and the track record for projects with coast-to-coast scope isn’t good. Every state will have to agree with every other state on uniform self-driving car standards — data as discrete as how often the systems return control to the driver. They’ll have to agree on how to build autonomous-ready roads, and which other public programs to cut or which taxes to raise.

The whole notion of self-driving cars raises basic questions automakers alone can’t answer. Would it be better to build dedicated self-driving lanes? Or simpler to smother the roads in vehicle-to-infrastructure communications? Or both?

With self-driving cars there are no simple answers. The original interstate system cost $230 billion to build.

Decades of planning and hundreds of billions of dollars — maybe trillions — are at stake. It’s time to ask how we’re going to pay for it.

Marty Padgett is the editorial director for Internet Brands Automotive. For more on the costs of self-driving cars, read the full opinion piece.

Above: VB Profiles Connected Cars Landscape. (Disclosure: VB Profiles is a cooperative effort between VentureBeat and Spoke Intelligence.) This article is part of our connected cars series. You can download a high-resolution version of the landscape featuring 250 companies by clicking the image.