IBM is in negotiations to buy Sun Microsystems for $6.5 billion, the Wall Street Journal reported today.

If the deal happens, it would be one of the biggest consolidations in the computer industry since Hewlett-Packard merged with Compaq in 2001.

IBM would eliminate a major competitor if it follows through with the all-cash deal, which would be a big premium over Tuesday’s closing price of $4.97 a share. The deal would strengthen IBM’s hand against HP, which is now the largest company in the information technology industry, and it would be the largest acquisition in IBM’s history.

The Journal cited sources who said that while talks are under way, a transaction might not occur. Neither Sun nor IBM confirmed the talks.

A deal would change things in the core server market where IBM and Sun compete. International Data Corp. says that IBM had 31.4 percent of the server market in 2008, while HP had 29.5 percent, Dell had 11.6 percent, and Sun had 10.6 percent. But Sun is strong in some areas, such as machines based on its own Solaris software and the financial services market.

Earlier this week, Cisco shook things up in servers by announcing its own line of servers to compete with IBM and others. Last year, HP made a move on IBM by acquiring EDS, which has an army of people working on data center services, which is IBM’s core turf. Sun recently announced a cloud-computing initiative.