Tony Fadell, the man who brought the idea of a portable audio player to Apple, has now cut all of his ties with the company, according the New York Times. He initially stepped down from his role as senior VP of Apple’s iPod division in November 2008 for “personal reasons”, but remained close to the company as a special advisor to Steve Jobs.

It’s common for companies to pay former executives as “advisors” or “consultants” to keep them off the market, especially in California, where strong employee protection laws would otherwise prevent the enforcement of noncompete clauses. So the ending of this arrangement frees Fadell up in many ways.

Fadell first tried to develop his own hard-disk based portable music player with his company Fuse in the 90s. After Fuse failed to gain a second round of funding, he brought the idea to RealNetworks in 2000 — but he left the company after six weeks because of disagreements with CEO Rob Glaser.

In 2001, Fadell brought the idea to Apple — and well, you know how that turned out. He worked for former Apple senior VP Jon Rubinstein (now CEO of Palm), and replaced Rubinstein when he retired in 2006. In addition to helping jump start the iPod, Fadell was also instrumental in developing the iPhone. When he stepped down in 2008, he was replaced by IBM veteran Mark Papermaster.

When the NYT reached him by phone, Fadell mentioned that he wants to leave a bigger legacy than just the iPod:

“My primary focus will be helping the environment by working with consumer green-tech companies,” he said. “I’m determined to tell my kids and grandkids amazing stories beyond my iPod and iPhone ones.”

Whatever he ends up doing will certainly be worth paying attention to. There’s a lot of latent potential in green technologies, and I have a feeling that it will make as big an impact during this decade as the Internet did in the last. Given Fadell’s history of groundbreaking ideas, I suspect that he has something special in mind for the green tech industry.