In early Oct. 2010, then working as a columnist at the San Jose Mercury News, I asked a simple question after the announcement that Dick Costolo was taking over as Twitter CEO:

Is the new Twitter CEO the right person for the big job ahead?

The column listed the big challenges Costolo then faced replacing cofounder Ev Williams. He was going to be Twitter’s third CEO in the four years since its founding, an unusual turnover in Silicon Valley where visionary founders are worshipped like deities.

Going back and reading that column now, I would say that Costolo overall has done a remarkable job. At the same time, he’s hardly leaving on a high note, having missed first quarter earnings, failed to convince Wall Street that he’s got a plan, and even triggering the mother-of-all-love-hate-blog-post-critiques from angel investor and major shareholder Chris Sacca.

With that in mind, here are some thoughts on Costolo’s tenure and how he answered my original question:

The good

In Nick Bilton’s “Hatching Twitter,” he quotes Facebook founder Mark Zuckerberg as saying:

“[Twitter is] such a mess – it’s as if they drove a clown car into a gold mine and fell in.”

The site was still constantly crashing. Indeed, it crashed the day Costolo was appointed CEO. It hadn’t figured out how to effectively monetize its rapidly growing user base. Just months after its first developers’ conference, the company had a confusing message about which services it would build and what would be left for third parties.

And on a fundamental level, Twitter was still quite confusing for new users.

Rather than going out and hiring a seasoned executive, like an Eric Schmidt at Google, Twitter promoted its then chief operating officer, Costolo. While he had previously been a CEO, Twitter was already bigger than the three companies he had personally founded.

At the time, I wrote: “There are several things about Costolo that give me reservations about his ability to run a company now valued at more than $1 billion by its investors.”

Since then, I think Costolo has proven himself more than worthy. Downtime is rare. He harpooned the #FailWhale. He steered Twitter through another big era of growth, if never quite as big as expected. He expanded the advertising revenue significantly. He grew employment from 300 to 3,900.

And, of course, he took Twitter public.

The bad

Starting last year, the drumbeats calling for Costolo to go as CEO began to grow. There was simply too much that wasn’t getting fixed. The feeling that Twitter was not seizing all of its opportunities still lingered.

Twitter’s user growth was slowing. For a company whose service gets pimped for free across every media platform, that’s sort of remarkable, really. While we can debate whether Twitter is mainstream, the fact remains that it’s still too hard for new users to understand.

Five years should have been enough time to fix that problem.

Meanwhile, revenues have been growing, but after nine years, Twitter still doesn’t turn a profit. And that leads to constant speculation that it might acquired by Google or someone else. Not super for morale.

Costolo almost completely overhauled the upper ranks of management last year, and yet some of the fundamental problems didn’t seen to get fixed. At some point, there’s only one head left to chop off.

And while it’s not worth repeating in detail, if you read (or skim) the Sacca blog post, you get a good feeling for perhaps Twitter’s biggest shortcoming: Its inability to consistently develop new products that expand use and engagement.

Facebook is great at this. And while Twitter has finally seemed to be picking up the pace (lifting the character limit on direct messages yesterday), innovation on the platform still seems more driven by users.

The mess

Of course, if one wants to be more optimistic, one could call this, “The opportunity.”

After all, Twitter still has incredible recognition around the world. At the same time, as someone who has used Twitter since December 2007, I find myself turning to it much less than Facebook. For some live events, it’s great. But even then, the flood of tweets can be a bit useless and chaotic.

And in terms of conversations, it’s nearly impossible to follow them. Twitter recently introduced a tweak that helps a little. But I still find it too difficult and a turn-off. And if someone starts flaming you and lots of other people starting jumping in, following the madness can be impossible.

While Twitter was tweaking direct messaging for several years, Facebook spun off Messenger into a whole different product that has 700 million users. And now Messenger is its own platform. Messaging was just one of Twitter’s many huge missed opportunities.

Twitter has Periscope for live-streaming video, which holds a lot of potential. Vine is going nicely. It needs to rethink some kind of music. It needs to make following live events far easier. And it needs to leverage the wealth of content we’ve already given it and help us find the best stuff.

The company will also need to figure out just what role it wants developers to play. When it says it wants their help, then also cuts off some at the knees by limiting access to data, it can be quite confusing.

There are a thousand possibilities here. The potential remains huge. But I think it’s essential Twitter find a CEO with a proven track record as a product person. Someone who can manage the seemingly impossible balancing act of expanding Twitter’s utility while simplifying the experience.

If not, well, investment bankers are no doubt smelling blood in the water, and pitching tents outside Twitter’s San Francisco headquarters.