Earlier this month, Apple’s long-running list of Top Podcasts began to exhibit some unusual issues — no-name podcasts vaulting over popular, well-established ones — but the company appeared to quickly fix its chart. Unfortunately, the problems have popped up again, and an analysis from podcast industry tracker Chartable suggests that paid click farms are now gaming the list, which it calls “the closest thing to the Billboard Top 100 in the podcast world.”

In theory, Apple’s podcast popularity rankings might not matter — podcasts are free, and Apple’s only one source of such rankings. But after introducing its Podcast Directory in 2005, Apple became the world’s largest aggregator of such programming, and its rankings serve two purposes: showing listeners what’s hot, and helping advertisers determine which shows to support, thereby keeping their creators afloat.

The core problem is that Apple’s Top Podcasts chart appears to use a poor and easily manipulated ranking metric. Chartable believes that it’s based entirely upon a podcast’s total number of new subscribers over the past week, with weights assigned to movement in the past one to three days. Additionally, Apple displays a “listeners also subscribed to” list for each podcast, which helps users find related content, as well as providing a window into potential fraud.

While Chartable stops short of saying that one particularly unusual podcast, Bulletproof Real Estate, has certainly cheated its way to the top of the chart, it points out a flurry of suspicious activity that appeared to start on October 1:

  • As of September 30, the show ranked #218 on the charts.
  • By October 7, the show was #1, displacing the top-ranked podcast Serial, despite no individual episode achieving a higher rank than #271.
  • “Listeners also subscribed to” showed Bulletproof Real Estate subscribers also selecting multiple and seemingly unrelated podcasts from a podcasting network called “Warrior Empire.”
  • Additional podcasts seemingly came out of nowhere to take places in the top 10, unseating well-established podcasts, with unusual activity noted across clusters of Warrior Empire offerings, as well as other “also subscribed to” podcasts.

After deeper analysis, Chartable concluded that “something deeply strange is happening across a sizable number of podcasts,” likely click farming, whereby clusters of devices and accounts are operated by one source “to artificially boost subscriber counts.” It contrasts its findings for Bulletproof Real Estate with legitimately popular podcasts, which have individually popular episodes, a history of many ratings, and popularity across multiple countries, charts, and podcast players. In short, there’s ample evidence that the charts are broken and that rankings can effectively be bought.

The question is whether Apple can or will provide a meaningfully solution to the problem. Thirteen years after the Podcast Directory debuted in iTunes, the feature probably hasn’t made the company much money, and isn’t likely to do so in the future. However, Apple has gone out of its way to work with some podcasters at certain events, and it did take steps to fix its charts after manipulation was reported in early October. In August, Apple CEO Tim Cook personally intervened to pull Infowars podcasts from the directory.

Finding a way to measure popularity that is more reliable than merely tracking new subscribers would be helpful. Unfortunately, Apple’s current role as a directory rather than a server for podcast content may make tracing actual downloads challenging — not impossible, but unlikely.