Why craigslist booted Oodle… and more to come?

Here’s our Merc story today (free registration) about why San Francisco classifieds company craigslist asked Oodle to stop crawling its site for listings, and explores the move’s wider significance.

Turns out, craigslist’s terms of use (see bottom of this page for relevant snippets) limits Oodle and other sites that want to crawl its content to a maximum of 100 listings, and Oodle went way beyond that. Others do too, by the way, including Jobster, Indeed and SimplyHired.

But craigslist Chief Executive Jim Buckmaster told us he singled out Oodle for several reasons. He said some users had written in asking why Oodle was using craigslist listings. “Folks seem to resent anyone looking to profit off the craigslist community,” he said.

Also, Oodle was particularly…

aggressive. Buckmaster said its robotic activity was “very resource intensive” for craigslist. “Oodle drew our attention by scraping more of our listings than any other such company, and by using our name in their marketing and press releases,” he said.

Something we couldn’t fit into the story was our conversation with the folks at Indeed and SimplyHired, two of the job sites that also crawl craigslist to take listings for excerpting. What is apparent is none of these guys, including Oodle’s Donato, have a very open relationship with craiglist’s Buckmaster. They seemed to have not talked directly with him, but Buckmaster didn’t seem particularly interested in talking with them either. Oodle’s Donato told us he had emailed Buckmaster, trying to arrange a way to placate craigslist by crawling it at more convenient hours, such as slower periods during the night. But soon the correspondence dropped off.

When told of craigslists’ 100 listing maximum, the Indeed and SimplyHired guys seemed surprised at the limit and suggested craigslist must have recently changed its terms of use. But no, Buckmaster said the most recent change was about four months ago.

SimplyHired’s head of marketing, Dave McClure, when told about the terms of use, said “We probably need to talk with them.” But he said he was surprised that craigslist was getting upset. “It’s a bit curious coming from craigslist.” He said everything was “hunky dory” with other sites that SimplyHired gets listing excerpts from, including Monster and Careerbuilder.

Paul Forster, chief executive of Indeed, said he hadn’t received any complaints from craigslist, but was also unaware of the 100 listing maximum. Meanwhile, Jobster, who we also tried to contact, responded with an automated reply Wednesday saying they would get back to us within the next business day. But we still haven’t heard from them.

Relevant snippets of craigslists’s terms of use (emphasis ours):

–This license does not include any collection, aggregation, copying, duplication, display or derivative use of the Service nor any use of data mining, robots, spiders, or similar data gathering and extraction tools for any purpose unless expressly permitted by craigslist

–craigslist permits you to display on your website, or create a hyperlink on your website to, individual postings on the Service so long as such use is for noncommercial and/or news reporting purposes only (e.g., for use in personal web blogs or personal online media).

–If the total number of such postings displayed or linked to on your website exceeds one hundred (100) postings, your use will be presumed to be in violation of these Terms, absent express permission granted by craigslist to do so