(Editors note: Auren Hoffman, who runs a Silicon Valleys start-up called RapLeaf, writes below about the Black Hat Tax on consumer Internet businesses. It is timely, because spam is as bad as ever. Yesterday, it emerged that Comcast even blacklisted the innocent email group called The WELL, despite numerous efforts by the group to be rated “legitimate.”)

A couple of months back, I was grabbing a smoothie with James Currier and was going on and on about consumer internet and how it’s so great and easy. But then James chimed in with something very interesting and profound… something I have not been able to shake since:

“Most consumer Internet sites today have an inherent tax of about 25% on them due to scamming, phishing, hacking, and government requests. That 25% is based on time and mindshare. And the 25% is only going to get worse. This is troubling.”

This is the Black Hat Tax (if someone wants to contribute this to Wikipedia, by all means go ahead). This Black Hat Tax is something we’re certainly facing at RapLeaf.

PayPal’s start-up story is an example • they combated fraud in an ongoing intense battle that consumed everyone in PayPal, from the engineering team to the CEO. If you’re curious about the details, you should check out PayPal Wars.

Since James and I spoke, I have spent some time talking to consumer start-ups as well as Fortune 500 companies. All of these companies, particularly those in the dating and social network space, are increasingly spending their time thwarting bad guys.

After surveying most dating sites, I found that one of the top three issues combating fraud on some level. A frequent scam is to contact an unsuspecting middle-aged man with a profile of a good looking woman saying “my husband is beating me here in [insert international city], please send $2000 so I can buy a plane ticket and escape.” The unsuspecting chap sends the money only to never hear from the person again. Apparently there are scam factories in places like the Philippines that have thousands of people working to scam unsuspecting dupes in this way.

Government requests can also constrain time and resources, particularly for social networking companies. A particular social networking site CIO was consumed with a THREE day government request to obtain information on someone promoting pedophilia. Not only did this IT person have to get a lot of information to the federal authorities, but he had to ensure that the information was backed-up and could not be erased for at least three years. Oh let’s not forget that the work was disgusting as the person had to sift through some really disturbing pictures.
Most consumer Internet companies send out a ton of email, and if those pesky spam filters flag you as spam, your users will never see your messages (this is considered a “false positive”). The time it takes to get white-listed is time-consuming and frustrating. Yahoo alone requires you to fill out a 3-4 page detailed questionnaire just to avoid false positives with their email setup. RapLeaf is actually going through this pain right now. Thank you spammers for wasting our time.

What’s troubling is that these nefarious characters are getting more sophisticated too. And while thousands of sites are working feverishly to implement best security practices, the bad guys only need to find one hole.

So while I stand by my original point • that launching a consumer internet company is really easy • maintaining that site over time is becoming increasingly difficult and expensive. The Black Hat Tax is costing consumer Internet companies billions of dollars and it is a much higher percentage tax than off-line brick-and-mortar shops invest in security and anti-fraud matters.

As stated before, James actually thinks the Black Hat Tax is 25% for most consumer Internet companies right now, which I think is a fair assessment given that over $20 billion is spent on spam alone. This number is measured in terms of productivity losses, as a company’s engineering and IT team can spend up to 25% of its time dealing with black hat issues (combating fraud, dealing with spam, fighting bots, preventing phishing, etc). Most executives at consumer internet companies spend at least a day a week on black hats. But more importantly, executive meetings and strategy sessions are increasingly dominated by these issues.

If you think about it, that means over 25% of your engineering and management time is about preventing fraud and thwarting the bad guys, rather than improving the offering. What an onerous tax! And what’s worse? James believes these numbers are only getting worse.