At least, according to Vancouver-based project founder and core architect Rastin Mehr.

Mehr’s open-source framework for making everything social won a spot in the current Mozilla WebFWD accelerator cohort, which the organization best known for the Firefox browser created to help open-source organizations build successful companies.

It’s like an MBA program for a startup, says Mehr.

“Anahita is a social networking platform framework that we’ve been building for the last four years,” Mehr told me yesterday. “We think in the future a lot of the web services are going to run on some kind of social networking structure as an underlying layer.”

That sounds a lot like Facebook and Mark Zuckerberg’s vision of building a social layer for the Internet. And the world’s biggest social network has made a lot of that vision reality by becoming the first social network with one billion users.

How can Anahita compete? Primarily by not competing.

“Mark Zuckerberg has done it but it’s not an internet social layer, it’s a Facebook social layer,” Mehr says. “What Anahita provides is a platform for building social apps. We have all the tools we need to build and validate apps faster than anyone else.”

Anahita focuses on helping developers make their internal and external applications for web, enterprise, and startups social. Not by copying App.net, which is primarily the plumbing to which you can attach a social frontend, and not by layering in social on top of an existing framework, like a Gigya gamification solution or a Janrain social login integration … but by building an app from the ground up within a social ecosystem.

“You can’t add social things, you have to build on top of social,” says Mehr. “We provide all the generic building blocks … for a network of salespeople, or the location of products, or which people have been working on those products, or anything.”

That’s because at its core, Anahita consists of three objects out of which developers can build the entire universe of their application: Nodes, Graphs, and Stories. An Actor node, for instance, has an identity, a story to tell, a graph of apps that it can use, and a social graph of people to which it is connected.

Source: John Koetsier

Cofounder Ash Sanieyan.

“Once you define these, you can build a person, build a group, build a spaceship, or build a Cylon,” Mehr says with a smile. “These are all actors … what you see as a profile on a social media site is essentially an actor.”

The framework is LAMP technology, built in PHP and intended to be released on Linux. With it, clients and users have built online learning portals, websites, Internal social networks, social e-commerce experiences, online magazines, and niche social networks.

But building a business with the open source software is another matter. That’s where WebFWD comes in.

“You always have holes, weaknesses … but when you go thru WebFWD, they help you patch all these missing elements,” Mehr told me. “You learn what path to go to find a business model.”

Anahita has had a freemium model, allowing developers to download the software for free, but charging for premium support via annual subscriptions. The challenge that Mehr and cofounder Ash Sanieyan faced, however, was focusing an infrastructure that can literally be used to build almost anything to a finer point that business partners, users, and investors could grasp.

Again, WebFWD helped.

“They work with you on how to pitch … they bring public speakers and pitching coaches, and on the very first day we had to pitch 4 times to four different audiences,” Mehr said. “And they rip you apart in public, so you develop a very thick skin by the time you come out.”

The ripping and the thick skin have come in handy. Since joining WebFWD, Mehr and Sanieyan have pitched four VCs, built their network of contacts in the Valley, and become much better prepared to meet potential investors.

Although WebFWD is an accelerator, it doesn’t provide capital and doesn’t take equity. Instead, it operates simply to give back to the community by helping open-source innovators build successful companies while staying true to open-source ideals. Anahita’s goal is to build that kind of successful company by becoming the go-to infrastructure for anyone to build online social software — just as Linux has become the go-to infrastructure on which to build server platforms, Mac OS X, Android, and more.

I asked Mehr if WebFWD had paid the team’s expenses to make the trip down to San Francisco.

“No,” he said. “But they give you a lot of free food.”

photo credit: tobimcfly via photopin

Development testing service Coverity’s annual scan report, which is based on data from almost 500 software projects with a total of over 450 million lines of code, says that almost 230,000 defects were found and fixed. And while the average defect density per thousand lines of code was almost identical between open source and proprietary, there was an interesting diversion in the results.

Open source projects, Coverity says, tend to have .69 bugs per thousand lines of code, virtually the same as proprietary software, which tends to have .68 errors per thousand lines. But large closed-source projects — over one million lines of code — tend to have 33 percent fewer errors than small closed-source projects, with .66 errors over each thousand lines of larger projects compared to .98 in smaller projects. And small open source projects have a massive 70 percent fewer errors than large open source software, with only .44 defects compared to .75.

The difference, according to Coverity, is that small open source projects are labors of love by individual developers or small teams, who carefully comb through their code to reduce errors. Large open source projects, on the other hand, tend to lack standardized processes to ensure code quality, and so the error rate increases.

In commercial or closed-source software, developers experience almost the opposite conditions. Large projects tend to have well-defined formal testing processes, which ensure higher code quality, and small projects tend to be hasty, quick endeavors that show the effects of growing pains, as no standardized testing is in place.

In other words, if you’re looking for bug-free apps, look for a small open source project or a large proprietary piece of software, because those have the best chance of having few defects and high overall code quality.

All of the data in infographic form:

photo credit: gui.tavares via photopin cc

Rob Isaac is a New Zealand-based developer, technical analyst, and consultant. After the news that Google would be creating its own, Chrome-specific version of the Webkit browser rendering engine, called Blink, Isaac created this “translation” of Google’s developer FAQ for Blink. This FAQ was originally published on his website.

1 Why is Chrome spawning a new browser engine?

The WebKit maintainers wouldn’t let us attack Apple directly, by changing WebKit in ways that would make it perform badly on OS X and iOS.

Because they share a rendering engine, developer effort to ensure Chrome compatibility currently benefits Apple platforms for free. To prevent this, we must make Chrome and WebKit behave differently.

1.1 What sorts of things should I expect from Chrome?

Nothing yet. This is a political move, not a technical one.

However, while the Chrome user interface will not change in any significant way, we will be silently overwriting all existing installations of Chrome with our new rendering engine without your knowledge or consent.

1.2 Is this new browser engine going to fragment the web platform’s compatibility more?

Yes.

We intend to distract people from this obvious problem by continually implying that our as-yet unwritten replacement is somehow much better and more sophisticated than the rendering engine that until yesterday was more than good enough to permit us to achieve total dominance of the Windows desktop browsing market in less than two years.

This strategy has worked extremely well for Netscape, Microsoft, Apple and us in previous iterations of the browser wars, and we firmly believe that everyone in this industry was born yesterday and they will not recognise this for the total bullshit it so clearly is.

1.3 Hold up, isn’t more browsers sharing WebKit better for compatibility?

Yes. See 1.

1.4 How does this affect web standards?

We have sufficient market share on the desktop that a few months from now, we will be in a position to unilaterally dictate them.

We hope to leverage this control to achieve the same dominance in mobile eventually.

1.5 Will we see a -chrome vendor prefix now?

No. See 1.4.

1.6 So we have an even more fragmented mobile WebKit story?

Yes.

We encourage you to adopt Chrome on Android for your mobile browsing needs.

1.7 What’s stopping Chrome from shipping proprietary features?

Nothing.

1.8 Is this just a ruse to land the Dart VM or Native Client?

We’ve decided to avoid discussing unpopular topics like those for the time being.

1.9 What should we expect to see from Chrome and Blink in the next 12 months? What about the long term?

We have a direct strategic interest in destroying Apple’s mobile platforms because their lack of participation in our advertising and social ecosystems does not benefit our long term goals. You should expect Chrome and Blink changes in the short term to be focused in this direction.

In the longer term, we aim to have sufficient control over the installed base of web browsers to dictate whatever conditions we consider most appropriate to our business goals at the time.

1.10 Is this going to be open source?

Not really.

While you can certainly read the source code, we’re fully aware that actually tracking and understanding a modern HTML renderer is extremely difficult. In addition, the first changes we will make are intended specifically to break compatibility with WebKit, so the only organisation with sufficient resources to track our changes will no longer be able to do so.

In practice, this allows us to call the project “open” while simultaneously ensuring Google will be the only effective contributor to the Chrome and Blink source now and in the future. We’ve had enormous success co-opting the language of open source in the past to imply our products are better, and we aim to continue with that strategy.

1.11 Opera recently announced they adopted Chromium for their browsers. What’s their plan?

Opera have such a tiny market share that they have no choice other than to follow whatever strategy Chromium adopts. In this case, it means they will adopt the Blink renderer as quickly as possible.

1.12 Why is this is good for me as a web developer?

It isn’t. Our primary goal is to use your development efforts as leverage against our competitors. See 1.9.

photo credit: nitot via photopin cc

July 9-10, 2013
San Francisco, CA

Tickets On Sale Now

Google pledged to not use its arsenal of patent weapons offensively today, taking a stand on open source and patents that is anti-patent troll, pro-competition, and pro-freedom to create, innovate, and code.

At least, on a whopping ten patents.

Google says it is “committed to promoting innovation” and that therefore, it is “pledging the free use of certain of its patents” used in both software and hardware products. Open systems eventually win, Google says, but open platforms like Android have faced increasing levels of attack via what Google calls “bogus patents” in the hands of Apple, Microsoft, and Oracle. Therefore, to protect innovation and everyone’s ability to deliver great products and services, Google says, it’s announcing the Open Patent Non-Assertion Pledge.

But don’t take that to mean that Apple is now in the clear.

The ten lonely patents Google is starting with are not mobile patents, they aren’t focused on Android, and are therefore not in the most litigious arena of the current era which has seen Apple and Google proxy Samsung pitted in numerous patent infringement cases across the globe: smartphones. Don’t forget that Google-owned Motorola sued Apple with its 17,000-strong patent portfolio just last year in an attempt to ban the import of pretty much Apple’s entire product line: iPhones, iPads, and iPod touches.

Oh, and one more massive caveat? The project or product using Google’s patented technologies must be open source. Good luck with that one, Apple and Microsoft.

Still, it’s a start. And it’s a noble endeavor.

But it’s interesting, even odd, that Google chose to cite the Android-under-attack example, because right now, given the patents that Google has added to the pledge, what this is really about is open source software projects like Linux or Apache. The ten patents Google is pledging are all about data management, data analysis, and data processing. Which means that technologies like Hadoop (an open source technology for processing large amounts of data across multiple distributed servers) which will not now have to worry about stepping on Google’s patented toes when building in new capabilities.

And that’s all good and excellent.

But it’s not going to do anything for Android or to create patent peace between warring competitors like Apple and Samsung, or Apple and Google for that matter. To do that, the pledge would need to be broadened to closed-source products and tailored a little to be more comfortable to for-profit enterprises.

This is at least a start, and Google will add more patents to the pool over time.

But it’s also probably reflective of an internal Google ethos that not only believes that open systems win, but that in a world where open systems increasingly dominate, Google wins.

Here’s Google’s open patent non-assertion pledge:

Google promises to each person or entity that develops, distributes or uses Free or Open Source Software (a “Pledge Recipient”) that Google will not bring a lawsuit or other legal proceeding against a Pledge Recipient for patent infringement under any Pledged Patents based on the Pledge Recipient’s (i) development, manufacture, use, sale, offer for sale, lease, license, exportation, importation or distribution of any Free or Open Source Software, or (ii) internal-only use of Free or Open Software, either as obtained by Pledge Recipient or as modified by Pledge Recipient, in standalone form or combined with hardware or with any other software (“Internal-Only Use”). The preceding Pledge does not apply to any infringement of the Pledged Patents by hardware or by software that is not Free or Open Source Software, or by Free or Open Source Software combined with special purpose hardware or with software that is not Free or Open Source Software (except Internal-Only Use).

It is Google’s intent that the Pledge be legally binding, irrevocable (except as otherwise provided under “Defensive Termination” below) and enforceable against Google and entities controlled by Google, and their successors and assigns. Thus, Google will require any person or entity to whom it sells or transfers any of the Pledged Patents to agree, in writing, to abide by the Pledge and to place a similar requirement on any subsequent transferees to do the same.

The Pledge is not an assurance that any of the Pledged Patents cover any particular software or hardware or are enforceable, that the Pledged Patents are all patents that do or may cover any particular Free or Open Source Software, that any activities covered by the Pledge will not infringe patents or other intellectual property rights of a third party, or that Google will add any other patents to the list of Pledged Patents. Except as expressly stated in the Pledge, no other rights are waived or granted by Google or received by a Pledge Recipient, whether by implication, estoppel, or otherwise.

photo credit: Cre8iveDoodles ~ Has been ill…Back Soon! via photopin cc

A new report shows Linux experience is in greater demand — and, hiring managers say, harder to find — than in past years.

The 2013 Linux Jobs Report, released today by the Linux Foundation, surveyed 850 hiring managers and 2,600 Linus pros and found that Linux might be a good area of focus for aspiring techsters.

Dice’s annual salary survey shows that salaries for Linux folks are rising at double the rate of other tech salaries. Yet the number of practicing Linux pros seems to be dwindling.

Hiring managers in the Linux survey said they were finding it difficult to source good Linux talent — 90 percent said so this year as opposed to 80 percent last year. And 93 percent of those 850 hiring managers said they will be hiring a Linux person before Q3 rolls around, a 4 percent increase from last year’s survey.

Of course, with all that demand, currently employed Linux professionals are feeling like the belles of the ball, with 75 percent fielding cold calls from recruiters in the past six months. Linux pros told the foundation that when considering a move, they do take into account work-life balance (read: a normal, 40-hour work week would be nice) and work-from-home options, but the biggest pull is all about the Benjamins, i.e., extremely competitive salaries.

And the Linux job title hiring companies are most eager to fill? The humble and infinitely flexible sysadmin. The Linux Foundation says this is representative of “the growth of Linux in the enterprise to support cloud computing and big data.”

Here’s an infographic the foundation whipped up to show off other survey results:

But it’s also not often that someone invents an open-source game development platform that can make Minecraft-style games that will, with a little luck, soon be running on web browsers everywhere: laptops, Android phones, and iPhones.

Twenty-two days ago, Max Ogden was a bored developer whose latest startup, Gather, was not, shall we say, making a lot of hay. So he was looking for something new to occupy his time.

“I came back from Europe in the winter working on a bunch of little indoor projects — it’s been freezing here in the Bay area,” Ogden told me today. “Then I saw the Minecraft documentary right after Christmas.”

He had given the wildly popular sandbox builder to his 10- and 11-year-old nephews for Christmas, and they loved it, so he started to think about building something for it — a mod perhaps, or an extension of Minecraft. And was startled to find that Minecraft was totally closed source, with no API (though one is coming soon). Instead, he discovered that developers who want to mod Minecraft hack it, decompile the code, build their mods, and then release them … to be broken with every new version of the game.

A forest in Voxel.

So he came up with the idea of building not just Minecraft, but the toolset to build any Minecraft-like game, all inside the browser, using common old Javascript and OpenGL, an industry-standard toolkit for building interactive 2-D and 3-D applications.

“OpenGL has been around for a long time, but Chrome became the first browser just last month that lets you take over someone’s mouse pointer, which is totally needed for games,” Ogden said.

Having found his project, Ogden did nothing else for the past three weeks, staying up late, “going crazy,” and cranking out code. He found numerous little snippets of code that others had worked on that helped, speeding the process, and brought in a friend, James Halliday, to help solve some particularly tough problems.

“In about two days of working with James, it all came together,” Ogden said. “We had something that looked like a game, and we looked at each other and said: ‘Holy cow, that was quick.’”

The result that looked like a game was Voxel.js. It’s not precisely a game itself, but a game-building toolkit for modern browsers. You can try early examples of game environments built with it already, right in your (Chrome) browser, including one with a virtual drone simulator.

A Minecraft skin

“Kyle Robinson, who runs hackathons for AR quadricopter drones, built a virtual drone simulator for it,” Odgen told me, marveling. “It has a command line, you can tell it to take off, spin, and it has a little camera to ‘see’ the terrain that shows up like an iPad in the game.”

Which is all very meta: watching a landscape of a virtual world via a virtual drone that you control in a game running inside a browser on your computer. It sounds impressive, until you hear that another acquaintance of Odgen is planning to run Voxel.js on a Raspberry Pi running Firefox OS (coming soon) on a real AR quadricopter videoing the actual landscape while also running the game and viewing a virtual landscape.

Just try to wrap your head around that.

All the code is open source, and Odgen is welcoming any and all hackers to make contributions, adding modules like water, better physics, or creatures. Seven already have. All of which could soon have the game-building environment running on iPhones as well as Android smartphones. While Android should be relatively easy as soon as Google updates mobile Chrome to support OpenGL more fully, iOS is another story. It turns out that Apple supports OpenGL in mobile Safari, but for iAds only.

“So you can run WebGL on iPhone … if you make your own browser,” he said.

Fortunately, a project named Impact is working on a solution, rendering Javascript to iOS’s native Objective-C language.

All of which means that a Minecraft-like game built with Voxel.js could conceivably run in a browser on an iPhone. And on an Android smartphone. And in your web browser on your laptop. And, if you really, really, really must, on a tiny little $35 Raspberry Pi, flying high on a quadricopter above the drunken old ladies of Oakland.

Which, frankly, would be awesome.

July 9-10, 2013
San Francisco, CA

Tickets On Sale Now

Why is the founder and former president of the Free Software Foundation of Europe currently leading a for-profit software company in the groupware space?

I asked Georg Greve, a former physicist and nanotechnologist, exactly that question.

“There are three pillars of productivity for modern knowledge workers,” Greve replied. “One is the browser, the second is office applications, and the third is groupware. Free software has tackled the first two very well … but on the groupware side, almost nothing has happened.”

Greve is the CEO of Kolab Systems, which produces Kolab groupware, an enterprise-scale email, calendaring, contact management, and task management suite that is fully open-source, freely available, and interoperable with multiple web, desktop, and tablet clients. The company’s clients include Fortune 500 companies — which Greve cannot contractually name — who have 60,000 employees using Kolab, as well as the entire school system of Bazel, Switzerland, and the German Federal Office for Information Security.

In fact, that’s where the solution originated: an open source German government software project initiated in 2001.

Source: Kolab

Kolab works on Windows, Mac, or Linux.

“They needed a fully audit able, fully open-source solution that was designed around security awareness,” Greve said from Switzerland, where he lives and works. “But they couldn’t find anything.”

So they built one — or at least the foundations of what would become a full groupware solution. Most solutions, Greve says, come to the enterprise with “freedom stripped.” Kolab comes free not just in the standard beer sense — free to use — but also in the speech sense: free to alter, modify, and adapt.

Greve joined as CEO in 2010 when it became clear that the companies who had contracted with the German government to provide the solution needed focused attention on Kolab. Version 3, with an almost entirely refactored codebase, a completely new storage layer, and 100 percent open standards, is being released in January.

“We’re using IMAP as the database, which means we get insane scalability,” Georg said, adding that the company had partnered with Opera to get the project completed. “We’ve also built in full mobile connectivity … all the cloudy stuff … and added support for a whole range of native platforms: Mozilla Thunderbird, Lightning, Outlook … on Windows, Mac, and Linux.”

The data format is xCal and xCard, and enterprise clients can interact with the data via an open API, integrating their groupware solution into any other solution or even creating their own administration clients.

Kolab is bootstrapped with a little cash from Greve, other executives, and others and has a subscription revenue model similar to RedHat: training, certification, service level agreements, prioritized updates and fixes, and support.

It was to provide that third pillar of productivity that Georg joined Kolab from the Free Software Foundation of Europe. OpenOffice and Firefox area available for the other two; The Gimp, WordPress, and of course Linux itself, plus many other open source projects, provide options in other areas. But groupware was a bit of a green field.

Of course, it was also to fulfill an unusual long-time dream for the open-source believer:

“I had always wanted to go into business.”

photo credit: AGoK via photopin cc

In a recent blog post the lords of the universe over at code-hosting community GitHub laid out the biggest new open-source projects of 2012.

The GitHubbers track notability of OSS projects in a couple ways. First, they look at the number or “stars” a project has — that is, how many people find the project interesting enough to keep tabs on in a passive sense.

Second, GitHub also keeps track of how many active contributors a project has.

The company’s record-keeping for 2012 tallies up the number of new-this-year OSS projects with the most stars and the number of contributors that 2012 brought to all OSS projects, regardless of when they were started.

Without further ado, we give you — the 2012 Hubbies!*

First up, the new-in-2012 rising stars:

And here are the projects that racked up the greatest numbers of unique contributors in 2012:

*This is a disgusting term of our own devising. GitHub is not to be held responsible.

Raspberry Pi announced the Pi Store today on its blog, saying that the app store will make it easier for newbies to get into the Raspberry experience, and will also provide the ability to for people to share their creations … maybe even making a little money on the way.

The tiny computer from the non-profit Raspberry Pi foundation first shipped in April of this year, but only reached volume in July. The Pi is a tiny computer that fits in the palm of your hand, but contains a 700 MHz processor, 512 GB of RAM, and Ethernet, HDMI, USB, and audio ports. It runs a version of Linux — including one titled “Raspbian wheezy” — stored on a SD card, like one in your camera.

Raspberry Pi app store

Current apps on the store include Freeciv, an open source sim game, Iridium Rising, a 3-D space fighter game which is also free but not open source, and LibreOffice, a full Office-compatible productivity suite. 23 titles were announced this morning, and two more have already been added today.

Developers who want to submit apps or content to the Pi Store can upload compiled binaries, raw Python code, or simply pictures, audio, or video. Pi owners will find that the store’s recommendation engine will show them the best content that, over time, will become more customized to their likes and dislikes.

The Pi Store is visible online here, and is built into updated versions of the latest Raspberry Pi operating systems. Technically capable people can also add it to existing installs by jumping into the command line and typing a Debian-style command: sudo apt-get update && sudo apt-get install pistore

Adding apps to Pi will definitely make it more usable and useful … and could make the operating system as beautifully functional as some have made their Raspberry Pi cases.

Ubuntu is one of the most popular versions of Linux. Stallman is talking about its new network search feature, which he believes spies on the users:

Ubuntu, a widely used and influential GNU/Linux distribution, has installed surveillance code. When the user searches her own local files for a string using the Ubuntu desktop, Ubuntu sends that string to one of Canonical’s servers. (Canonical is the company that develops Ubuntu.)

Canonical CEO Mark Shuttleworth talked about the feature on his personal blog, prophetically subtitled “here be dragons.” Essentially, searching your files on your computer is also, by default, an online search. That online search includes potentially relevant results from Amazon, and if you buy something, Canonical gets a cut. This is not advertising, according to Shuttleworth:

“We’re not putting ads in Ubuntu. We’re integrating online scope results into the home lens of the dash.”

That extremely fine, perhaps microscopic distinction has escaped some of Canonical’s customers, who are wondering why, in the first place, a desktop search should be integrated with an online search, and why, in the second place, that online search wouldn’t be a Google search instead of a online retailer.

As JunCTionS says in a comment on Shuttleworth’s blog post:

Sorry if this is clear to everyone else, but you don’t seem to mention any typical websearch engine. I imagine there are even more Ubuntu users that use Google than those that use Amazon. Will it also search Google?

… it sounds to me that this would be more useful than an Amazon search engine.

For Stallman, however, the core issue is not advertising, although that’s certainly unwelcome. The core issue is the exchange of personal user information … even though Canonical does not send any personal information to Amazon, running the Amazon search query on its own servers based on information that it retains.

That has failed to mollify RMS, who wrote that “it is just as bad for Canonical to collect your personal information as it would have been for Amazon to collect it.”

Shuttleworth’s answer seems to be: just trust us. After all, we control your machine anyways — we have administrator privileges on your computer:

We are not telling Amazon what you are searching for. Your anonymity is preserved because we handle the query on your behalf.

Don’t trust us? Erm, we have root. You do trust us with your data already. You trust us not to screw up on your machine with every update. You trust Debian, and you trust a large swathe of the open source community. And most importantly, you trust us to address it when, being human, we err.

That is not very compelling or simpatico.

In a post on the Canonical blog today, the company addressed the issue again, at least to a degree. After running through the new capabilities — searches for the Beatles will bring up their music on Amazon, where it can be instantly purchased without opening a browser — Canonical says that privacy has been a primary concern while developing this service:

Privacy is extremely important to Canonical. The data we collect is not user-identifiable (we automatically anonymize user logs and that information is never available to the teams delivering services to end users), we make users aware of what data will be collected and which third party services will be queried through a notice right in the Dash, and we only collect data that allows us to deliver a great search experience to Ubuntu users.  We also recognize that there is always a minority of users who prefer complete data protection, often choosing to avoid services like Google, Facebook or Twitter for those reasons – and for those users, we have made it dead easy to switch the online search tools off with a simple toggle in settings.

Aside from the issue of how unusual it would be for someone to be searching their own computer for commercially-useful queries like “the beatles,” or “Lord of the Rings movie,” this is unlikely to satisfy privacy advocates.

And it most certainly will not satisfy RMS.

photo credit: Maurizio Scorianz via photopin cc, Hat tip: Ars Technica

It’s unlikely that you, Dear Reader, will ever experience the big-data challenges or infrastructure demands of a Facebook-scale, billion-user software platform. But if you do, you’ll be happy to know that the company is sharing a sip of its secret sauce today.

Called Corona, the aforementioned sip contains a more efficient way to handle scheduling for Apache Hadoop MapReduce.

Facebook’s been Hadoop/MapReduce powered for some time now, “and that served us well for several years,” writes the Corona team today on the company’s dev blog. “But by early 2011, we started reaching the limits of that system. … It was pretty clear that we would ultimately need a better scheduling framework.”

The team briefly flirted with YARN as an alternative, but there were too many incompatibilities with Facebook’s version of HDFS, and the team was fairly sure YARN couldn’t handle Facebook-scale workloads.

So the Facebook infrastructure team started working on a MapReduce scheduler that would scale better, be easier to upgrade, have lower latency for small jobs, make better use of clusters, and schedule “based on actual task resource requirements rather than a count of map and reduce tasks.”

The result is Corona, a push-based scheduling framework that removes cluster resource management from job coordination, tracking the nodes and free resources continuously for minimal latency.

And if that sentence totally made sense to you, you can check out Corona on GitHub now — that’s the version Facebook is currently running in production.

“Corona has allowed us to achieve our initial goals of greater scalability, lower latency, no-downtime upgrades, and better resource management,” the team concluded.

“It has also helped us achieve better scheduling fairness, faster job restartability, a cleaner code base, and the ability to integrate with other systems for scheduling.”

Facebook will be making upgrades to Corona as time goes by, since it’s now a core part of the company’s infrastructure.

Based on a tip from a reader, it looks like Authentec actually has patched the security flaw without announcing that fact via its main website.

The company’s support website has the information, and a download available for a new version that was apparently released in September. The release note for that version includes this line: “Changed passport encryption implementation.”

Unfortunately, because the product was removed from the main website and the person I spoke to at Authentec simply told me that it was discontinued — not that it was patched — I proceeded with the story. My apologies.

Please see a follow-up story here.

Apple-owned security company Authentec has still not patched a massive vulnerability in its Windows software more than a month after it was first discovered by ElcomSoft. Now software that exploits the vulnerability has been released as an open-source project on Github.

In addition, VentureBeat has discovered, Authentec has discontinued both the original security software and its replacement … and deleted the evidence from its website (though not from Google’s cache).

Almost three months ago, Apple bought AuthenTec, a security company that builds sensors for PCs and phones to verify users and protect communications. One of the company’s products was Protector Suite, a secure way to log into Windows machines with your fingerprint.

The only problem? The software stores inadequately encrypted passwords in the Windows Registry. In fact, according to ElemSoft, the passwords were almost in plain text. To put it bluntly, this “security solution” actually made PCs more vulnerable.

Although the problem was discovered in August, Protector Suite has not been patched by Authentec or Apple. Now independent security researchers have crafted a tool, UPEK Protector Suite Password Decrypter, allowing hackers access to account passwords easily. From the release notes:

This is a little .NET 4.0 C# console application that demonstrates how to decrypt Windows logon credentials from registry keys created by UPEK (now AuthenTec)’s Protector Suite software.

Source: Google Cache

Authentec’s TrueSuite, the new (but also-discontinued) Protector Suite.

Apple is not known for its quickness to patch vulnerable software quickly, whether on the Mac App Store or in software that ships with Mac OS X itself … even when the malware is affecting Macs, as Windows experts have noticed. This vulnerability, which affects PCs from Dell, Asus, Samsung, Sony, and Lenovo, was obviously not at the top of the priority list also.

I called Authentec and was told that the company no longer supported UPEK Protector Suite, and that the new software that replaced UPEK was called TrueSuite.

However, the web page for TrueSuite appears to have been deleted from Authentec’s site and now simply redirects to the company’s homepage. In addition, the company’s website says that its “smart sensor” products have been discontinued.

However, Google retains a cache of the TrueSuite page, which confirms that:

TrueSuite® is AuthenTec’s identity management software that is designed to make a fingerprint-enabled PC simpler and more secure, while increasing user convenience and personalization. TrueSuite software is tailored for consumers who demand simplicity, improved usability, and one-touch access to their digital ID and social networks.

So TrueSuite is the new UPEK Protector Suite … except that it is discontinued. And unsupported.

But the question remains: Will Apple live up to its subsidiary’s obligations and patch the holes in its now-legacy software so that users of UPEK Protector Suite can enjoy some level of security?

I’ve asked Apple PR for comment and will update this post when I hear back.

photo credit: 13mx via photopin cc

Sitting across from Facebook’s senior open programs manager David Recordon at the company’s Palo Alto headquarters, we asked the young open-source expert if working on open source software at a proprietary software company presented him with any ethical dilemmas.

“Absolutely not,” he responded. “Look at the amount of open-source software that we do release. We release far more of our infrastructure that we develop than any other company like us.

“And it’s hard. It requires effort to take software for your own environment and make it something that’s useful to others, too. Making a healthy project and accepting contributions takes time and focus.”

The social media company has, without question, taken the time to work on those projects. The hackers at Facebook have done perhaps more than any other single entity to advance and optimize PHP, the programming language on which the network is primarily built.

Facebook’s OSS projects

Recordon can rattle off any number of important OSS projects released by Facebook during his two-year tenure at the company. Before he joined Facebook, Recordon was a founding board member of the OpenID Foundation, a nonprofit devoted to improving the way identities and logins are managed around the web.

Now, Recordon is building the team at Facebook that focuses on OSS and web standards, which includes technologies such as HTML5 and Oauth. He oversees both bringing in and pushing out worthy open-source projects to the company’s engineers and the larger community.

Some of those projects include Phabricator, a suite of web apps for code review and how Facebook does their own development; Cassandra, an open source distributed database management system; the waves-making HipHop, which transforms source code from PHP to C++; the company’s Javascript optimization efforts, called Primer; XHP, a PHP extension which augments the syntax of the language such that XML document fragments become valid PHP expressions; and Thrift, a software framework for scalable cross-language services development, to name but a few.

More remarkable still, all of these diverse and useful projects have come from a relatively small business over the course of less than four years. This is what Facebook engineers are doing in their spare time, folks.

“We value moving fast,” Recordon said. “The rate at which we build infrastructure and make changes, I haven’t seen anything like it. That’s core to our culture.”

Facebook’s hacker culture

Recordon describes the company’s expectations of engineers as “very entrepreneurial. We value the impact a single person or a small team can have. Video calling was built by one engineer and one designer. The messenger app was done by a few engineers. Those groups have a huge impact.”

While Facebook is out looking for those smart, motivated engineers to hack within the company independently or in small groups, hackers themselves are attracted to exactly that kind of opportunity — and working on open-source projects can be a huge selling point in Facebook’s recruitment process.

“Engineers enjoy working on open source,” Recordon said. “Culturally, it allows engineers to talk about what they’re working on publicly. Open-source software also allows people to see the kind of infrastructure we build. It gets people in some areas a taste of the code we’re running in production.”

In addition to working on in-house OSS projects, Facebook engineers are frequently core contributors to other open-source projects, such as Hadoop and Hive. “But those are tools [the data infrastructure team] uses to get their job done,” said Recordon.

Facebook’s OSS workflow

In a hacker-centeric culture that values independent work, how does Facebook, the organization, decide which projects get institutional support and which also get open-sourced? We asked Recordon what the process was like for HipHop.

“HipHop started three years ago, when the site was going through a tremendous growth curve,” Recordon told us. “We needed to optimize the PHP behind the site, and there were three competing projects at the time trying to do that. One was looking at tweaks around PHP itself. Another was working on a Java runtime for PHP, and then there was HipHop. On the risk/reward scale, HipHop was high risk, high reward. As one or two engineers were working on these things, it became clear that HipHop was the right solution, but there was a lot of testing and comparing… Test it, iterate, and code wins. We look at these projects logically.”

Of course, a company built by engineers would place logic at the forefront of every decision, even the decision on whether or not a piece of software should be made publicly available and shareable.

“Companies can see those pieces [of software] as far more core to their business,” said Recordon. “But our ability to serve PHP faster is not core to our business. But cheaper/faster development tools for other companies is a real competitive advantage.” Hence, logic dictates that because there’s no business loss if Facebook’s infrastructure is open-sourced, then open-sourced it should be.

The future of open source

But there’s more to open-source at Facebook than just its back-end software and PHP optimizations; with the Open Compute Project, Facebook is also trying to open-source its server and data center design, and it’s also thinking about what open-source means for data and APIs.

“I think it’s pretty clear there’s no question about whether companies should be using open-source software or not,” said Recordon. “That was answered over the past decade. The question now is about open hardware. Many of the things that we have today for OSS we don’t have for hardware and standards.

“The world continues to shift from open source being just about the code of the software to the APIs and data above it to the hardware below it.”

Check out DevBeat, VentureBeat’s brand new channel specifically for developers. The channel will break relevant news and provide insightful commentary aimed to assist developers. DevBeat is sponsored by the Intel AppUp developer program.