Ok, ok, make the obvious Twitter downtime jokes; go ahead, get it out of your system.
Done? Good. Here’s some fun news:
Twitter has just open-sourced Iago, a load generator it developed internally and that it has been using in a dogfoody sort of way.
(For you non-developer readers, a load generator fakes high-volume website traffic to test new features and services before they become available to the public, allowing the developers to tweak the feature or service for large scale use without watching it crash and burn in front of a live audience.)
“Iago has been used at Twitter throughout our stack, from our core database interfaces, storage sub-systems and domain logic, up to the systems accepting front-end web requests,” Twitter open-source guy Chris Aniszczyk wrote this morning on the company blog.
“We routinely evaluate new hardware with it, have extended it to support correctness testing at scale, and use it to test highly specific endpoints such as the new tailored trends, personalized search, and Discovery releases. We’ve used it to model anticipated load for large events as well as the overall growth of our system over time. It’s also good for providing background traffic while other tests are running, simply to provide the correct mix of usage that we will encounter in production.”
Aniszczyk says Iago is particularly well-suited to Twitter’s needs for three reasons: high performance (precise and predictable load generation), built-in support for multiple protocols, and the fact that it’s designed to be extensible.
Iago is now open-sourced at GitHub under the Apache Public License 2.0; check it out for yourself, all ye interested parties.
VB’s research team is studying web-personalization... Chime in here, and we’ll share the results.