IBM is working on a robotics system to drastically reduce the need for physically present engineers in industries like oil, shipping, and even aerospace.

The maintenance, repair, and operations (MRO) system is designed to let engineers to work “mobilely” on all kinds of machinery. It uses a combination of virtual reality software and sophisticated robotics hardware.

Here’s a handy video showing how it all comes together:

Basically, the Smart Mobility MRO system enables lesser skilled engineers to roam the land maintaining and repairing equipment. They’re monitored by an offsite, more expert engineer supervisor, who tracks them via GPS.

Then, the onsite engineer uses a smartphone and QR codes to find the right equipment in need of fixin’ as well as instructions on how to fix it. Using virtual reality, the phone can also overlay on a given site the location of another engineer or a first-aid station. It sounds like simple stuff, but when you’re in a huge manufacturing facility for the first (or second, or third) time in your life, simple stuff like this can help the job get done much more efficiently.

The robotics part comes in when the onsite engineer gets stuck and needs help from his more expert offsite supervisor. A robotic arm, equipped with audio and video hardware, provides a direct link from the supervisor to the situation and machinery. The arm also contains a projector, so the supervisor can literally draw out a plan of attack and overlay it on the onsite engineer’s view.

In short, the technology enables companies to have fewer high-skilled engineers out on the road at any given time. They can save costs by hiring junior engineers with the confidence that those folks have constant access to backup when it’s needed.

The Smart Mobility MRO project is the result of a collaboration between IBM and the University of Sheffield Advanced Manufacturing Research Centre (AMRC), which delves into the many issues and problems in advanced manufacturing.

“Over the past 60 years, I have had the privilege of seeing how engineering developments can make a profound impact on people’s lives,” said the queen in a statement. “I am delighted to lend my support to this prize and I hope that it inspires many more people across the globe to develop life-changing engineering creations in the years to come.”

The five winners will split $1.5 million (£1 million). The award is less about the financial gain (a drop in the bucket for these wealthy men) but rather about acknowledgement and prestige from real-life royalty and a voice of tradition speaking out for the future.

According to the site’s Q & A, the point of the prize is to draw attention and prestige to engineering in the U.K. and emphasize the Internet’s growing political, business, and international importance:

“Perceptions of engineering are often outdated. If people think of it at all, they tend to associate engineering with heavy industry and civil infrastructure. This is not only a limited view of what engineering is really about, it means many young, creative people – especially women – don’t consider a career in engineering. The prize will recognise and celebrate the best and also serve to illuminate the sheer excitement of modern engineering. It will provide an unparalleled opportunity to demonstrate how engineers and engineering are making a real difference across the world. The Queen Elizabeth Prize for Engineering will excite and inspire a whole generation of young people.”

The prize was announced backed in November 2011 and received unilateral support from politicians and corporations, including BAE Systems, BP, GlaxoSmithKline, Jaguar Land Rover, Shell, Siemens, and Toshiba, who donated to the trust. The QEPrize Foundation is chaired by Lord Browne Madingley and the day-to-day operations are handled by the Royal Academy of Engineering.  Nominations opened on February 28, 2012, and then a panel of judges, consisting of “eminent international figures representing the range of engineering disciplines and reflecting every region of the world” deliberated to select the winner(s.)

The award is not limited to Internet technology, although the first honorees are all from this field. Pouzin, Kahn, and Cerf are pioneers in Internet Protocol (IP), which makes up the fundamental architecture of the Internet. Andreessen wrote the Mosaic browser that made the web more accessible, and Berners-Lee (who was knighted by Queen Elizabeth in 2004) “invented” the World Wide Web.

The formal presentation of the prize will happen in June just before Kate Middleton’s due date, which, let’s be honest, is what Brits are really excited about.

Photo Credit: dbking/Flickr

Early in the morning, 80 developers are sitting in a room at Facebook’s headquarters. Their first task of the day will be to write some code and push it live onto Facebook.com.

The catch is, it’s their first day on the job.

“Welcome to working at Facebook,” intones David Recordon, an engineer leading the new recruits through their very first bootcamp.

What’s Facebook’s bootcamp?

“Whatever it is, it’s worth it, because you’ll learn something from it,” Recordon says.

Bootcamp is part of Facebook lore. It’s how the company molds each fresh crop of code-focused new hires into Facebookers. It’s a cultural indoctrination coupled with a hands-on, do-or-die, learning-the-hard-way introduction to the company’s massive codebase.

And everyone who works in Facebook engineering — regardless of product focus, previous training, office location, or seniority — makes the trek to the company’s Menlo Park headquarters for bootcamp.

Moving fast, breaking things

“Building software has always been a social activity if it’s done at any scale,” says Mike Shaver, one of the company’s engineering directors, in a meeting later that day.

That marriage between social and software is the core of bootcamp. It’s not a top-down marathon of meetings with tiers of managers and painfully boring training videos. New recruits aren’t treated like helpless new kids. Right from the start, everyone is required to row in the same direction and to solve problems.

“When Facebook engineering is at its best, we aspire to move the needle for a billion people. And then [we're] going all out see if we can do it,” says Shaver. “And then maybe having a beer afterwards.”

While some of the recruits in today’s batch are new graduates reluctant to ask questions, let alone make suggestions to change features and products for a billion users, others are startup veterans who take a certain pride in disruptive thought.

“A lot of the people we talk to from startups … they realize they wanted to have this impact and this influence in the product process,” Shaver says. “But they are often the most fun to watch early on. They try to see how far they can push it out.”

Pushing those boundaries is part of Facebook’s culture — its “move fast and break things” credo. For bootcamp, it’s fully expected that someone, somewhere will break something in the codebase. And that’s okay.

“We’d rather you go and focus on moving quickly with some increased risk … but when you do break something, there are people around who will help you fix it,” Recordon tells the new Facebookers.

Of course, it doesn’t hurt that each person in the room is, at least in a few ways, a genius. “It’s a high engineering bar. It’s the highest I’ve seen,” says Shaver, “and a lot of people who are that good, they’re used to being the smartest people in the room.”

How bootcamp works

Charu Gupta is an electrical engineer with 20 years of experience, mostly in the tech sector. She leads Facebook’s “mission control” team, a group within engineering that manages programs like bootcamp and oversees lateral movement to and from different teams and projects.

“Bootcamp is really a very unique program that’s incredibly effective in getting our engineers onboarded very quickly and giving them an opportunity to learn our common tools and framework,” she says in a phone call after the bootcamp session has concluded for the day. “Giving new recruits a very safe way to actually do — we think learning by doing is actually the best way to learn anything.”

And it’s not just the new hires who are learning. “We’ve actually found a secondary benefit, is that it helps our mentors develop new teaching and management skills,” she says.

Recordon is a perfect example of this. He’s been working at Facebook for several years; still a young man, bootcamp is giving him a taste of large-scale management.

“I enjoy helping other people grow,” he tells us in a one-on-one chat. “I care a lot about how we grow the company and engineering, how we think about scaling our culture.”

He tells us the bootcamp origin story: Early in the company’s history, Facebook engineering godfather Andrew Bosworth, a.k.a. Boz, told Mark Zuckerberg that for their first six weeks, Boz wanted all the new coding hires to report directly to him. Zuck gave the scheme the green light, and bootcamp was born.

Nowadays, bootcamps run regularly every other week. And while the experience does help new hires get comfy with the codebase and learn how to solve problems and think like a Facebooker, it also gives them and the company a better idea of where that engineer will end up working — at least for the first year or so on the job.

That’s something else unique about Facebook’s hiring process: Engineers are hired as engineers, not as members of a specific team.

“Through the first four weeks of bootcamp, you’re getting exposure to different parts of engineering through code … starting to figure out which one of these broad areas you’re most interested in, then looking at the teams that are trying to grow right now,” Recordon explains.

Because of Facebook’s truly massive scale, he tells us, “I’ve not heard of very many situations where it doesn’t work out.”

Every year and a half or so, devs are permitted to take a month off to just hack around, explore other teams and products, and contemplate switching to a new field within the company.

“If you think about the smartest engineers that you know, it’s rare that they continue to do the same thing for three or four or five years,” Recordon says. “People want to solve a hard problem, then they want to solve another hard problem.”

But whatever problems they’re solving, they’ll solve them in Facebook’s hacker-centric way.

“We expect new hires to do this,” Recordon continues, “to take our company values and apply them to the different situations that they’re in. To move fast, to be open, to focus on impact. Part of their job is to question whether they’re doing that.”

And ultimately, that’s the indoctrination that bootcamp is focused on.

The rise and fall of brogramming

It’s no secret that Facebook engineering’s origins are typical of the Silicon Valley idiom: young, white, and male. And still, you’ll notice a lot of young, white men among the company’s dev staff and engineering managers.

But Facebook, like many of its counterparts in tech, is actively, aggressively pushing itself away from that idiom and toward something more balanced and more representative of its global audience of users.

“We want to encourage more women and other minorities to feel comfortable going into software in the first place,” says Shaver. “It’s very important that we’re not just explicitly inclusive but that the environment feels comfortable.

“And honestly, sometimes it won’t. Those are problems to be solved.”

One of the more obvious problems to solve is the inclusion of more women on the engineering teams and broadening the pipeline of women in science, technology, engineering, and math (STEM) fields in general.

“We have a number of women in tech roles at Facebook who are really passionate about encouraging women to make choices in STEM,” Gupta says, “things they do very deliberately in Facebook. The large majority of our technical women are part of that outreach.” The company already participates in a slew of groups and events in the Silicon Valley area, and more initiatives are underway, Gupta says. “We’re spinning up some newer programs to really broaden our education and awareness.”

The ideal of cultural diversity at Facebook isn’t just something its leaders aspire to; it is, Shaver says, an active part of the recruitment process. “When we interview people, that’s something we’re looking for in a cultural fit,” he says. “‘This new engineer, is she going to move us more toward the culture we want to have, or away from it?”

That cultural ideal is one that ends up trickling down to other Valley startups and startups in other areas, startups founded by former Facebookers and by non-Facebookers who simply look up to the cult that Zuck built.

“We are conscious of the fact that we have a unique culture,” says Gupta. “We’re very proud of our culture, and we love our culture. And we absolutely love the fact that people leave Facebook and start new ventures and take a piece of that culture with them.”

Quality of life

A big part of killing off the remnants of brogrammerism lies in ensuring quality of life for its employees. The 80-hour work weeks and work-hard-play-hard overtones that are de rigeur at many startups can’t fly as well at a company with a headcount in the thousands.

“If you’re a new grad, you come out here and you’re gonna sprint,” Shaver says. “You feel like you have to prove yourself, and it’s a pattern. School is ‘all the things’ … We do have a pretty good focus on that.”

After all, when an engineer is doing “all the things” around the clock, literally sleeping and eating Facebook seven days a week, everyone suffers.

“One of the challenges is around this need for breadth of interest, that raw curiosity, especially in senior people,” says Shaver. And a burnt-out dev who buries himself in code and never sees the sun? That guy isn’t curious, and his scope has become too narrow for real problem-solving.

Shaver says that to combat overworking and burnout, he has to personally have serious conversations with people about taking their paid time off or, for new parents, their full four months of leave.

Ultimately, he says, “I’m pretty sure everybody is having fun in aggregate because the job is too hard to do if you’re not having fun.”

Still, Facebook strives to maintain the good parts of its original culture. It trains new recruits with lore about the good old days of rowdy young devs disrupting the hell out of everything from crosswalk placement to architecture and interior design. It promotes hackathons and all-nighters. It values free and open-source software, takes its global social impact seriously, frowns on needless heirarchy, and worships creative experimentation.

“When I was making the choice to come to Facebook, it was so exciting to me that Facebook is creating the new playbook of how a technical company runs,” Gupta says. “We have people that are really conscious of this, that are highly concerned citizens of the world, and that are fundamentally very smart.

“The scale has changed, but our core values have remained consistent.”

Image credits: Jolie O’Dell/VentureBeat

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Twitter has just upgraded the machinery behind its search, this time with added real-time human computation!

Since machines are terrible at things like irony and instantly forming new associations and contexts between seemingly unrelated terms (e.g., “binders of women” and “presidential debate”), the company’s brilliant engineers have decided to call in the big guns: Actual. People.

Although the prevailing wisdom would have it that we meatbags are ridiculously underpowered in the computational power category, Twitter devs Edwin Chen and Alpa Jain write this morning on the company’s engineering blog that meatbags will be used to create annotations for newly trending search terms.

From the blog:

First, we monitor for which search queries are currently popular. Behind the scenes: we run a Storm* topology that tracks statistics on search queries. … As soon as we discover a new popular search query, we send it to our human evaluators, who are asked a variety of questions about the query [via a custom pool of specialized workers from Amazon's Mechanical Turk service]. … Finally, after a response from an evaluator is received, we push the information to our backend systems, so that the next time a user searches for a query, our machine learning models will make use of the additional information. For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer.”

* Storm is Twitter’s trend-spotting software that quickly identifies spikes in search queries as they occur. You can check it out on GitHub if that sounds interesting to you. Storm, an open-source project, was built at BackType, which Twitter acquired.

Twitter says it uses the meatbag method for other tasks and also focuses on making its machine learning better with human input. For those worried about the quality of said input, Twitter assures the public that only the finest of Mechanical Turk workers are being tapped to handle these kinds of tasks. As the blog notes, “Having highly trusted workers means we don’t need to wait for multiple annotations on a single search query to confirm validity, so we can send responses to our backend as soon as a single judge responds. This entire pipeline is designed for real-time, after all, so the lower the latency on the human evaluation part, the better.”

If all that up there was too wordy for ya, here’s the news in singing telegram form:

Image credit: Shutterstock

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

On election night, Twitter eclipsed many of its own records and even beat Facebook as the preferred medium to announce and spread news.

So how is it that the once crash-prone service saw nary a fail whale amid last night’s social media frenzy? Twitter’s engineering czar chalks it up to Ruby — rather, to the fact that Twitter has turned its back on the Ruby stack for good.

Throughout the day yesterday, Twitter users sent around 31 million election-related tweets. At the height of result-tallying activity, the service was getting 327,452 tweets per minute (TPM) about the election alone, with tweets on all topics totaling 874,560 TPM at last night’s peak, far outstripping previous records.

While Twitter used to see brief spikes during major media events, Twitter infrastructure VP Mazen Rawashdeh wrote today on the company blog that election night was a sustained, hours-long onslaught of activity — a traffic pattern the company also experienced at lower volumes during the Olympic closing ceremonies and the VMAs.

“Over time, we have been working to build an infrastructure that can withstand an ever-increasing load,” Rawashdeh said.

“For example, we’ve been steadily optimizing the Ruby runtime [see Twitter's detailed post on dealing with the Ruby garbage collector]. And, as part of our ongoing migration away from Ruby, we’ve reconfigured the service so traffic from our mobile clients hits the Java Virtual Machine (JVM) stack, avoiding the Ruby stack altogether.”

Ouch, Ruby! You need some aloe for that burn?

Conflict manufacturing aside, Twitter has for some time known that as its place in the world of media grows, it’s becoming more of a public information utility and less a mere microblogging service. As such, downtime is unacceptable. We know it, and they know it, too.

“The bottom line: No matter when, where, or how people use Twitter, we need to remain accessible 24/7, around the world,” said Rawashdeh. “We’re hard at work delivering on that vision.”

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

But you don’t expect to find it in the construction industry.

Detroit-based Greenlancer, however, is doing just that, starting with green energy projects. I spoke to co-founders Patrick McCabe and Michael Sharber on a recent trip to Michigan.

A recent solar project

“We’re taking traditional engineering services and turned them into a product,” Sharber told me. “We have a virtual network of engineers, and so instead of hiring an engineering firm, our customers can simply fill out a web form.”

And, McCabe adds, Greenlancer accomplishes the task at about a third of the cost of a traditional engineering firm — in just a fifth of the time.

Not quite believing my ears, I asked for a rundown of the process.

A typical project for Greenlancer is a new solar plant for a building or a city … something like 200 kilowatts in size, with a price tag of perhaps $750,000 to $1,000,000. Hiring the company, and qualified, certified, and insured engineers will   show up onsite to collect any needed site data. A CAD modeler will draw up architectural charts. A technical writer will create a technical report, and another engineer will run calculations for system load, tolerances, and so on. Yet another contractor will review and certify the entire report.

And none of them ever need meet.

“This is a revolution in the construction industry,” says Sharber. “We’re starting in green tech, but if we’re successful we’ll move to bigger markets as well.”

Design schematics for a project permit application

Greenlancer calls it an “assembly swarm.” The company website is, essentially, a factory, which breaks down each project into modules, each of which is responsible for creating a specific component of an engineering report. The software identifies all needed components, lays out a plan of action, and assigns tasks.

Then engineers come on to the site to bid on those tasks.

“We do it all,” says McCabe,” from feasibility and site assessments to concept designs  – what components will be needed in which configuration — to a 25-year cash flow analysis, to the hard-core engineering with blueprints and regulatory compliance.”

It’s a remarkable model, perhaps not so much for the actual technology — which is impressive — as for the application to an industry which, Farber told me, hasn’t changed in decades.

Greenlancer has already completed 290 projects in 25 states, and has raised $160,000 in a private seed round after graduating from Detroit’s Bizdom accelerator. The company will be seeking an initial round of institutional funding in January.

Here’s Greenlancer’s introductory video:

GreenLancer from Bizdom Detroit on Vimeo.

photo credit: Bilfinger Berger Group via photo pin cc

Perhaps you’ve noticed the flurry of Facebook mobile app updates today: two new Facebook apps for Android and an upgrade for the social network’s iOS app, too.

It’s no coincidence; in fact, you can expect the same speed and quality for future updates, too.

In an engineering blog post, Facebooker Christian Legnitto, who’s responsible for pushing out mobile code for the company, says that Facebookers working on mobile have been taking the same doubled-down approach to release cycles that the rest of Facebook’s engineers use for the website code pushes.

As a result, the company has released its best apps yet, Facebook and Facebook Messenger for Android and Facebook for iOS, all within four weeks of each other. In fact, Facebook for iOS got two releases between last month and today.

Pushing out new versions “early and often” is a big part of Facebook’s hacker ethic, but Legnitto notes that this pace can present problems when applied to a mobile release cycle. Facebook.com gets thousands of test features and code pushes each week, but native mobile app architecture makes the same process impossible. For example, you can’t do a rolling release to a few thousand users at a time; mobile apps are all or nothing. And you can’t release buggy code; what would merely slow down a website might completely crash a mobile app.

But with a few tweaks, Legnitto says the release cycle for the website has been adapted for mobile with great results, including a predictable and precise schedule for updates to mobile apps.

“Today’s Facebook for Android update came just four weeks since the last version, and our goal is to deliver another in a month,” he says. “We’re now delivering on regular ship cycles for Facebook for iOS, Camera, and Messenger as well.”

Because downloading a mobile app is a bit more speedbumpy than signing into an automatically updated web page, Legnitto says, “We also wanted to balance getting improvements out to people quickly while minimizing disruptions for users. Our 4-8 week release timelines feel like a good trade-off for now.”

And as many users have noted since downloading the latest versions of Facebook’s mobile apps for iOS and Android, quality is becoming an ever-higher standard for the mobile team.

“We’ve adopted this new date-driven release process so that people get important stability, speed, and feature improvements as soon as they are ready,” says Legnitto. “This helps us drive toward higher quality — one of our top priorities on mobile.”

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Here’s a sweet little tidbit from the developer world: Twitter and Etsy have worked out an engineer exchange program that will have the two companies trading devs back and forth, all the better for both teams’ growth and development.

It’s like that ABC show Wife Swap, but with developers instead of wives and probably a bit less awkwardness and drama. Probably.

In a post on the program, Etsy engineering VP Marc Hedlund (pictured above) said the swap will help all engineers from both teams absorb the best parts of one another’s culture, work practices, and generally, ” what makes each other tick.”

The swaps will last a week and will send engineers traveling coast to coast from Twitter’s San Francisco digs to Etsy’s Brooklyn headquarters. The devs will go through an accelerated bootcamp/orientation process and then will start pushing code into production.

“It takes a level of trust to let an unknown engineer into the fold, let them sit in on meetings, and make changes to code,” Hedlund writes of the program.

“Of course, some people would be uncomfortable with letting this happen; companies we’ve both worked for would have fits before allowing it. But we believe the value of cross-pollination of ideas and practices is far too high to be blocked by these concerns. While this is an experiment, we’re hopeful it makes both teams stronger, and we’ll be looking for other exchanges to do soon.”

Of course, it doesn’t hurt that Twitter and Etsy have similar (read: hipster) cultures already in place and that the companies don’t compete with their products. But Hedlund has make some interesting moves in the past to make Etsy’s engineering team stronger.

In fact, just this past summer, Hedlund championed an effort to create grants to train women in computer science with the end goal of bringing more female engineers into the Etsy fold.

“The company has historically had many very strong and talented women working with us, but not enough of them in Engineering and Operations,” said Hedlund in an exclusive VentureBeat interview at the time. “We felt that we had a better shot at making a meaningful difference on this issue than almost anyone out there, and that in some ways we stood to benefit more, too.”

Top image courtesy of James Duncan Davidson.

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

When Alice Brooks was a little girl, she asked her father for a Barbie doll. He gave her a saw, which she used to hack a dollhouse.

Alice, now 24, excelled at school and went on to study mechanical engineering at Massachusetts Institute of Technology (MIT). This summer, while enrolled in graduate school at Stanford University, a project has been brewing: can a toy inspire the next generation of young girls to love science, technology, engineering and math?

Convinced of the merits of the idea, Alice teamed up with two fellow grad students, Bettina Chen and Jennifer Kessler, to form a ‘toys for girls’ company known as “Maykah”. As grad students at Stanford, and throughout their academic careers, the women noticed that in all their courses (advanced linear algebra, electromagnetic engineering, and so on) very few of their classmates were female.

“As we got further into our educations and careers, we saw the number of women around us decreasing,” said Brooks, in an interview with VentureBeat.

The founders believe the problem is rooted in childhood. “When we looked around at girls’ toys today, we did not see the kinds of toys that inspired us when we were young,” the girls explained in a blog post for Women 2.0, an organization dedicated to increasing the number of female entrepreneurs. “We build toys to inspire the next generation of female technology innovators.”

They are designing prototypes for the toys at StartX, a highly competitive accelerator for Stanford students. Roominate, their first toy, was inspired by that early dollhouse memory (see video, below).

As an alternative to playing with dolls, Roominate lets young girls attach and custom-build a miniature room with working circuits.

When the three founders launched the idea for Roominate on Kickstarter, a website to crowdsource funding for projects, and proclaimed that every girl should grow up with a love of all subjects and the skills to become “an engineer, architect, and a visionary,” they far exceeded their $25,000 funding goal. On the page, they cite statistics than 11 percent of engineers are women, and only 15 percent of college freshman intend to major in scientific or engineering discipline.

Maykah pulled in $85,965 to build and mass-market a toy to inspire young girls who are great at solving, deducing, and experimenting.

The founders of Maykah, a toy company for girls.

Maykah has struck a chord in Silicon Valley, one of the many tech hubs that is experiencing a dearth of female engineers and designers. According to the National Center for Women and Information Technology, a nonprofit, only 25 percent of the tech industry’s workforce are women.

Alice and her cofounders say they were exposed to technology and engineering at a young age. Rather than playing with princesses or dolls, Bettina build hundreds of lego creations with her older brother, and Jennifer recalls solving math riddles in her head with her dad. ”We believe there is a connection,” they wrote on their Kickstarter page.

Roominate is available for parents to order online on Monday. The founders have not yet determined the final price.

Images via Kickstarter 

It’s safe to say that growing up I never thought I’d be a VC.

It’s not that there’s anything inherently wrong with venture capital. It’s just it’s not part of what I thought being a good geek was all about.

There’s a very well-defined model for becoming a successful Silicon Valley technologist. You study computer science, math, or engineering in college. Then you intern at one of the great software titans. Perhaps you routinely commit code to an open source project on Github, suffer from a crushing caffeine addiction and/or a nocturnal sleep schedule, and own a track jacket emblazoned with a tech company’s logo.

But few good geeks become VCs. It’s hard to stay a razor-sharp hacker when you spend more time in Excel than you do in Eclipse, and ditching sandals for a pair of loafers can be hard to stomach. In fact, much of what the tech community thinks VC’s are like is antithetical to the image of the model Silicon Valley geek – enough so that many of my friends from college lovingly refer to my decision to become an a VC as “joining the dark side.”

For most of my life it seemed like I fit well into the model of what a geek should be. I started playing with computers and programming early on (I could type before I could write cursive). By the end of high school, I knew I wanted to spend the rest of my life making technology that solved big problems, and in 2006 I entered San Jose State as a computer science and math major. I was a flip-flop clad fixture at our school’s computer science club, and I definitely spent more time on TopCoder than I did at frat parties.

But when I started doing summer internships in engineering, I realized that I didn’t fit as neatly into the model geek image as I previously thought. I had the pleasure of spending my freshman and sophomore summers at Electronic Arts and SAP, and at both places I learned that building great software wasn’t just about writing well-documented code and hitting the compile button.

Whether you’re building a CRM suite or a video game, making rockstar software is an interdisciplinary effort. Technology is really just a tool to solve problems, and understanding how to solve those problems and fully what those problems are, requires an interdisciplinary team. I learned engineering was only one part of the process of innovating and creating great technology.

More startlingly, I learned I liked navigating and managing the product process even more than I liked writing code. I wanted to make technology that made a difference, and I wanted to explore how I could use my passion for working with people and with code.

So, to use startup terminology, I pivoted. In my remaining time in college I explored the dark arts of economics and business. Exploration led to obsession, and I ended up graduating as an economics major and computer science minor.

My professional life mirrored my life back at school, and I explored other roles in tech that would allow me the opportunity to work with lots of different people – engineers, managers, customers, etc. Product management turned out to be a perfect fit..

After two summers interning, I joined NetApp full time in 2010 as the company’s first NCG (New College Grad) product manager. NetApp was an amazing place to be a young PM. Having responsibility over a product line can be difficult for someone fresh from college, but I was surrounded by amazing co-workers that mentored me as I learned the mechanics of product management and cultivated my leadership skills.

NetApp was an amazing experience, but after two years I decided I wanted to try something different. In my last year as a PM I started getting very interested in the startup community in San Francisco. Once a month, I drove into the city to attend a tech meetup and pitch session called SF New Tech.

I found myself fascinated with the people I met there. The “make it or die trying” attitude of founders, the willingness to sacrifice everything to make a difference –the ideological drive and passion necessary to succeed in a startup spoke to how I approached technology and the tech industry.

Startups also spoke to the entire reason why I was in tech in the first place. My parents ran a small dot-com in the late 90’s, and a lot of what I learned early on about the tech industry came from them and their experiences navigating the first tech boom.
I wanted to leverage my background to help people like my parents and the people I met at SF New Tech. After careful consideration and some research, I realized venture capital could be a great way to become bilingual in both “geek” and “suit”.

One of the first lessons I learned in venture was the value of serendipity. Serendipity is core to the success of being a VC. Finding the next Facebook or Google is firmly rooted in being at the right place at the right time, and a significant portion of being at that crucial place and time is governed by serendipity.

My way into VC was fittingly serendipitous. I met my friend Adam randomly at a mutual friend’s birthday dinner, where we bonded over a shared appreciation of good drinks and cryptography. As we got to know each other a bit more, I learned Adam was an associate at a fairly unique VC firm called GGV Capital. A year after we met, Adam tracked me down – once again at a party – to offer me the opportunity to replace him at GGV.

Passion is a critical resource at GGV Capital, and they’ve allowed me incredible flexibility to explore mine as an associate. We care a lot about what we invest in, and it’s critical we work with teams who are just as excited about technology and making a difference as we are – if not more. I’m very excited about big data, cloud computing, gaming, and information security, so I plan to drill deep on all of these topics and work with entrepreneurs changing the game in these fields.

It doesn’t really matter whether your business card lists you as an engineer, a product manager, an entrepreneur, or a venture capitalist. Whether you hack code or write term sheets, the only thing that really matters is how you uniquely contribute to the process of innovating and creating great technology.

Andrew Manoske is an associate at GGV Capital, an expansion-stage venture capital firm focused on the U.S. and China with $1 billion under management.

Quit your technology job. Get a Ph.D in the humanities. That’s the way to get ahead in the technology sector. That, at least, is what philosopher Damon Horowitz told a crowd of attendees at the BiblioTech Conference at Stanford University in 2011.

Horowitz is also a serial entrepreneur who co-founded a company, Aardvark, which sold to Google for $50 million. He is presently the In-House Philosopher / Director of Engineering at Google. Wait, you say, that’s insane. At a time when record numbers of people, among them those with high-level degrees, are receiving public assistance, what kind of fool would get a degree in a subject with no clear job prospects beyond higher education or teaching?

In Silicon Valley, engineers are honor students and everyone else is taking remedial math. Venture capitalists often express disdain for startup CEOs who are not engineers. Silicon Valley parents send their kids to college expecting them to major in a science, technology, engineering or math (STEM) discipline. The theory goes as follows: STEM degree holders will get higher pay upon graduation and get a leg up in the career sprint.

The trouble is that theory is wrong. In 2008, my research team at Duke and Harvard surveyed652 U.S.-born chief executive officers and heads of product engineering at 502 technology companies. We found that they tended to be highly educated: 92 percent held bachelor’s degrees, and 47 percent held higher degrees. But only 37 percent held degrees in engineering or computer technology, and just two percent held them in mathematics. The rest have degrees in fields as diverse as business, accounting, finance, healthcare, arts, and the humanities.

Yes, gaining a degree made a big difference in the sales and employment of the company that a founder started. But the field that the degree was in was not a significant factor. Over the past two years, I have interviewed the founders of more than 300 Silicon Valley startups. The most common traits I have observed are a passion to change the world and the confidence to defy the odds and succeed.

Any discussion of this nature must return to a comparison of Steve Jobs and Bill Gates. True, Jobs was technically competent. But he had, if anything, an eclectic educational background where he spent as much time in seeming arcana such as philosophy and calligraphy as he did on math and engineering.

I’d take that a step further. I believe humanity majors make the best project managers, the best product managers, and, ultimately, the most visionary technology leaders. The reason is simple. Technologists and engineers focus on features and too often get wrapped up in elements that may be cool for geeks but are useless for most people. In contrast, humanities majors can more easily focus on people and how they interact with technology. A history major who has studied the Enlightenment or the rise and fall of the Roman Empire may be more likely to understand the human elements of technology. She may more readily understand how ease of use and design can make the difference between an interesting historical footnote and a world-changing technology. A psychologist is more likely to know how to motivate people or to understand what users want.

This brings me back to Damon Horowitz. He was a highly accomplished artificial intelligence (AI) researcher with a master’s degree from MIT. Damon was in hot demand, making big bucks and founding companies, several of which were acquired for nice, tidy sums. The trouble was, he realized his work was not actually solving the underlying problems of AI in any meaningful way. Damon felt he didn’t understand the philosophy of intelligence and human thought well enough to get beyond the beautiful, “intoxicating” prison of computer code he lived in.

So Horowitz quit his tech job and went to Stanford to get his doctorate in philosophy. He was amazed at how much he had to learn. For Horowitz, going back was a transformational shift that opened his eyes not only to key foundational arguments and theories about the nature of intelligence, but it also gave him improved capabilities in strategic vision, creative problem solving and other critical traits. Horowitz believes his degree helped him envision Aardvark, which was an interesting hybrid search system that involved people sending out queries to fellow users who were connected through an automated interface that helped askers properly shape and target their questions.

Don’t get me wrong. The world needs engineers. And no, I am not actually advising people to quit their jobs and get Ph.Ds in philosophy. For some people, it might make sense, but for others it wouldn’t. The point I’m trying to get across is more nuanced: We need musicians, artists, and psychologists, as much as we need biomedical engineers and computer programmers.

For tech entrepreneurs and managers, there is no “right” major or field of study. While having a degree in slinging code may present a short-term advantage at startup time, it may comprise an equally important disadvantage if the degree came at the cost of other critical “soft leadership” skills required to focus, lead and grow companies. So, it’s time for Silicon Valley to get over its obsession with engineers. And, if you run a startup, hire that psychology Ph.D. You may get a lot more than you bargained for.

Wadhwa is a fellow at the Rock Center for Corporate Governance at Stanford University and is affiliated with several other universities. Read more about Vivek Wadhwa’s affiliations.

Originally published on WashingtonPost.com.

Photo credit: mararie/Flickr

Facebook is showing off glimpses of its newest facility up north. Located in Seattle, the new space will be home to some of the social network’s engineering talent.

We’re guessing the new office will be ideal for capturing restless Microsofties and Amazonians enamored of the ripsticks-and-Zuck culture Facebook has cultivated over the years.

“Just a couple of years ago, the entire Facebook engineering team was in one building in Palo Alto, California,” Facebook engineer Ari Steinberg wrote today on the company blog. In those halcyon days of legend on University Avenue, Steinberg continues, “Everyone was within a 30 second walk of each other. But in early 2010, we got to a point where we knew we needed to look beyond Silicon Valley to keep building Facebook and ship everything we wanted to ship.”

Calling Seattle “the logical choice” for Facebook’s next big satellite, Steinberg noted that Facebook’s Seattle office now holds close to 100 engineers.

So far, the Seattle team has shipped products and features including Desktop Messenger for Windows, video calling, and other projects shared with the Bay Area team.

“Our new building feels like it represents Facebook as well,” Steinberg concludes.

“Everyone sits out in the open with their team, usually just four or five people. There’s delicious food, snack stations, free laundry, and four months of parental leave. But there are also a couple of things that are uniquely Seattle: Every Wednesday, we eat lunch together at a local restaurant, and in March, we all headed up to Steven’s Pass for a ski trip. Windows wrap around the entire floor, offering panoramic views of the city, Lake Union, the Space Needle, Puget Sound and the Olympic and Cascade Mountains.”

Here’s a sneak peek at the Facebook Seattle offices:

Top image courtesy of Jolie O’Dell

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Earlier this week, Facebook launched a boatload of new features for third-party apps. If you want a look under the hood, you’re in for a treat this morning.

In a new blog post, Facebook engineer Karan Mangla explains how the new features came to be.

To recap the new features and corresponding APIs, Facebook gave third-party developers the ability to add location and friends as properties to any update, photo, or link. To accomplish this feat, the social network rolled out a whole herd of new APIs and documentation. You have your location-setting API, your friend-tagging API, your improved places search API, and your location-reading API.

Mangla said the company has been working hard on beefing up Places and other location functionality for the past year and a half. Location-tagging is now a universal feature for all photos and statuses, and there’s even a new map view for Timeline, so you can see your activities represented geographically rather than chronologically, if you choose.

Creating that map was a special challenge. Without the “lazy load” type of scrolling action in a chronological Timeline page, all location data from a user had to be fetched and presented at once, which Mangal said created a huge data load for Facebook to process. To handle the issue, the team “created infrastructure to farm out data fetching to multiple servers,” Mangal wrote. “On every page load, a single server fetches the IDs of all pieces of content that can be displayed for the current user. This server then breaks up this data into smaller chunks, and each chunk is sent in a request to another server to actually fetch the data and do privacy checks. The responses from these servers [are] then combined to create the timeline map display.”

Also, to make location features more accessible, particularly to GPS-lacking feature phone users, the team built out a new location search feature that took into account places a user was likely to be that also matched up with the first three letters in a search query.

“Developing the universal search infrastructure allowed us to work around this issue and provide users the ability to search for places on any phone, improving search quality with location if it’s available,” Mangal wrote. “This significantly increased the set of users who could check in to a place via their mobile phones.”

You can check out the rest of Mangla’s comments on the Facebook Engineering Blog

Image courtesy of Anneka, Shutterstock

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

But is Node just as good a fit for web agencies and developers that build dozens, scores, or even hundreds of sites a year?

Making the switch away from tried-and-tested platforms such as Ruby, PHP, Python or .NET for a fresh technology barely out of its infancy, especially when it’s not your product and when you’re working for global brands that simply can’t fail, is a risky proposition.

So why would you want to use Node.js? As a high-volume web agency, here are the reasons we chose to go with Node and the tips and tricks we learned along the way.

Why we chose Node.js

» It’s JavaScript!

Every developer also knows at least some JavaScript. Introducing Node.js, then, is relatively easy.

Everyone in a given web shop knows the basics and just has to learn about the event loop, callbacks and to how to use async flow control.

At our own agency, as people got to know Node, we actually saw that our browser-side JavaScript code improved in quality and structure.

» Code re-use at every level: browser, back end & database

JavaScript is the language of the browser, but JavaScript also powers many of the new NoSQL databases. We tried a couple of them for building content management system and quick fell in love with MongoDB.

MongoDB uses JavaScript for querying data, which means at the very worst that we can copy and paste code and use it in different layers of the system. What might be written as a parser for the browser might be used to format a report executed on the database.

Taking this one step further, we are standardising the include mechanism to actually reuse code and modules across the layers. This means all layers can include the same file, massively reducing the maintenance needed of code and cutting down the time required to write tests.

» Strong, responsive and enthusiastic community

IRC, meetups, bloggers, Twitter and Github are all alive with the chatter and support of Node.js. But not only are they alive; there is an excitement which you don’t normally see on this scale.

So for our developers and many others, this is an invaluable resource when bugs, issues and obstacles do occur.

» Large productivity gains in HTML & CSS using Jade & Stylus

HTML and CSS guys love working on Node because using Node means we’ll be using Jade and Stylus by TJ Holowaychuck. We loved HAML and SASS before Node.js, and now we can’t imagine using anything  other than Jade and Stylus.

» Performance and scalability

We have found that Node.js scales really well. The non-blocking event loop allows for a phenomenal amount of traffic compared to our old, highly-optimized PHP stack running through Apache.

On our first project for a national newspaper, we used a front-end nginx proxy for load balancing to the various node instances and were ready to add extra instances during the peaks caused by TV and national newspaper advertising. The peaks came but the load on the first instance stayed low.

The single Node.js instance hardly broke a sweat, despite seeing one of the highest requests per seconds we’d ever seen.

With PHP and PostgreSQL we could scale up, but it felt really hard and gave us many sleepless nights. Using Node.js with a MongoDB backend scaling up is quick and easy; but because Node can handle more traffic, you don’t need to as quickly.

» Wealth of hosting options: No.de, Joyent’s SmartMachine, Heroku, Nodejitsu

We host our production sites on our own private cloud, but for smaller companies there are many cost effective and easy to setup hosting providers. We’ve got a couple of new Node.js projects in development and will be deploying them on Joyent’s SmartMachines.

» Make your developers famous

I always try to ensure our developers work on open-source projects in contracted hours as much as possible. Encouraging them to start projects that benefit the community as well as the business.

When building projects for our clients, we look for modules that have common functionality which could be packaged up and made into open source projects:

For example, Gzippo is connect middle-ware developed by one of our staff to perform gzip compression of static assets.

We like to manage compression and expiry times of static assets in our Node.js layer because it gives us the freedom to either serve directly from Node or to stick a caching proxy in front.

This way it is served from the caching proxy for all other requests and keeps all the configuration in the Node app, meaning you only need node.js on your development machine to work on projects and do QA using Pagespeed or ySlow.

» Developer happiness

There is something intoxicating about coding JavaScript on the server side; throw in the event-loop and it’s heaven for developers. They just can’t get enough of it!

Maybe it’s because we’ve been coding PHP for such a long time, but Node.js has really inspired our developers. If you were to ask any one of them, “How do you find coding in Node.js?”, you can bet your bottom dollar you’d hear “I love it!” right back.

A sample Node development stack

This is a snapshot of our ever-evolving development stack. There are a couple of staples, but for the most part if we find something better we can just switch it out.

• Node.js

0.4.12 and 0.6.6

• Binary management: n

Point releases often break backwards compatibility. We manage our version using the excellent tool n, also by TJ Holowaychuk.

• Flow control: Async.js

Anyone wandering into the world of Node.js quickly make a choice how they are going to handle flow control. There are a number of methods out there. We’ve stuck with one of the most popular Async.js and depend on it in nearly all modules.

• HTTP, routing and view rendering: Express.js & Connect

The de facto standard Node.js web application framework is the Sinatra inspired Express.js which builds on top of the Connect middle-ware framework by Sencha Labs.

• Presentation: Jade & Stylus

We use Jade as our template engine in the view layer to generate the HTML, and we use Stylus for creating CSS. If you’ve not checked them out then you should, they are a compelling reason for using Node.js. Here is an example of the sort of thing that our designer are doing with Stylus.

• Homebrewed Node modules: Serby on Github
• We’ve built a number of Node modules which we reuse from project to project. Validation, data mapping, et cetera — Basic utility stuff that epitomizes our style of development.
• Logging: Winston

We use Winston writing to files and also pushing log data to the excellent Loggly cloud logging service, which consolidates the logs from each of our node.js instances.

• Testing: Mocha

There are a lot of testing frameworks for Node.js. Knowing which to chose it hard. We looked at a few and made the decision to go with something very simple, in this case Expresso and Should.js. New projects are now using Mocha, the successor to Expresso.

Conclusion

We are on our fifth large project with Node.js (for us, “large” is around 150 worker-days) and are continuing to support the previous projects whilst developing a handful of small and medium-size projects in between.

Less than a year has passed since we adopted Node.js, but the project stats are already looking really good. Initially, the projects we undertook did take longer as our developers learned the intricacies of Node and as we found our feet with the hosting, editors and tools needed to develop and deploy a large scale applications. However, our last project in node was actually under budget, something that rarely occurs at Clock.

Once you get used to event-based async programming, settle on a set of standard development patterns and an architectural style. You quickly start to reap the benefits of working solely in JavaScript. This makes Node.js not only fun but extremely effective to program in.

Paul Serby is CTO of Clock, a UK web agency, where he directs technology choices, architects software, manages large development projects, writes code, makes tea and preaches the virtues of good software design. Clock developed Sun Perks, one of Britian’s largest consumer facing Node.js implementations for News International. Followed shortly by another ground breaking service, the Eat Out Dining Card for The Times. Both leveraging Node.js and a whole host of young and exciting supplementary technology.<

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Engineers do cool stuff. They build cities, save lives, create music and design computer systems. Plus, they make a ton of money, relatively speaking.

All these things are the stuff teen dreams are made of, and just hearing about them can help turn young students — including teenage girls — on to engineering as a college major and career option.

In an Intel-commissioned study of 1,000 teenagers, researchers found that around 63 percent of teens ages 13 to 18 had never considered a career in engineering.

But after hearing how much money engineers make ($75,000 annually, on average), around 60 percent of the subjects said they were more likely to consider engineering as a career. Learning that engineers suffer less during periods of high unemployment also went over well, persuading more than 50 percent of the teens in the study to look at engineering careers.

The majority of the teens in the study said they were also more interested in engineering “by understanding what engineers do, such as playing a role in rescuing the Chilean miners who were trapped in 2010, delivering clean water to poor communities in Africa, designing the protective pads worn by athletes and constructing dams and levees that keep entire cities safe,” the study’s findings read.

That’s really the most important take-away from the study: Teens become more interested in engineering simply by knowing what engineers do and what opportunities exist for engineers.

Currently, around one-third of teens can’t name any potential job opportunities in engineering fields. Roughly 13 percent don’t think that an engineering degree would be more likely to lead to a great job than any other major. And a full 20 percent of these teens have no concept of just how much engineering shapes the world around us.

Engineering, however, is a multifaceted field with many areas of specialization. And teens are apparently intrigued by those different areas.

Fifty-three percent of teens in the study said they were more likely to consider an engineering career after they learned that engineers help make music and video games. And here’s one for the nerds: 50 percent of the teens said they were more interested in engineering due to engineers’ roles in texting and social networking.

The teens also showed some interest in how engineers can achieve widespread social benefit. Around 52 percent of them said they would think twice about the career after learning about how engineers helped to rescue trapped Chilean miners or create clean water for folks in underdeveloped areas.

What about the girls?

Since there’s a markedly lower number of women choosing engineering education and careers, we asked Intel’s researchers about the specifics between teen boys’ and teen girls’ motivations.

“There was a difference in interest between male and female students,” a representative told VentureBeat.

“After telling the teens facts about engineering, such as the breadth of what engineers actually do and how much money they earn, our survey found that girls are harder to persuade than boys because even after the messaging, only 35 percent of girls will consider engineering. After messaging, 60 percent of boys will consider it.”

However, while money-focused stats left the young ladies cold, they were motivated by how much social benefit engineers can create.

“As other studies have indicated, messages that emphasized the emotional appeal of engineering — for example, that engineers play a role in delivering clean water to communities in Africa — were most effective in getting girls to change their minds about the field,” the spokesperson continued.

There was also an interesting gender split in the types of engineering the teens found interesting. While all the teens found computer and software engineering to be the most fascinating area of study (22 percent of the teens polled said they wanted to get into that kind of tech), the study showed that only girls were more likely to be interested in architectural engineering (18 percent of female respondents versus 9 percent of male respondents).

Check out DevBeat, VentureBeat’s brand new channel specifically for developers. The channel will break relevant news and provide insightful commentary aimed to assist developers. DevBeat is sponsored by the Intel AppUp developer program.

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Facebook will open an engineering office in New York City in early 2012, Mayor Michael Bloomberg and Facebook COO Sheryl Sandberg jointly announced Friday.

“For us, this isn’t a satellite office,” Mike Schroepfer, Facebook’s vice president of engineering, said. The office will be core to Facebook’s engineering stack, he added.

The new office is part of a calculated effort by the social network to expand its presence in New York and recruit top engineering talent outside of Silicon Valley. Facebook currently employs engineers only in its Palo Alto, California office and a small Seattle office it opened last year. It is also planning to build an office in Menlo Park, California, as part of its planned expansion to 9,000 employees. The company currently employs about 2,000 people, mostly in Palo Alto.

Facebook’s engineering manager Serkan Piantino will lead the new office. Piantino previously led the engineering team that created News Feed and Timeline.

Facebook first kicked off its aggressive New York City takeover when it leased two floors at 335 Madison Avenue one year ago. Facebook already has 100 employees working out of that office, and will hire as much talent as required but it does not have specific goals in mind, Sandberg said.

“We’re going to grow as quickly as we can in New York,” Sandberg said. “We’ve had a great presence in New York for a long time. New York has been a really important home for us.”

Microsoft engineers spoke publicly for the first time yesterday about how they built the Kinect motion sensing system, offering a rare glimpse inside the secret world of product design.

After being taken to task for designing Xbox 360 game consoles that failed in large numbers, Microsoft turned a corner: Very few customers have returned the Kinect add-on for the Xbox. One of the reasons was the engineering discipline that the company applied in the wake of its defect scandal.

People are enjoying the system, which lets them control games with their body motions rather than standard game controllers.

At a chip conference yesterday, Microsoft engineers said they deliberately over-designed the Kinect system so that it could withstand anything that consumers could throw at it: hot temperatures, drops, careless shipping, abusive gamers, a sudden loss of power, and even surge protection from lightning strikes. (To be clear, it won’t survive if hit by lightning. But if your house or electrical wires are hit by lightning and the power surges, then Kinect has a chance of surviving). The result: the Kinect became the best-selling consumer electronics product in history, selling more than 8 million units in its first holiday selling season.

The talk at the Hot Chips conference at Stanford University was unique, as the Kinect engineers had never been allowed to talk about their labors publicly before.

Some people might have the impression that Microsoft simply bought the technology for Kinect, since it did in fact buy several companies that focused on detecting motion in a three-dimensional space. Despite those acquisitions, Microsoft actually did a lot of the engineering in-house. It had to pull together the expertise to create a high-level 3D sensing system — with gesture recognition, video and audio — that had to be dead-simple for consumers to use, and it had to build the system from scratch in 18 months. You can get a glimpse of the complexity from the picture at right.

“Kinect had to be approachable,” said Dawson Yee (pictured below right), one of the engineers on the Kinect team, which was part of the hardware design team at Microsoft in Seattle.  ”It had to be easy to set up and work like magic. You plug it in, and it works.”

Microsoft’s marketers helped a bit by showing the engineers who would be their target consumers. While Microsoft already had hardcore gamers, such as fans of Halo, it didn’t have the broader mass market of non-gaming consumers that the Nintendo Wii was getting. So the marketers went around the world visiting the homes of consumers to get a sense for what they wanted.

The Kinect team knew that they had to design an attractive product. If you’re going to put this system in front of your TV in the most visible spot in the living room, it has to look cool. So industrial designers were enlisted so that consumers could look at the product without doing a double take.

“It can’t look junky or overbearing,” Yee said.

On top of meeting consumer needs, Microsoft had to create the device so that it complied with regulations on wireless emissions and other sorts of strict safety guidelines. It also had to be better than the regulations. For instance, if a user put a finger on the Kinect and found that it was hot, that could be alarming, even though its temperature could be very well be within the regulatory guidelines.

“You had to test it by dropping it on concrete,” said Yee, who has worked at Microsoft for 12 years and at Intel for a decade before that. “That was the level of robustness.”

One of the tough things was that Microsoft’s hardware engineers had to design the system out of thin air. They didn’t know exactly how software creators would use it. Nintendo had made the motion-sensing Wii, which depends on hand-held Wiimotes. But Microsoft had to design something with different kinds of sensors that were capable of sending signals around a room and then receiving them so that it could produce a digital representation of the spatial features of that room, without depending on remote controllers. The system had to see what was in the room, who was moving, and what they were doing. While the Wii could detect your arm motion, Kinect could go further. If you want to kick a ball in a Kinect game, you make a motion with your leg to kick a ball.

To that end, Microsoft acquired 3DV Systems, a motion-sensing chip maker. But it ultimately used a motion-sensing chip (the PS 1080 depth sensor, pictured in the schematic) from PrimeSense. Microsoft also licensed technology from GestureTek, which had patents in the space. All of that gave the company design freedom to create what it needed without encroaching on patents held by others. Just before Microsoft shipped, it bought another 3D sensing firm, Canesta.

The PrimeSense system uses a near-infrared laser diode to send out a signal into the room. PrimeSense encodes information in light patterns as the signal goes  out, and the distortion of the pattern is what the receiving camera looks for to figure out depth. The camera gets the infrared light back and the processor creates an image of the room and the people in it. The chip deciphers the image and where it’s moving.

Microsoft’s own software finishing the image detection and sends it own as information to the game console. The camera can see any number of people in a room, but there could be limits on how many moving people can be tracked at a time. The sensor has to be responsive to fast movements, but it can’t consume too much power. The optics had to be exactly placed within the unit during the assembly process.

The difficult thing is that a lot of things can interfere with the signal. Ambient light can cause distortions in the measurements. And there are tough perception problems, like someone wearing a white T-shirt too close to the camera, or someone standing far from the camera in a dark shirt against a dark background.

“We would not have shipped a product if we did not solve every single one of these issues,” Yee said.

Since some of the technology, such as optics, was new to Microsoft, the company had to hire new engineers such as Scott McEldowney (pictured, left), who had 18 years of optical experience and who also gave the talk alongside Yee.

Some users have complained that the Kinect isn’t accurate enough or that its sweet spot in a room is too small for some living spaces. But for the most part, Kinect is remarkably accurate. To make it more accurate would have cost a lot more money without too much gain, said McEldowney.

“We knew this thing was going to be viewed as a toy and so it was going to be abused,” McEldowney said.

As it was, suppliers balked when Microsoft asked them for high-quality parts at low prices. The suppliers said that Microsoft was demanding the same quality as telecommunications gear, but for a low price. Haggling ensued.

One of the amazing things was that the suppliers were able to ramp up manufacturing so that Microsoft could ship 8 million units in a single season. That is unheard-of for a new product, and Sony’s rival PlayStation Move system shipped a lot fewer units.

Each system has to be calibrated before it ships, but the system also has to be capable of calibrating itself in tests with the user. The requirement for calibration meant that the system had to have a tilt motor which could automatically raise or lower the sensor. With the motor came more requirements for precision manufacturing and reliability. When you turn on Kinect, the first thing the camera does is look for the floor. When it finds the floor, it knows a user won’t be far away.

The system was “over-designed” to be more accurate than necessary because the engineers anticipated future applications that would need the accuracy. Microsoft anticipated people would hack the system and deliberately left the universal serial bus (USB 2.0) open. After shipping Kinect, Microsoft was surprised at the enthusiasm of the hackers who modified the system. Microsoft then shipped a software development kit to allow users to modify systems for their own applications.

The engineers didn’t have any example to follow, so they started by figuring out the limits of what was possible in terms of the laws of physics, the state of the art of the available technology, the cost target, the schedule, and the basic functions they thought the system would need. There were practical limitations on manufacturing and the supply chain that had to drive what the hardware people designed. Otherwise, they would never be able to ship a system on schedule for the right cost.

“We went as far as the material limits would allow us,” Yee said.

Part of the engineering problem was integrating a video camera, 3D sensor, and audio array into the product in the same confined space. The devices had to be placed precisely so they were synchronized and didn’t interfere with each other. The camera had to have a wide field of view, but it couldn’t introduce errors that could cause inaccurate depth estimates.

The audio array was a Microsoft research invention with four microphones that enabled Microsoft to identify spoken words and tell the direction that a sound was coming from. The box includes a Marvell 88ap1 audio chip and a Texas Instruments TAS1020 motor controller. The audio has to be capable of handling speech commands, doing simultaneous voice chat between gamers, and video conferencing. For that purpose, Microsoft with higher-quality wideband 24-bit audio. The quality had to be good enough so Kinect could pick up a quiet voice or a loud voice. The Marvell chipped handled the audio.

As the hardware plans came together, so did the team building the software and the marketing as well. The marketers focused their ads on the reactions of players who were playing it. Kudo Tsunoda, creative director for Kinect games, told us last year that the whole goal was to let people who were otherwise intimidated by controllers to jump in and play.

“I worked on Kinect for three years, but the technology that Kinect is based on has been in the works for almost a decade with Microsoft research,” Tsunoda said last year. “It was in the last three years that we made it so anyone could use it and it could go into millions of homes.”

Because of all of the high-tech gear in the box, Microsoft didn’t exactly hit its target for costs. But it probably did OK with the Kinect in terms of the bottom line. An outside testing firm estimated that the cost of a Kinect system is about $56 for the hardware alone. Microsoft sells Kinect for $149.

In conclusion, Yee said, “We nailed it.”