After relaunching as a social video network more than a year ago, Telly (formerly the Twitter video sharing service TwitVid) is now focusing even more on video discovery with the launch of My Telly.

The new product algorithmically generates videos that interest you on Telly’s website and mobile apps, based on what your friends have liked, posted, watched, and discussed. Additionally, the company announced that it has reached 7 million unique users, doubling its user base over the past seven months.

“We started Telly to fix video discovery across the web,” said Telly chief executive Mo Al Adham in an interview with VentureBeat. But the company learned it also needed to do more than just show you the videos your friends are posting on social networks. “We had to add more signal to the feed … we asked ourselves how do we pain a more complete picture of what your social graph is doing,” he added.

The answer for Telly was algorithms — instead of just fishing the videos your friends posted online, My Telly also looks at what your social network is engaged with, and it also keeps track of your viewing habits to figure out what you like. It also takes into account the device you’re using, so longer videos will be prioritized on the desktop, while your smartphone will get shorter clips.

The focus on algorithms is likely a smart move for Telly, as algorithms are increasingly becoming important for all media discovery services.  Consumers are increasingly expecting web services to know what they like, and to present meaningful content without much legwork.

Telly still allows you to upload your own videos, but that’s clearly not as big a feature as it was in the TwitVid days. At the same time, video uploads let Telly straddle a line the normally separates social video apps. ShowYou, for example, is entirely focused on video discovery, while Vine is finding success with its simple video sharing.

“We’ve rallied the entire team around having one unified mission … a mission to communicate to the end users and give them a great content experience on Telly,” Al Adham says. Engagement is surprisingly high on the service, with most users spending around 17 minutes on the site, and typically watching four videos, with every visit.

Looking ahead, Al Adham tells me the company is also looking closely at smart TV platforms, starting with open platforms like Google TV. Like most video services, he wants Telly to be available on every screen a user is watching.

San Francisco-based Telly raised around $6.5 million as TwitVid from Azure Capital, Draper Fisher Jurvetson, and Siemer Ventures.

Imagine a world where every machine — from the smart meter to the cell phone — is designed to mimic the human brain by getting smarter over time.

A new “big-data” product called Grok promises to make those brainy machines a reality. It’s the first product from Numenta, which Palm cofounders Jeff Hawkins and Donna Dubinsky started in 2005. Numenta recently announced its technology, and now it’s brought new leadership on board prior to going from beta testing to general release. New chief executive Rami Branitzky (pictured above, right) is a former executive at enterprise software behemoth SAP.

But the real brains behind the technology is Hawkins (pictured above, left), a neuroscientist and technology entrepreneur, who I spoke to recently by phone. He painted a vision of an urban environment where “everything is intelligent, everything is learning.”

Hawkins has a knack for predicting the future. Two decades ago, he told a group of engineers at Intel that momentum would shift to the mobile space, and people would carry tiny computers in their pockets. “That talk was one of the worst-received talks I ever gave. I don’t think anyone believed me,” he revealed during a recent keynote at the International Symposium of Computer Science. During his subsequent 15-year-long “side-track” into mobile computing, Hawkins proved critics wrong when he invented the Palm Pilot.

Hawkins’ latest invention, Grok, stems from years of neuroscience research.

Grok uses algorithmic frameworks to find trends in scattered data and make predictions about what patterns come next. Instead of analyzing mountains of data that companies are storing, it is one of the first to work off streams of real-time information from sensors.

“There is an explosion of things that are wide and connected [and] an emphasis on storage and processing all that data,” Hawkins said. “People are storing a lot of data but not solving the problem of what to do with it.”

He refers to technology giants like EMC and NetApp that specialize in storing vast quantities of noisy data. The latest technology products to hit the market are designed to go one step further than this by making this data useful to businesses and government agencies. From SAP with its next-generation database to a venture-backed startup like Platfora, there are dozens of contenders in the increasingly crowded market.

But what’s unique about Grok is the automation layer, and its continuous ability to learn. The product can be fed into companies’ existing control systems to automate processes that previously required manual adjustment. Instead of reacting to problems, users can anticipate them and optimize accordingly.

Grok is currently being tested on a few customers in the ad-tech, video processing, and energy management worlds. One global energy management company is using Grok to predict energy consumption in a building every hour for 24 hours and adjust output accordingly. Likewise, publishers and mobile app developers are leveraging the product to boost ad revenues by ensuring that inventory is optimally placed across the various ad networks.

Hawkins told me the most exciting challenges are “green-field,” meaning there’s no legacy technology solution to displace. “Once we have the data source, within half an hour we will begin to generate results,” he said.

This isn’t too disimilar to the way the human neocortex, the part of the brain that includes the perception and reasoning functions, discovers sensory data. In his body of work, Hawkins argues that the brain is a big data engine– it’s a sophisticated predictive modeling system. The brain has to learn everything by processing a continuous data stream of events. Just like an intelligent computer, it develops a model to make predictions, detect anomalies, and take action.

Given Hawkins’ reputation in the tech community, customer acquisition shouldn’t be a problem. He became widely-known with the success of his book On Intelligence, which delves into the basics of neuroscience and draws parallels with next-generation computing. “When I wrote it seven years ago, I was approached by technology companies — potential customers — but I didn’t have anything to sell,” he said.

So Hawkins founded Numenta, the Redwood City-based predictive software company, and brought on former Palm Computing CEO Dubinsky (pictured above, center). With an expanded team and new product, the internally-funded company is also on the look-out for traditional sources of capital and strategic partners.

New CEO Branitzky will be steering the company to new sources of revenue. ”I have seen a lot of enterprise software,” he told me. “I know where the gaps are in the space.”

For its first customers, Grok has improved top-line performance for its first customers by at least 10 percent. With positive feedback from the beta, the team feels ready to start selling the product in the first half of 2013. They are planning to market to small- to medium-sized businesses that do not have the budget for a data scientist or data service team. According to Branitzky, these companies typically have some kind of “prediction problem,” which is where Grok comes in.

Echoing the rise of mobile technologies, venture capital firms and large companies poured millions of dollars into big data technologies this year. And yet, Hawkins feels he’s an early arrival to the party, and we haven’t seen anything yet in terms of what this technology can do. “We’re definitely not entering into a market that has been around for a while,” he said. “This is an unmet need and our customers have no solution.”

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

One of the hottest marketing terms right now is “big data.”

Like “the cloud” last year, it’s ubiquitous: Every tech company seems to be working on some kind of pitch to show how it can handle huge volumes of data and turn it into a strategic advantage for you.

I don’t blame them. Nobody wants to be accused of having “small data,” after all. That would just be embarrassing.

The truth is, though, most companies would be better off with small data — or really any data — than they are today. An embarrassing number of business decisions are made without reference to real data.

I realized this point over the weekend, while reading Daniel Kahneman’s eye-opening 2011 book, Thinking, Fast and Slow. Kahneman, who won the Nobel Prize for economics in 2002, has spent his career studying how people make decisions. He and his many research partners have found that most of our decisions are based on quick, intuitive responses, rather than on our brain’s more deliberative, analytic capabilities. The intuitive part of your brain is amazingly powerful at coming to rapid, synthetic judgments based on a large amount of information. Unfortunately, it’s terrible at making judgements about anything where there is a statistical uncertainty about the outcome, and even trained statisticians have a poor intuitive sense of statistical probabilities.

Kahneman details the many examples where supposed experts are provably inept at predicting the future. Political analysts can’t reliably predict the outcome of elections. Sports fans can’t predict who’s going to win a game. Individual investors are terrible at picking stocks, and hedge fund managers aren’t much better, doing only marginally better than random chance.

Once I’d read Kahneman’s book, it became obvious that much of the tech industry suffers from the same problems he describes.

• When a venture capitalist decides to invest in a startup, it’s often based on hunches and on “pattern matching,” the VC term for betting on things that, in their opinion, look like something that’s been successful before.
• When a startup launches a new product, it’s usually done without any real data. And that’s fine, except that startups don’t usually set themselves up to collect usage data and act on it rapidly, a model espoused by Eric Ries and others in the “lean startup” movement.
• When companies hire people, their decisions about whom to hire often come down to personal chemistry between the candidate and the hiring manager.
• When a large company decides on a marketing strategy, it’s often based on the hunches of senior marketing managers or on the advice of marketing consultants.
• When journalists — myself included — decide on an angle for a story about a new company, it’s usually based on some kind of hunch about the company rather than on exhaustive analysis of the company’s statistically likely outcomes.

There are exceptions: In environments that are relatively predictable, where repeated experience provides immediate feedback about the validity of your judgments, you can become expert enough to make valid predictions. That’s how chess masters get so good at analyzing positions at a single glance, and why experienced firemen have a “sixth sense” about when a floor is about to collapse.

Sadly, the tech world is not very predictable, nor does it provide immediate feedback. In situations like this, Kahneman advises, any kind of algorithm — even one based on common sense and scribbled on the back of an envelope — has more predictive power than an expert judgment.

That’s why I’m inclined to believe Vinod Khosla when he says that software can ultimately replace 80 percent of doctors. Predictive algorithms, well-designed checklists, and caring nurses can probably take care of people better than doctors can in many cases — leaving doctors to focus on the complex situations where their expertise provides real value.

So why aren’t people using algorithms more often to make business decisions? One problem is that they simply don’t have access to enough data about the outcomes of previous decisions. That’s where so-called big data companies could make a real difference.

For VentureBeat’s part, we’ve been making an effort to get more systematic about collecting data on the companies and products we cover, so we can make more accurate judgments about them — or give you the data to make your own judgments. For instance, our news team contact form, which lets you send news alerts to our reporters, is more structured than a typical email. That’s because it feeds into a database that, over time, will become a valuable resource for VentureBeat and its readers.

But we’re still only a tiny part of the way along this journey towards more algorithmic decision-making. Ditto for most of the tech industry.

How are you using data and algorithms? Let me know in the comments below or by email. I’d like to hear from you.

Photo credit: saturn ♄ via photo pin

Crowdfunding website IndieGoGo has flipped the switch on its promotional algorithm, which pushes campaigns to a coveted spot on its front page. The company says the algorithm adds more democracy to a human-bias driven industry.

“Like America, IndieGoGo is equal opportunity for all, but no guarantees,” said co-founder and chief executive Slava Rubin in an interview with VentureBeat.

IndieGoGo, like competitor Kickstarter, allows groups to go online, post a funding request, and accept money. Anyone, whether they have a big idea or individual project, can create a page and set a funding goal. Goals are not set in stone, however. Some competing crowdfunding websites won’t allow its users to collect promised money unless it has hit a funding goal. IndieGoGo, on the other hand, lets people choose whether to “fix” their funding to an all-or-nothing structure, or take whatever they can get.

The algorithm, which Rubin explained lightly given its proprietary nature, ranks different funding projects based on interactions with their IndieGoGo pages. It takes into account how many comments and social shares the page has, how many times the author uses the page to communicate, how often they update the page, as well as if the project is actually attracting funding. These factors all determine whether a project is featured on the homepage, as well as in newsletters and other promotional materials.

This process of choosing what to feature used to be a combination of both human decision and mechanical algorithms, but Rubin wanted the human element completely removed. For him, adding a human element took away the “democracy” of crowdfunding. He wanted everyone to have an equal chance of finding investors. In theory, this move should also promote users to engage on the website and draw in more traffic.

“We are really putting the power of exposure into the campaign owners’ hands and the community,” said Rubin. “If you do your work well, you do updates, you get people to fund, view, and promote…you know you’ll have a great opportunity to be on the homepage.”

Competitor Kickstarter recently grabbed attention for hitting its highest funding round yet: $1 million raised by two separate companies on February 9, 2011. The iPhone Elevation Dock hit $1 million, soon to be outdone by Double Fine Adventures, which hit $1 million in 24 hours that same day. Another project raised $1 million soon thereafter, putting the total for Kickstarter at three million-dollar-funded companies in two weeks. IndieGoGo’s highest amount funded is a bit more modest at $325,000.

IndieGoGo was founded in January 2008 and is headquartered in San Francisco, Calif. Recently, the company took on its own funding, a $1.5 million round led by MHS Capital (an investor in VentureBeat, see our ethics statement here) and Zynga co-founder Steve Schoettler.

Money tossing Iimage via Shutterstock

Have you ever wondered how Google’s search got so good? An insightful new video released by Google uses employee interviews and engaging graphics to show how Google’s core search product has evolved since it began in 1997.

While it’s easy to focus on evolving Google products like Gmail and Google+, Google still makes most of its revenues on search and advertising. And its core search product is still one of the most helpful ways to find content, pictures and videos around the web.

The video features short interviews from Google employees such as Vice President Marissa Mayer, Fellow Amit Singhal and Director of Product Management Johanna Wright. Each person helps explain a piece of how Google’s search evolved, including landmarks like AdWords, Google News, image search and Google Instant.

On top of adding new features, Google has stayed committed to delivering better and faster search results over the past few years. In June 2010, Google debuted a new search indexing infrastructure called Caffeine that improved the the frequency of real-time results by aggregating more pages and indexing in half the time. In early November, the company also tweaked its basic search algorithm to promote fresher content, affecting 35 percent of searches.

The image above, which is featured in the company’s blog post announcing the video, can be clicked on and expanded if you want to better see the chronology of search features that have been added to Google over the years.

And, finally, the fascinating six-minute video can be viewed below:

While it’s easy to focus on evolving Google products like Gmail and Google+, Google still makes most of its revenues on search and advertising. Thus, the company needs to keep you coming back for more searches, and wants you to be dependent on reliable and relevant results. It also wants to keep you away from Microsoft’s Bing.

“Given the incredibly fast pace at which information moves in today’s world, the most recent information can be from the last week, day or even minute, and depending on the search terms, the algorithm needs to be able to figure out if a result from a week ago about a TV show is recent, or if a result from a week ago about breaking news is too old,” wrote Google Fellow Amit Singhal on the Official Google Blog.

Google has made a commitment to delivering better and faster search results over the past few years. In June 2010, Google debuted a new search indexing infrastructure called Caffeine that helped improve the the frequency of real-time results by aggregating more pages and indexing in half the time. The changes announced today are different from Caffeine implantation because that was purely an infrastructure change where this is algorithmic.

The areas of search most impacted, according to Google, will be:

• Recent events or hot topics: For recent events or hot topics that begin trending on the web, you want to find the latest information immediately. Now when you search for current events like [occupy oakland protest], or for the latest news about the [nba lockout], you’ll see more high-quality pages that might only be minutes old.

• Regularly recurring events: Some events take place on a regularly recurring basis, such as annual conferences like [ICALP] or an event like the [presidential election]. Without specifying with your keywords, it’s implied that you expect to see the most recent event, and not one from 50 years ago. There are also things that recur more frequently, so now when you’re searching for the latest [NFL scores], [dancing with the stars] results or [exxon earnings], you’ll see the latest information.

• Frequent updates: There are also searches for information that changes often, but isn’t really a hot topic or a recurring event. For example, if you’re researching the [best slr cameras], or you’re in the market for a new car and want [subaru impreza reviews], you probably want the most up to date information.

The privacy bargain we make with tech companies usually involves giving up some personal data in return for a free service, as with Facebook or many mobile applications.

Doctorow argues that it’s hard for people to assign a value to personal data when the full consequences of giving up that data are still unknown. How do you determine whether the privacy bargain is a fair one?

“It’s hard to get worked up about things where the failure and the deed are separated by a long way,” said Doctorow. “It’s the same reason that people start smoking.”

He insists data-driven companies such as Facebook actively exploit users by soliciting as much data as possible. “Facebook trains you to undervalue your privacy. These companies are [full of] social scientists now and those people have read their Skinner (an American behaviorist), have read their Adler (founder of the school of the school of individual psychology) and they understand intermittent reinforcement.” In exchange for posting status updates, photos and other information, Facebook users are intermittently rewarded with attention from people they care about. This mechanism can have addictive qualities similar to gambling.

“Eli Pariser, who wrote The Filter Bubble, told me someone at Facebook explained to him that they know that men who have female friends who post photos of themselves, spend more time on the site,” reports Doctorow. “They know that women who see their friends post photos, upload photos in response. So if a man who used the site a lot then dropped off, they look for women in his social group, show them pictures of their girlfriends, the women post pictures back and then the men stay on. This is not the bargain.”

Another form of social manipulation practiced by tech companies involves search results and news feeds.

“The algorithms by which things like Facebook decide what to show you and what to hide are totally opaque. There’s this kind of weird, big lie about how an algorithm is not a form of editorial control. Google will say ‘we have organic search results’ in contrast with what Alta Vista used to do, where they would take payment to put a result first. It’s ‘organic’ because it’s done with math, but actually it’s editorial by another name. All the companies that do editorial by algorithm claim that there’s something about math that makes it free of bias and will.”

Tech companies often do not offer clear or easy privacy choices to users. Facebook constantly changes its privacy settings to push the default towards more public data, and its Byzantine custom privacy settings are bewildering for a new user. “Complexifying a proposition is usually there to stop you from finding out whether the deal is good,” comments Doctorow.

With mobile applications, the choice is often between giving the application all the data it requests or not installing it at all. “Imagine apps that let you iterate through privacy decisions when they arise, not making a lot of a priori decisions,” explains Doctorow. “Apps that start from a presumption of privacy, and when your privacy settings interfere with your stated desire to access a service, in that moment you are prompted to make the decision.”

More generally, Doctorow says we need simpler cookie managers: “One of the things you can do is give people meaningful choices in their browsers. That would be way more useful to me than giving them hard to enforce, impossible to audit, privacy legislation.”

He also thinks that the way we approach educating children about privacy is flawed. “We have this weird contradiction in our school system where all the grown-ups in the school spend all their time wagging their fingers at kids saying ‘Get off the Facebook, every disclosure you make is something precious that you lose forever ‘ but ‘I’m spying on every click you do, spying on every IM you send, spying on all your Facebook conversations’ just like a parent who has 3 fags in his mouth and says ‘You shouldn’t smoke because it’s bad for you,’” he says.

“We could start by teaching kids to jailbreak every device, break every firewall, to do all the things that will make them good at privacy. It’s a learned skill. If kids can compete to see who can divulge the least information to the grown-ups in their lives, we will, by definition, get kids who are better at not divulging information than kids who are punished every time they try to prevent grown-ups from looking at their information.”

One of the reasons that we undervalue our personal data seems to be that the threat is not visceral and concrete. “In technology we often have this core problem of taking a fairly abstract social harm and rendering it concrete,” concludes Doctorow. “I think science fiction is rubbish at predicting the future, but it can create narratives that become part of our discourse. Imagine it’s 1947 and Orwell hasn’t written 1984 yet, and you’re trying to explain to someone why you don’t want to be electronically surveiled.”