Presented by Lucidworks

“We’re in the infancy of search.” 

        — Dave Schubmehl, Sr. Analyst IDC

On the cusp of search’s 30th birthday, that statement seems a bit startling. But keep in mind it took 20 years from when the first car was made to when Henry Ford put the  Model T on the road. It would be another 100 years before a commercially viable electric car was available. And now — almost 125 years after inception — we are on the brink of the self-driving car.

So if we are still in the infancy of enterprise search and discovery — as Schubmehl suggests — where are we headed?

Retail points the way

The best bet is to look at your search brethren in commerce and retail. Required to compete against low-cost online retailers and facing declining in-store sales, B2C companies have pushed for search applications that provide every shopper with a single view into all inventory — regardless of where it is shelved.

This means retailers have to deal with sometimes millions of sales keeping units (SKUs) within a catalog. And tracking those is no easy task because manufacturers rarely adhere to a uniform schema or canonical structure, so stores need a way to classify and categorize items rapidly.

That, of course, has to happen to make those SKUs available not just for search but for excellent search. Too many results, too few, or too many irrelevant ones, and discouraged shoppers will flee — usually to a competitor.

Workers are also shopping — for knowledge

So what do shopper habits and understanding customer intent have to do with workplace knowledge?

Well, just like the world of commerce, the workplace environment is fraught with options — so many channels and silos where information can be squirreled away and hard to find. Is that PDF you need right now to do your job in your email? Is it in one of the myriads of collaboration tools your team uses? Or could it be in one of your very specialized systems of record like Salesforce, Netsuite, Marketo, Dropbox, DocuSign, or Jira?

Many of these solutions are in the cloud, with disparate data formats. While these systems offer search, workers have to execute individual searches across all these various and sundry silos. Further, assets stored by one department may be called something else by another department. So searching heterogeneous data, across multiple silos, with unique vocabularies, is pretty much exactly what commerce brands have had to deal with all these years.

The most successful organizations are taking their cue from the most successful retailers and implementing search engines powered by AI (and, more specifically, machine learning) to:

  • Understand intent to better gauge what users are looking for
  • Predict and anticipate what they might be looking for next
  • Iterate and improve these results based on the behavior of similar users

Machine learning helps figure out intent

In search, the goal of machine learning is to use automated learning techniques to improve results — whether searching a list of products in a catalog or a list of PDFs on an intranet. This includes being able to search across all data types and data sources quickly.

Many organizations start with basic text search and spend years trying to manually optimize synonyms lists, business rules, ontologies, field weights, and countless other aspects of their search configuration.

But, some are beginning to realize that most of these processes can be automated. This not only saves employees from performing the tedious task of wading through endless log files — but it allows those resources to provide higher value functions for their companies.

Autocomplete points users to what they really want

In the case of retail, one of the tactics relied on is autocomplete, which is, of course, that feature that fills in the word as you type. Autosuggest is a related feature that offers a list of semantic matches. Automating synonym detection and misspellings is an important part of both.

These features are critical in helping people navigate to what they are really looking for. They can either refine the search (for example, a green chair under the category of sustainable furniture vs. a green chair under upholstery choices) or offer the correct spelling.

Computational models analyze historical and real-time user signals (what they searched for, how they revised the query, what they clicked on, what they didn’t, what they downloaded, what they ignored) to help determine intent. So as people search, browse and discover, the machine learns — providing ever-more precise results.

Clustering and classification aids ingestion

One of the ways of dealing with heterogenous data is by using machine learning. One of the biggest challenges is badly labeled and unlabeled data. Clustering and classification techniques let the machines ingest large amounts of data — regardless of type or schema. The models are refined by incorporating user feedback signals.

Imagine, too, being able to add in directory services and authentication mechanics like LDAP (Lightweight Directory Access Protocol) and other methods that let the machine validate user permissions by role within the company, including locale, so they only see search results they have access to given their team, role, or region.

Enterprise search revisited

Machine learning has produced amazing results for commerce companies. Hyper-personalization that predicts the types of products individuals may want to buy has seen conversions go up significantly.

The same techniques can do the same with enterprise data. If you are a geologist working in Alaska — chances are you’re looking for surveys from that area. Or if you are in customer support for an automotive parts manufacturer, and you have the customer’s information, relevant support documentation should be weighted so that it shows first.

And intelligent customer support centers are evolving into even smarter customer self-service portals. Machine learning is giving way to deep learning to achieve true neural information retrieval (neural IR).  This deep learning is crucial for natural language understanding. Like for customers, it’s easy to see how machine learning will help anticipate what employees are looking for.

Machine learning techniques are the growth engine for search improvements — especially as data continues to grow.

While retailer needs are pushing search out of its infancy, workplace demands are just as vital. Improving search increases efficiency, reduces duplicating work, and creates economic opportunity. Generation Z, the cohort after millennials who make up 25% of the U.S. population, are beginning to push into the workforce. These digital natives will only suffer irrelevant search for so long — before they, like shoppers, jump to other businesses.

Diane Burley is VP Content for Lucidworks. A former journalist and multi-media executive, she has been a content officer for technology companies. As a storyteller, Diane helps executives in all industries understand technology’s role in solving many of the challenges they face today. In her off-hours, she is a court-appointed advocate for a foster teen.

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. Content produced by our editorial team is never influenced by advertisers or sponsors in any way. For more information, contact