Presented by Elastic

From HAL 9000 in “2001: A Space Odyssey” (1968) to KITT in “Knight Rider” (1980s), to JARVIS in “Iron Man” (2008-), to the emergence of today’s Siri, Alexa and chatbots, we have been waiting for truly intelligent digital assistants for a long time. With the explosion around large language models (LLMs), the wait is nearly over. We will soon have access to business and personal AI-powered digital assistants that understand our naturally phrased questions, know enough about us to formulate a relevant answer and present the answer in a way that is helpful to a current situation.

However, not all digital assistants will be created equal. Whether it’s depth or breadth of knowledge, ease of use or cost, we will have to choose our assistants wisely to get the most value from them. As we review the technology landscape, the best digital assistants — those that offer the deepest insight, the greatest flexibility and the most value — will likely come from an open industry approach.

Open vs. closed approaches

An open approach to software means that the code is housed in public repositories committed to an open development process and transparent and direct engagement with a community. In a closed environment, the code is a tightly held secret.

The general advantages of the open approach are well understood. It provides access to an often large and collaborative community of developers to enable rapid customization and innovation. It enables companies to adopt a solution at little or no cost and then add fee-based features or services as needed. And it offers greater visibility into the source code, enabling independent verification of security capabilities and fostering greater trust. Open solutions like Apache Web Server, Linux, Mozilla and WordPress helped establish the foundation of today’s communications infrastructure. Git, an open and distributed software version control system, is transforming source code management. And open solutions like Google’s TensorFlow, Facebook’s Pytorch and the Hugging Face Transformers library are helping to power the next wave of AI development. 

Meanwhile, the drawbacks  of a closed approach are particularly troubling in the age of AI. Only a handful of corporate leaders control decisions about how to respond to industry and customer trends and what new capabilities to add to the platform. These decision-makers are often focused on maximizing profits, which can limit the directions they take and the benefits they deliver to smaller customer segments.

The closed approach creates vendor lock-in, which limits the customer’s ability to access innovations not prioritized by the vendor. It can also restrict a customer’s ability to control costs because switching to a new solution can be so expensive and disruptive that closed vendors have little incentive to drive down their costs.

With AI in a period of such rapid evolution, adopting a closed solution can seriously limit an organization’s access to the latest AI capabilities, undermining its ability to innovate, adapt and compete.

The closed, “black box” approach is also supposed to enable greater security because the code is protected, and vulnerabilities can be patched before they become a widespread problem. However, no single company will ever master all the cybersecurity challenges. This means customers are only as safe as the vendor’s ability to hire the best cybersecurity experts and developers and keep up with evolving threats, all of which may be impacted by financial or operational decisions that have nothing to do with security. Further, there’s no guarantee that the code will be patched promptly or that the vendor will even acknowledge the threat so that customers can protect themselves.

An open digital assistant

In the era of generative AI, security and transparency are paramount. The benefits of digital assistants are possible thanks to the ability to train the new LLMs on tons and tons of relevant, high-quality and often proprietary data. If this data isn’t protected, the potential for bad advice, leaked private information and other abuses is huge.

To satisfy customers and regulators, companies that depend on AI must be able to verify who has a right to access what data, whether consent to collect the data is necessary and granted, whether the underlying machine learning models violate reasonable assumptions of privacy and whether the use of the data and model could in any way constitute “inappropriate behavior,” such as bias against particular groups of people.

In an open environment, one of the most important goals for the community is to collaborate to ensure transparency and improve the security of the software. Vendors share code, detection rules and artifacts to improve every organization’s ability (paying customer or not) to understand the behavior of the AI model and protect their systems and data from intrusions and exploits.

Along with security and transparency, rapid innovation is more important than ever. The pace of new AI development is unprecedented. Digital transformation has accelerated every business process, including development, leading to a flood of daily announcements related to the use of generative AI. Businesses in every industry are looking for ways to leverage the new technology to accelerate their own innovation and enhance customer experiences. These companies just can’t afford to be locked into inflexible solutions that don’t rapidly incorporate the latest innovations.

An open approach to digital assistants encourages collaboration across very large communities to deliver more innovation faster and test these innovations thoroughly, so companies can take advantage of the latest LLMs, make connections to their other enterprise software and proprietary data sets and create truly useful digital assistants that were the stuff of fantasy just a couple of years ago.

Elastic is setting the course ahead

Elastic has spent the past decade building open enterprise tools and is committed to security and transparency. The foundation of this effort is the Elasticsearch Relevance Engine™ (ESRE™), which combines the best of today’s new AI transformer models with Elasticsearch’s unrivaled search capabilities. ESRE gives developers a full suite of sophisticated retrieval algorithms and the ability to integrate with their choice of LLMs. ESRE is also accessible via a simple and unified API that lets developers immediately start applying increased search relevance to any use case.

Based on ESRE, the company’s new Elastic AI Assistant is the type of open solution that will help advance the discussion around the power of generative AI and digital assistants to transform work. The solution leverages proprietary data and a range of LLMs to help businesses create domain-specific applications that enable users to perform skilled tasks, regardless of their individual skill level.

In the area of security, for example, the Elastic AI Assistant allows users without deep security expertise to easily interact with the Elastic Security solution using natural language. A user who wants to collect security information from the infrastructure but is unsure of the best way to do so in Elastic can simply ask the Elastic AI Assistant to help. The AI Assistant can also respond to a security alert trigger by leveraging all the available public and proprietary data sources to provide information about the nature of the alert and a best-practices approach to remediating it if necessary — all in an easy-to-read form. The AI Assistant is even helping companies slash the time required for migrating to the Elastic Security solution from other solutions by letting users continue entering queries in the form they are familiar with, while the AI Assistant automatically converts them into queries appropriate for Elastic.

The application of the Elastic AI Assistant to cybersecurity is just the tip of the iceberg of its potential to simplify and accelerate business processes. And that’s just the beginning. The technology is here. The benefits are clear. It’s just a matter of time before the use cases are defined, the security issues are solved, and the solutions are delivered, launching the era of truly useful digital assistants that help in every area of our lives.

Dig deeper: Learn more here about the Elasticsearch Relevance Engine / Elastic AI Assistant.

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.