12 tips for designing and managing an AI-driven product

Here’s a question that will keep future Artificial Intelligence (AI) entrepreneurs up at night: How do you manage a product when the software starts writing itself?

We’re not quite there yet, but as we build smarter, more complex software that has elements driven by AI we’re also making less predictable software. We know that AI will bring more capabilities to software, but it will also make software harder to design and manage since it will sometimes behave in unplanned ways. This is just a phenomenon that comes along with making complex systems. And that’s where we are going with software. This is where complexity theory meets software.

For most of us who have been entrepreneurs, executives, engineers, and product managers in the software industry, we have designed and managed software for decades while safely assuming a reasonable level of input-output certainty. This means when we input data we can easily figure out what the correct output should be. This is because we have been working mostly on simple systems: If you entered A and B into the input, C would come out. If you don’t get C, you know you have a defect that needs to be addressed. With simple systems, you can use the same set of test cases over and over again and expect the same outputs each time.

Intelligent agents and other dynamic AI-based systems turn this concept on its head as self-learning software continually adapts its outputs based on inputs from various interactions with other systems and people. Some systems today have gotten pretty complex (especially in the enterprise), but introducing more AI-based algorithms will accelerate complexity beyond where we’ve been in the past. We’ll have systems that go from being difficult to decipher to being indecipherable. And with intelligent agents, we’re massively increasing the number of potential inputs (sometimes, the input could be any combination of words in an entire language), which again dramatically increases the number of potential ways to interpret the input and provide a wider array of outputs.

For example, neural nets provide outputs based on inputs, but in between the input and output is the black box of computation. We won’t know exactly why the outputs were generated from those particular inputs. And new training (how the algorithm updates its learning) means that the outputs may change, even given the same inputs. So dynamic updates from a continuously learning piece of software means that there will be layers of learning that happen in real time that will impact outputs in ways that won’t be predictable. And some of these outputs will be fed into other parts of the system, creating additional layers of complexity. We are moving toward more complex system design. The term for the new, unexpected things produced by complex systems is “emergence.” And our software will only increase in emergent behaviors as we make them more complex.

For me, this is more of an observation and area of planning than a concern. We work everyday with people who are unpredictable. No one knows exactly all the reasons people do what they do from moment to moment. Yet humans have found ways to collaborate and get work done. For software, we’ll need to think through the issues as we build systems that become more complex. So, based on experience, I’ve created some fundamental tips that can help with the above issues as well as other issues when building AI-driven products and AI-based intelligent agents. Note: Depending on what you are building and your particular goals, you may need to ignore or alter some of the tips.

1. Domain focus

Limiting your domain can help limit complexity. It’s a good idea to simplify and focus some of the things that you have control of, like your software’s domain of expertise. Keep your product constrained to a narrow domain (focused on a logical set of tasks to do for the customer and a logical set of knowledge around an expertise, for example) and learn more before you expand into other domains.

2. Learning feedback loops

Every interaction is a chance to learn. Your systems should learn something from all (or almost all) interactions with humans and other systems. Feedback loops are needed for your software to self-correct and learn and to give you information about how to adjust your product and plan for the future. Within your domain, be cognizant of what to optimize for at a high level, but don’t over-optimize too soon. Although the AI product can be murky as you explore product feedback loops, you need to choose a more general, larger set of capabilities at first and then look for problems that you will be solving for the user. As your user interacts with the product, your product optimizations can be based on actual customer usage over time.

3. Human-in-the-loop

Sometimes a human brain is needed to augment the system. “Human-in-the-loop” refers to the situation when you can have a human complete some tasks to improve a user experience or to figure out something that is too difficult for the system. Designing this as part of your system will be useful for doing work or validating parts of a process that the system can’t do well yet. And the actions that the human takes can feed back into the system to train it to do the task better for itself in the future. Many companies building AI products use a human-in-the-loop to jump in and do some sort of work as part of their back-end.

4. Use all the context you can get

Context adds intelligence (or at least the appearance of intelligence). We’re collecting more contextual data than ever, and this context information will be needed for better AI-driven systems across a wide spectrum of industries. For many systems that interact with humans, context will be king. The abilities of intelligent agents will be expanded or constrained based on how much contextual data (location, related data, personalized information, etc.) the application can get. To progress, contextual information will have to be collected directly from the user and any other applications that can be accessed.

5. Detect failure (create complex/emergent failure detection)

Emergent systems require real-time performance evaluation. As we develop systems that operate dynamically, we’ll also need to rethink Q&A. Mainly, we need to think about how to augment current Q&A processes. There is more work to be done here, but we will need models for real-time error detection so that we can fail gracefully or have the system jump into another path of action. One way this could be done would be similar to how humans do it — by getting feedback from an independent observer. I’m talking about an application that constantly observes the production system and looks for abnormal or inaccurate behavior. Once it detects this type of behavior, it would give feedback to the main/production system so it can adjust its actions. This functions like a real-time performance evaluation, except it would be entirely digital. I imagine that this application could look similar to virus or spam detection software, where applications can look for a fuzzy determination of “normal” versus “abnormal” behavior.

6. Create smart failover reactions

Expect the unexpected. Humans are unpredictable, and combining unpredictable humans with unpredictable machines exacerbates the issue. Plan for smart failover experiences that can ask for clarity or clearly communicate confusion to the user. Plan ahead so that the user won’t get confused by the dynamic nature of the system.

7. Collect and maintain trainable, quality data

Use interactive systems to collect good data interactively. When designing inputs to the system via any interface, think about how you can check for the quality and trainability of the data you are collecting. If you are designing an intelligent agent, you can ask the user clarifying questions in real time. If not, you can still build techniques to ensure data quality upon input. There may also be old datasets that could be used to get started with a new customer. Quality will be a factor here, as well. Old data sets may not be well maintained and may need to be cleaned up.

8. Create an AI flywheel

Data from users can make the system more valuable, which can help you obtain more users and data, which can, in turn, make the system even more valuable. With AI-driven products, information can be collected from all the users on the system (and other systems) to make the system smarter, which in turn makes the system more valuable and attracts more users. When you attract more users, data can be collected from them that can feed into the software, and so on. This creates a flywheel of data collection and an increasingly intelligent system that builds upon itself. This is a way to create unique value over time, and it makes it difficult for competitors to catch up as the cycle creates its own momentum.

9. Create value for the user ASAP

Give value while collecting data. Balance the collection of data from the user with something that’s beneficial for the user. The ideal scenario is to provide value while you are learning. Also, if possible, find value in old data that can be loaded into the system through integrations with other systems. It’s good to plan for all the great things you can do with data collected in the future, but you have to offer some immediate value so that people stick around.

10. When creating intelligent agents, trust and knowledge must be built via interactions

If you are building an intelligent agent, the onboarding never ends. When it comes to intelligent agents, the initial proactive experiences the users have with the agent, combined with subsequent interactions, will drive how the user can and will use the agent long-term. So smart onboarding (introduction to the agent) and ongoing education of the user is key. Humans develop familiarity with others through repeated interactions over time. This is how humans will also interact with intelligent agents. If the user and agent haven’t communicated in a while, then the human may even forget about the agent altogether. Also, it’s important to think about how the user will discover what the agent can do. The agent may need to send reminders of new skills it has acquired or even simply provide a visual menu of what it can do. The important thing is to think about how all the capabilities will be presented to the user so that the user understands what the agent can do and remembers the agent. The proactive nature of these communications will drive the usage and user expectations needed to get to the other things on this list.

11. When creating intelligent agents, a hybrid interface is usually the right interface

In the long-term, I predict that intelligent agents will communicate better with humans than humans communicate among themselves. That’s because intelligent agents will have a wider variety of communication methods and input options than humans do. The best path for chat-based or other visual user experiences will usually not be to create a totally text-driven experience. An interface that contains both text elements and visual elements (buttons, etc.) is what we call a hybrid interface and will allow a wide array of input and output options that can be used in the right context to communicate most efficiently. Also, it’s at this interface point of interaction with users that number 7 (collect quality data) can be enforced. Artful communication with the user is needed to ensure collection of good information that can make the software smarter.

12. Create performance metrics

Managing a system requires managing metrics. Metrics are always important for business, especially when you start getting a significant data set from larger numbers of users of your product. Success metrics for AI-driven products will all be slightly different, but they will fall into predictable categories:

1. Quality of data collected that can be used for training

2. Quality of the modeling in order to generate the right output

3. AI flywheel growth measurement (for some companies)

4. Customer success metrics (for your particular business, including quality of output to users)

As systems get more complex, the right metrics will be needed to ensure you are managing your complex system well.

Final note on managing AI-driven products

Finally, behind many of these thoughts is a common philosophy. We have to start thinking about managing complex computer systems based on the latest AI-driven capabilities that are driving emergent behavior. And that comes down to managing the parameters, rules, and checks and balances of the system in a way that provides stability. Think about managing an economy. You don’t manage an economy (well) by explicitly dictating what the prices of goods and services will be. You manage at the higher system level. You set forward a general set of rules (laws) that make sense for that system and manage a few system-level variables (like the federal funds rate). And the independent agents (in this case, people) will make self-optimizing decisions to set prices by interacting with each other based on their independent needs and wants. Management of complex software systems is similar and will mean designing for good information collection, setting the right parameters, picking the right success metrics for your software, and turning the right knobs at the system level in order to keep the system in the best state of success. Therefore, part of AI-driven product management is really complex system design and will need more thinking from the perspective of complex systems.

Will Murphy is the VP of Product and Business Development and a cofounder at Talla, an AI-powered service desk for IT and HR.