VentureBeat presents: AI Unleashed - An exclusive executive event for enterprise data leaders. Network and learn with industry peers. Learn More
In the 2000s, the cloud began to take off. Programmers and businesses started to procure virtual compute resources on demand to run their software and applications.
Over the last two decades, developers have grown accustomed to, and reliant on, instantly available infrastructure managed and maintained by someone else. And this is no surprise. Abstracting hardware and infrastructure away enables developers and companies to focus on product innovation and user features above all else.
Amazon Web Services, Microsoft Azure and Google Cloud have made storage and compute ubiquitous, on-demand and straightforward to deploy. And these hyperscalers have built robust, high-margin businesses atop this approach. Organizations reliant on the cloud have traded capital expenditures (servers and hardware) for operating expenditures (pay-as-you-go compute and storage resources).
Enter federated learning
Although the cloud’s ease of use is a boon to any upstart team trying to innovate at all costs, cloud-centric architecture is a significant cost as a company scales. In fact, 50% of large SaaS company revenue goes toward cloud infrastructure.
An exclusive invite-only evening of insights and networking, designed for senior enterprise executives overseeing data stacks and strategies.
As machine learning (ML) continues to grow in popularity and utility, organizations store an increasing amount of data in the cloud and train larger and larger models in search of higher model accuracy and greater user benefit. This further exacerbates reliance on cloud providers, and organizations find it difficult to repatriate workloads to on-premises solutions. In fact, doing so would require them to hire a stellar infrastructure team and re-architect their systems altogether.
Organizations are looking for tools that enable new product innovation and offer high accuracy with low latency while still being cost-effective.
Enter federated learning (FL) on the edge.
What is Federated Learning on the edge?
Federated learning, or collaborative learning, takes a different approach to data storage and compute. For example, whereas popular cloud-centric ML approaches send data from your phone to centralized servers and aggregate this data in a silo, FL on the edge keeps data on the device (that is, your mobile phone or your tablet). It works in the following way:
Step 1: Your edge device (or mobile phone) downloads an initial model from an FL server.
Step 2: On-device training is then conducted; data on the device improves the model.
Step 3: The encrypted training results are sent back to the server for model improvement while the underlying data sits safely on the user’s device.
Step 4: With the model on the device, you conduct training and inference on the edge in a completely distributed and decentralized way.
This loop continues iteratively and your model accuracy increases.
Federated learning benefits for the user
When you aren’t reliant on or bottlenecked by centralized data, the user benefits in dramatic ways. With FL on the edge, developers can reduce latency, reduce network calls and drive power efficiency all while promoting user privacy and improved model accuracy.
FL on the edge is enabled by the ever-increasing hardware capabilities of the phones in our pockets. Each year, on-device computation and battery life improve. As the smartphone processor and hardware in our pocket improves, FL techniques will unlock increasingly complex and personalized use cases.
Imagine, for example, software that sits on your phone in a privacy-centric way that can automatically draft replies to incoming emails with your individual tone, punctuation style, slang and other hyper-personalized attributes — all you have to do is click “Send.”
Enterprise pull is strong
In my conversations with multiple Fortune 500 companies, it has been blindingly obvious how much demand there is for FL on the edge across sectors. CTOs express how they’ve been searching for a solution to bring FL techniques on the edge to life. They reference the millions of dollars spent on infrastructure and model deployment that could otherwise be saved with an FL approach.
In my opinion, the three industries that have the most potential to reap the rewards of federated learning are finance, media and ecommerce. Let me explain why.
Use case 1: Finance — improved latency and security
Many large multinational financial companies (e.g. Mastercard, PayPal) are eager to adopt FL on the edge to assist them with identifying account takeovers, money laundering and fraud detection. More accurate models are sitting on the shelf and have not been approved for launch.
Why? These models increase latency just enough that the user experience is negatively impacted — we can all think of apps we no longer use because they took too long to open or crashed. Companies can’t afford to lose users for these reasons.
Instead, they accept a higher false negative rate and suffer excess account hijacking, laundering and fraud. FL on the edge empowers companies to simultaneously improve latency while showing relative uplift in model performance compared to traditional cloud-centric deployments.
Use case 2: Media — hyper-personalization
In the media sector, companies like Netflix and YouTube want to increase the relevance of their suggestions for movies or videos to watch. The Netflix Prize famously awarded $1 million for a 10% uplift in performance compared to its own algorithm.
FL on the edge has the potential to have a similar impact.
Today, when a new show is launched or a popular sporting event is live (like the Super Bowl), companies reduce the signals they gather from their users. Otherwise, the sheer volume of data (at a rate of millions of requests per second) causes a network bottleneck that prevents them from recommending content at scale.
With edge computing, companies can use these signals to suggest personalized content based on insight from individual users’ tastes and preferences.
Use case 3: Ecommerce — more timely and relevant suggestions
Ecommerce and marketplace companies want to increase click-through rates (CTR) and drive conversions based on real-time feature stores. This enables them to re-rank recommendations for customers and serve more accurate predictions without the lag of traditional cloud-based recommendations.
Imagine, for example, opening the Target app on your phone and getting highly personalized recommendations for products in a completely privacy-centric way — no identifying data would have left your phone. Federated learning can increase CTR thanks to a more performant, privacy-aware model that offers users more timely and relevant suggestions.
The market landscape
Thanks to technological advances, large corporations and startups alike are working to make FL ubiquitous so that companies and consumers alike can benefit. For companies, this likely means lower costs; for consumers, it may mean a better user experience.
There are already a few early players in the space: Amazon SageMaker allows developers to deploy ML models primarily on edge devices and embedded systems; Google’s Distributed Cloud extends its infrastructure to the edge; and upstart companies like NimbleEdge are reimagining the infrastructure stack.
While we are in the early innings, FL on the edge is here and the hyperscalers are in an incumbent’s dilemma. The revenue that cloud providers earn for compute, storage and data is at risk; modern vendors who have adopted edge computing architecture can offer customers premium ML model accuracy and reduced latency. This improves user experience and drives profitability — a value proposition that you can’t ignore for long.
Neeraj Hablani is a partner at Neotribe Ventures focused on early-stage companies making breakthrough technologies.
Welcome to the VentureBeat community!
DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.
If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.
You might even consider contributing an article of your own!