Airbnb CTO says graph neural networks will be big in 2021

Executives have to prioritize whether to experiment with cutting-edge technologies or wait to see results from other implementations first, Airbnb chief technology officer Vanja Josifovski said in a conversation with VentureBeat founder and CEO Matt Marshall at VentureBeat’s Transform 2021 virtual conference. Most enterprises — even large ones — have constrained resources, so they have to decide which technologies to invest in and which to wait out.

Typically, the decision is to use state of the art technologies in critical areas and avoid experimental or emerging technology in all the other areas, Josifovski said.

“It’s one of the hardest parts of my job because I do want to hire the best and smartest people, but then I do want to channel that ability into the areas that will provide business impact,” Josifovski said. “In some cases, [we] refrain from using state of the art until we think that we’ll get the return back.”

Josifovski and Marshall discussed some of the innovative trends in artificial intelligence (AI). “If we look at what’s happening today, there are some amazing technologies coming up,” Josifovski said, such as graph neural networks, transformer models, and language models.

Graph neural networks

Graph neural networks will be a major trend in 2021, Josifovski predicted. At its core, the deep learning paradigm is a different way of structuring data, like images, and sequencing data, like text. However, the data’s usage and the structure needed for the model to work can be rigid. Graph neural networks, in contrast, allow a more flexible architecture because the data defines the architecture of the model.

“Graph neural networks is a next iteration that allows us to use a lot more data within the deep learning framework in a much more natural way,” Josifovski said. “I feel that they will open a whole new area, where you’re going to be able to apply the deep learning paradigm a lot easier on a whole different set of data.”

Pinterest has used the model to build a recommendation feature, and Uber built a fraud detection model, for example.

Language models

While it is “an amazing technological achievement,” it may be too soon to work with large language models, Josifovski said. Being able to scale these models is a relatively new concept, but the challenge is finding the data to train the model. However, he added that using models in the production process requires predictability. There have been good examples of using the models to generate text and webpages. This is a good place to use the models because they aren’t “mission critical,” Josifovski said. In contrast, this type of work won’t fit well initially in machines like self-driving cars.

While language models don’t currently work with chatbots, Josifovski believes they will in the future.

Center for innovation?

In the early years, academia was the center of innovation and research for AI, with large companies developing some proprietary technologies, Josifovski said. Over time, waves of innovation in AI came from bigger companies, like Google, Amazon, Microsoft, and Facebook. As many of the technologies become commoditized, Josifovski predicts another shift, this time to smaller, independent companies. In areas like storage and cloud infrastructure management, well-resourced companies will provide the infrastructure to allow smaller players to develop AI.

“The center of gravity will slightly shift from the larger companies into smaller independent companies,” Josifovski said. “We will see a full ecosystem of companies that [have] been developed now and will shape the future.”