Ryff lets advertisers place any virtual object into commercials and films

Ryff has a big idea that it says could turn the $23 billion product placement market upside down. Product placement is the advertising tactic of placing a branded object, like a bottle of Coca-Cola, in a scene in a movie or a TV show.

Los Angeles-based Ryff has figured out how to do this digitally with cloud technology. Ryff figures out the places in video content where virtual objects can be placed in a scene where they seem like they are a natural part of the environment. That means the objects have to be rendered realistically enough so they can be mistaken for being part of a real scene, as recorded in a movie or TV show or a commercial, said Roy Taylor, CEO of Ryff, at an event on Thursday evening.

“We are on a new platform that makes images intelligent,” Taylor said. “Ryff is the world’s first image technology company using AI and visual computing to change the way we experience entertainment.”

Above: Ryff CEO Roy Taylor is a former Advanced Micro Devices and Nvidia executive.

Image Credit: Ryff

The company’s first product is called Placer, said cofounder Mark Turner, who first dreamed up the idea and shared it with Taylor. Placer uses computer vision and machine learning to identify places in videos where virtual objects can be placed. It identifies the brands that are already in the video, perhaps not intentionally. (That’s important for deleting objects that could bring on copyright lawsuits). Then the company works with brand owners to fashion an object that can be placed in the video.

“This is how we will bring billions of dollars into the media ecosystem,” Turner said.

It seems like a brilliant idea, but one that could be really hard to execute if the process turns out to be a lot more manual than anticipated. But Ryff has some believers. Tech Mahindra, one of India’s largest tech companies with more than 100,000 employees, is a cloud provider. Other supporters include Cisco, Dell, and Intel.

Amsterdam’s huge video production company EndemolShine (maker of shows like Black Mirror and Big Brother) is Ryff’s first major partner. In a demo, Ryff showed a Big Brother episode that was modified with new ads, implemented on the existing Verizon Digital Media Services platform with no modification required on their platform or client apps. In the episode, the Big Brother scene featured Ryff’s breakfast cereal placements because it was being viewed in the morning.

“Working with Ryff has opened up some really exciting conversations about how we can redefine product placement,” said Lisa Perrin, CEO of creative networks at Endemol Shine, in a statement. “Endemol Shine Group are at the forefront of innovation and we are continuously exploring the latest technologies, which have the potential to revolutionize how we create our content.”

LA Associates, the maker of a wide array of advertising content, is the marketing partner.

Above: A virtual object can be placed where the yellow marker is with Ryff’s Placer.

Image Credit: Ryff

The Los Angeles company came out of stealth last night at a party at a fancy mansion in Beverly Hills, California. I attended the event, and it had an interesting mix of techies, advertising agency executives, and Hollywood moguls.

“I looked at this and said, ‘Wow,'” said Tommy Drissi, head of Drissi Advertising (and a race car driver), onstage at the Ryff event. “This is a game changer.”

Taylor said that the company’s goal is to create a platform that can change any part of a video and insert a virtual object in real time. Over time, Ryff will be able to direct the changes to suit targeted advertising, so that the object being advertised is appropriate for the specific person watching it. The ad could even be hyperlocal, where a neighborhood shop could advertise to a particular consumer.

Above: Ryff’s Placer has put a bottle of Dove lotion in the scene.

Image Credit: Ryff

For instance, if the viewer is 19 years old, then Ryff’s Placer would show that viewer an ad for an object such as Red Bull, but it wouldn’t show an ad for alcoholic beverages. Working with a streaming company as a partner, Ryff could dynamically show a unique stream with individualized ads to suit certain viewers. (Ryff is targeting platforms such as over-the-top streaming networks, like Amazon or Hulu.)

“It’s massively disruptive,” Taylor said.

Ryff believes it can dynamically manipulate the images of products, people, backgrounds, and images in live or pre-recorded video — all modified to individual viewer preferences.

Currently, product placements in video are just like images in desktop publishing — infinitely adjustable during pre-production, but the second it is transmitted it becomes ‘flattened’ and an unchangeable part of the video frame contents.

Ryff’s intelligent image technology enables content objects to be as dynamic as data and offers creators and advertisers a new, flexible platform to seamlessly change, alter, or totally replace any object, person, or background in a video frame. Intelligent images can be changed at any time so different audience segments can experience a different product placement package in the same narrative story.

Above: Manish Vyas, president of a division at Tech Mahindra, addresses the crowd at the Ryff event.

Image Credit: Ryff

No person has to see the same product placement ad, even though all are watching the same video. And if Coke won’t place an ad in a spot, it can be replaced with a can of Pepsi. The objects should be interchangeable “media elements,” Taylor said.

The tech works not only with live shows, but with older libraries as well. For instance, you could place ads for new smartphones in an older movie like The Matrix, which came out when cell phones looked much different. It would actually enhance the movie for younger audiences if a brand placed a modern smartphone in the movie. And the product placement could generate new revenue for an old film.

“Where previously a car company might put a single model into a scene, and that would be fixed forever, they can now offer different models for different viewers in the same show at the same time. So, a family might see a sedan and a single person a coupe,” said Taylor. “Similarly, Studios can form a symbiotic relationship with individual viewers by offering them tailored placements fitting to their local tastes and customs.”

Above: Evan Smyth, CTO of Ryff.

Image Credit: Dean Takahasih

Of course, honchos like movie directors have to sign off on product placements, which are supposed to look real. To help address that, Ryff partnered with The Mill, a visual services company, to license The Mill’s Cyclops virtual production toolkit. Cyclops can slap any digital image on top of a box or even a race car with a digital QR code.

I saw a demo of the technology, where a branded bottle of vodka was placed into a scene. The animated bottle is placed on top of a box with a QR code. It’s supposed to look seamless, though the demo was a little jittery. Still, the idea is interesting. A brand advertiser can use Cyclops to show a movie director what a product placement would look like in a video, as if they were inserting an augmented reality image into a real world scene. The director could make a decision about using the virtual object on the spot, while still on the set.

The challenge is that the objects have to be shaped properly and placed perfectly in the scene or they will look unnatural, as if they were “floating” in the air above a surface, said Evan Smyth, a former Dreamworks technologist and current chief technology officer at Ryff, in an interview with VentureBeat.

Ryff hasn’t perfected the placement of objects yet, and one attendee noted that the tech seemed a little iffy. But Smyth said that the company is trying to figure out how much quality is good enough for the purpose of satisfying advertisers and viewers. That’s where the combination of machine learning, computer animation, and technical know-how come in.

Above: Mark Turner shows off Ryff’s Placer.

Image Credit: Dean Takahashi

“The trick is that it can’t be a manual process, because filmmakers can do that with green screens now,” Smyth said.

Ryff’s Placer product is designed not just for one insertion, as may be done in standard movie visual effects pipelines, but to enable insertions of multiple brands and products and for those insertions to happen automatically and for years after the content was created — without an artist needing to go back and manipulate the video again.

The goal is to scale the technology so you can put different kinds of objects, with foreign languages as needed, to address different audiences around the world, Smyth said.

Ryff is aware that somebody could use this technology to create “deepfakes,” or falsify images in a way that would weaken trust in media. The company said it will not make the content available to anyone who uses it for such purposes. The tech comes with that risk, but it also has a chance to bring in a lot of new advertising money as well as to lower costs.