We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 - 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Register today!

Facebook launched its Collection ad format for ecommerce in early March of this year amid a lot of fanfare. Given that I oversee millions of dollars of ad spend per month, I was more than a little curious about its impact. So I ran a $177K test, pitting the new Collection ads agains standard single video ads. Below are the results of that test.

Before we get to the results, though, here’s what you need to know about Facebook’s Collection ads if you haven’t been exposed to them yet.

A Collection ad is a mobile-only newsfeed ad that combines a video (or image) with a product catalogue of up to 50 products related to your featured video (this is the grid layout, which was the original release).

Four products (of your choice) are shown below the video, and if potential customers want to know more, clicking on your product catalogue below the video opens more choices.

Facebook claimed this ad format would reduce “the friction (and cost) of advertising coming from mobile phone and video ads.”

This sell was based on two sound premises:

1. Mobile usage is increasing. Collections claim to deliver faster load times and a more seamless mobile browsing experience. On releasing Collection ads, Facebook cited research claiming, “45 percent of all shopping journeys include a mobile action, but slow load times cause friction in that process.”

2. Video is increasingly important in buying decisions. And it’s continuing to grow as a major influencer in buying decisions.

This profit generating theory was backed up by citations from great brands with highly impressive results, such as Addidas, claiming a 43 percent decrease in cost per conversion and a 5.3x return on ad spend.

Do Collection’s promises apply to ‘non-big brand’ advertisers?

Most of our clients have an ad-spend budget of between $50 – $6,000 per day, and they expect their advertising dollars to work very hard for them.

As someone who lives and breathes performance marketing campaigns, I’m highly skeptical when anyone talks about how well their advertising did for a “well known” brand.

So the questions I set out to answer with my test were:

  • Do Collection ads outperform traditional ads for the average advertiser?
  • If so, how well do they perform?
  • And under what circumstances?

After 13 separate campaigns, 2.5 months and a $177,843.34 ad spend, I got my answers.

The campaign setup:

I studied my client base and identified 13 campaigns to fit the bill:

  1. All e-commerce stores
  2. All had a video ad that was selling successfully
  3. All had mobile newsfeed traffic as one of the most profitable (if not the most profitable) driver of sales from Facebook

For this test, I made sure both the single video Facebook ad and the video Facebook Collection ad were exactly the same, other than the ad format.

The ads were also tested in the same ad set, so there was absolutely no difference between the ads other than the format.

It’s also worth noting that each video ad was a square video with captions, as this generally produces much better results on mobile. If you’re not using this format, test it; you might just double your CTRs overnight.

So … which format won?

After spending $177,843.34, and generating over $1,122,853.55 in sales, we were ready to dive into our analysis. Below is breakdown of cost per sale (CPS), average order value(AOV), and ad spend (ROAS) by campaign (sales are discounted by approximately 25 percent):

ROAS analysis:

  • 69.23 percent (9 out of 13) clients increased their ROAS simply by switching to Collection ads
  • Of those whose Collection ads produced better results, the average increase on ROAS spend was a clear 29.40 percent
  • If Collection ads worked for a campaign, they produced a clear winner. One campaign (campaign 8) did 89.74 percent better than the single video counterpart
  • Overall ROAS change was on average +22.04 percent, with a range of –24.08 percent to +89.74 percent

CPS analysis:

With an overwhelmingly positive response to Collection ads in this test, I expected the CPS to follow a similar pattern in the opposite direction for each campaign. And …

  • Like the ROAS results, 69.23 percent (9 out of 13) campaigns showed a CPS decrease
  • Three of the campaigns where the CPS rose (campaigns 3, 4, and 5), were the same campaigns where the ROAS was negative
  • The highest performing campaign in terms of ROAS change was driven more by a significant drop in CPS than an increase in dollars/sale (campaign 8)
  • Overall CPS was still significantly less at -18.52 percent, with a range between -43.27 percent and +13.74 percent

AOV analysis:

  • 8 out of 13 campaigns (61.54 percent) showed a better outcome with Collection ads
  • Overall, average order value percentage change was negligible at -0.57%, with a range of between -24.83 percent and +23.49 percent

Overall, the results were extremely positive, with a high percentage of clients experiencing great results and an overall increase in ROAS.

If you prefer a more visual format, you can find that in this infographic.

Having managed millions in Facebook ad spend, I can say that without a doubt, audience selection can be the key difference between a losing campaign and a wildly profitable one that can scale businesses. This is why I focused most of the budget on my favorite four audiences for e-commerce:

  • Added to cart retargeting – visitors who viewed the primary product page AND visitors who added our product to cart (must satisfy both conditions)
  • Product category retargeting (exc. added to cart) – Visitors who viewed our product and product category page but DID NOT add to cart
  • Video view retargeting  – this is a retargeting list we created in Facebook of anyone who watched more than three seconds of a video we promoted on Facebook
  • Product category buyer lookalike audience (one of my favorite cold audiences) – This is a 1 percent Lookalike (LAL) audience of buyers who have bought at least one item from that product category, or products closely related to the promoted product.

This is testimony to Facebook’s success in reaching its objective of reducing friction.

Here are the results of the individual campaigns:

In conclusion, my team came away from the test with five main insights:

1. If a Collection ad works, it will quickly produce a clear winner, so start testing immediately. Even if your Facebook ads are marginally profitable, you could be leaving money on the table every single day that you’re not testing collection ads.

Of the campaigns where collection ads produced a better result, the average increase on ROAS was 29.40%, with one campaign doing close to 90 percent better. Across all campaigns (winning and losing), ROAS grew by an average of 22 percent on our collection ads, with results ranging from -24.08 percent to +89.74 percent.

Keep in mind, this is without any further segmentation. We used segmentation following this experiment to drive up returns and profits even further.

2. Even if a few of your Collection ads fail, the probabilities are on your side, so keep testing. 70 percent of our Collection ads beat our single video ads by delivering a higher ROAS.

3. Look for a drop in cost per sale as the main driver to higher returns. On average, we experienced a CPS drop of 18.52 percent, with results ranging from a 43.27 percent drop to a 13.74 percent increase. This was the main driver to higher returns on most of our campaigns.

4. Don’t expect collection ads to have much, if any, impact on your average order value. In fact, your average order value may decrease (as it did for us across the board by 0.57 percent). Still, the probability is on your side, with over 60 percent of campaigns showing people purchasing more per sale with Collection ads.

5. A big increase in ROAS will lead to an even bigger increase in profits. At an average margin of 40 percen, a 22 percent increase in ROAS led to profits from our campaigns growing by over 44 percent. One of our clients’ ROAS grew close to 90 percent, which led to an approximate growth in profits of 180 percent!

[A version of this story originally appeared on the author’s company blog.]

Ash Aryal is CEO of digital marketing agencies Digital Spotlight and Quantumlinx and manages millions of dollars in ad spend per month across Google, Facebook, and Bing for both local businesses and multinational Brands.

VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Learn more about membership.