Last week, Facebook global office admitted multiple errors in its metric reporting including organic reach, video, Instant Articles and referrals. In response to the misreporting, the AANA (Australian Association of National Advertisers) have released a statement calling for more third party verification for Facebook data.
Several weeks ago, Facebook admitted it was only counting video views of more than three seconds when calculating its “Average Duration of Video Viewed” metric. By eliminating videos under three seconds, the average engagement was inflated. The company said it was introducing a new metric, “Average Watch Time”, which will replace the earlier metric.
In an update on Facebook’s Metrics and Reporting page, the company said: “Today we’re updating our metrics to give our partners and the industry more clarity and confidence about the insights we provide. We know that having access to reliable metrics is important to the millions of partners who use our services to growth their business.”
Page Insights – Organic Reach
We’ve uncovered a bug in Page Insights. On one of our Pages dashboards, one summary number showing 7-day or 28-day organic page reach was miscalculated as a simple sum of daily reach instead of de-duplicating repeat visitors over those periods (see red circle in screenshot below).
However, the vast majority of reach data in the Page Insights dashboard (marked green in screenshots below) was unaffected, including all the graphs, daily and historical reach, per-post reach, exported and API reach data, and all data on the Reach tab. The de-duplicated 7-day summary in the overview dashboard will be 33 per cent lower on average and 28-day will be 55 per cent lower; data in other fields is unaffected. This bug has been live since May; we will be fixing this in the next few weeks. It does not affect paid reach
Video – Measuring Completions
When partners upload their videos to Facebook, the full video length is recorded, but when the video delivers to people’s devices, the length of the video can sometimes be a fraction of a second shorter or longer. This occasionally happens when the audio and video track don’t line up, owing to differences between video players and devices. While someone may watch a video to completion on their device, the audio may continue to play for a bit longer. This particular issue caused us to undercount the metric “video watches at 100 per cent” (previously named “video views to 100 per cent”). Moat found this and reported it to us. We are now updating how we read the video length to address this issue. This may result in roughly a 35 per cent increase in the count of “video watches at 100 per cent.” For example, if “video watches at 100 per cent” were 1 per cent, they would now be 1.35 per cent.
Instant Articles – Time Spent
Instant Articles is a way for any publisher to create fast, interactive mobile content on Facebook. We also provide publishers with insights to help them understand how their content is consumed. We’ve determined that the average time spent per article had been over-reported by 7-8 per cent on average since August of last year. This was caused by a calculation error: we were calculating the average across a histogram of time spent, instead of reflecting the total time spent reading an article divided by its total views. We have now fixed this issue.
Analytics for Apps – Referrals
In our Facebook Analytics for Apps dashboard, one metric called “Referrals” is miscalculated. This metric evaluates all posts produced by people via an app or website. We meant to count clicks that went directly to an app or website; however, we’ve also counted other clicks on those posts via the app or website, including clicks to view photos or video. We are working to fix this. Out of the referrals we currently report, on average about 30 per cent are actually clicks to consume content on Facebook. For power users of this metric (top apps that look at this data in the dashboard most frequently), we found that referrals have been overstated by approximately 6 per cent on average. Other measurements of referrals, such as those appearing in Facebook’s ads reporting tools, are unaffected.
Clarifying Metrics
We’ve been working for several months to improve reporting, and today we’re announcing several updates intended to make information clearer and easier to understand. Some of the updates we’re making are described below; for all we’ll be updating our in-product definitions.
- More descriptive names: We’ll make updates now and in the coming months to ensure our metrics are clearly named (e.g. “view content” → “website view of content,” and “video views” → “3-second video views”).
- Clarified calculations: We’ll clarify some calculations to make sure they reflect how our products are being used by advertisers. For example, with the launch of Canvas in February, we gave advertisers the ability to create an immersive native experience on Facebook. Within Canvas, they can also link to offsite content on Facebook’s in-app browser. While this experience is valuable to advertisers, the original intention of the Canvas View Duration metric was to capture time spent in Canvas. So we’re clarifying the metric calculation to reflect that it excludes time spent outside Canvas (linking to offsite content).
- More consistent definitions: We’ll also update our tips section and glossaries across all of our ads tools to make sure we are explaining our metrics in consistent ways (including noting which insights are estimates). For example, as we have rolled out new immersive experiences on Facebook like Canvas and lead forms, the link clicks metric has evolved over time to include clicks to those on-Facebook destinations, in addition to clicks to a website off of Facebook. We had updated the definition of links clicks in our ad creation tools, and we are now updating it across all of our ads tools. We’re also clarifying the description of our video views metrics (3, 10 and 30 seconds) to include the term “aggregate,” so as to more clearly explain it’s based on aggregate view time of at least the specified number of seconds.
- Better categorization: We’re making it easier for marketers to select certain metrics to include in their reports. To do this, we’re creating a way to customize the columns in reports to reflect how marketers set up their ads. This will let advertisers have a specific advertising objective map directly to the metrics we show, like checkouts, leads or registrations.
AANA STATEMENT RE FACEBOOK MISREPORTING REVELATIONS
Media is traditionally the single largest marketing expense for a brand owner and marketers need to be able to make and defend these investments with confidence. For decades, advertisers have relied upon a standard of accepted practice for traditional media that requires independent reporting and verification of their audiences and their usage patterns, which have typically been supported by all competitors in a specific media category.
It is this third party verification that Facebook is currently lacking. The latest revelations by Facebook cast a cloud of doubt over how reliable Facebook’s data and self-reporting is.
Equipping advertisers to achieve transparency in the media buying ecosystem has been high on the AANA’s agenda this year as it has been with our counterparts around the globe, like the Association of National Advertisers in the U.S.
The AANA believes the over-riding principles of comparability and accountability should govern all players in the media ecosystem. The AANA is encouraged by Facebook’s willingness to address the issue with third party providers and will await more detail. The AANA sees no reason why Facebook should not abide by the standards of independent verification and auditing that more traditional media players have signed up to for decades in order to assure advertisers of their media investments.
If Facebook and others don’t move voluntarily to an independent verification, it will require the major advertisers around the world to cooperate and crack this problem and achieve independent accountability in the digital space.
The AANA is in conversations with the ANA (Association of National Advertisers in the US) and supports their efforts to encourage Facebook to accept independent verification.
