The Pandora’s Box Of Programmatic Advertising

[This is the first part of a three-part series investigating the murky world of programmatic advertising.] How often do you think about your conception and comprehension of size and scale?

The world of programmatic digital advertising is an exercise in mind-bending complexity and jargon with ever-changing and evolving types of inventory, buying processes for that inventory and means to assess and measure the impact of a single creative placed on a single piece of inventory on a single website. It doesn’t help that the inventory, processes and measurement systems are given increasingly esoteric and unhelpful names.

But it is the scale of the programmatic advertising market that truly boggles the mind. The industry, which did not exist even thirty years ago is worth billions of dollars and, despite its size, it has an outsized impact on the world we live in and the way our world is run – because we want a free internet.

The free internet is funded by advertising and that money can be sent, through the programmatic buying process to sites ranging from the Daily Mail to YouTube and on to smaller more obscure websites. However, while the flow of those ad dollars has created millions of jobs around the world, it has also caused serious environmental harm and played a big factor in the general political meltdown the world has witnessed over the last decade or so.

The remedy does not lie with advertisers — though brands and agencies both large and small can and should play a major role in helping the industry to that point. The large adtech platforms bear the brunt of the responsibility for fixing the problem — and you will see that most of them did not want to talk to B&T for this series of articles. But, governments and regulators, as well as the large social platforms, also have significant roles to play in extricating adland from the quagmire it has found itself in.

Though, some in the industry that B&T has spoken to have said that while this quagmire was an unintended consequence, the large ad networks have worked deliberately to put media agencies on the back foot through unnecessary complexity and the clandestine machinations of the systems they designed.

A Numbers Game

At the end of March, Google proudly proclaimed that it had removed more than 5.2 billion adverts from across circulation on the internet, as part of its Ads Safety Report. In addition, the Search giant restricted some 4.3 billion adverts from being seen around the web and suspended more than 6.7 million advertiser accounts.

“We have thousands of people working around the clock to create and enforce effective advertiser and publisher policies to prevent abuse while enabling publishers and businesses of all sizes to thrive,” said Alejandro Borgia, Google’s director of product management, ads safety, upon the release of the report.

These adverts violated Google’s policies for advert content. These policies are extensive and are regularly updated by the company to address the pressing issues of the day. As one would expect, ads spruiking harmful products such as weapons and drugs are banned, as are ads designed to mislead users into making poor financial decisions.

Google also said that it “took action” against more than 1.5 billion publisher pages and 143,000 sites for violating its policies around ads appearing alongside harmful content.

Again, these policies for its AdSense system that places ads on websites, cover the things you might expect. Websites with sexual content are banned from Google’s ad network, as are those with gruesome or graphic images and text. Online gambling sites and those promoting the sale of recreational and pharmaceutical drugs are also banned.

But Google does not allow content that makes “demonstrably false” claims that could “significantly undermine participation or trust in an electoral or democratic process.” Nor does it allow sites that promote “harmful health claims” and lists anti-vaccine advocacy, denial of conditions such as AIDS or COVID and gay conversion therapy as examples.

All of that is very good and should rightly be applauded.

But, while removing more than 140,000 sites for violating its policies might sound impressive in isolation, there are more than 58 million websites using Google AdSense. B&T, for example, has around 54,000 different stories currently live on our site.

If we suppose that other sites have around the same — though we’re nowhere near as prolific as some other sites — this means that there could be more than three trillion one hundred thirty-two billion pages online using Google AdSense.

And that’s just Google. While it is the leader in the online ad space — to such an extent that the US Government is eyeing an antitrust lawsuit with the firm to separate the various parts of its business into different entities — it is just one of the firms involved within the digital ad world.

Plus, with the advent of generative AI, the scale of the problem is set to grow by orders of magnitude. At the start of the month, online misinformation-identifying company NewsGuard identified 49 news sites that appeared to be almost entirely written by AI in seven different languages.

The websites, which often fail to disclose ownership or control, produce a high volume of content related to a variety of topics, including politics, health, entertainment, finance, and technology,” wrote NewsGuard’s McKenzie Sadeghi and Lorenzo Arvanitis, “Some publish hundreds of articles a day. Some of the content advances false narratives.

“Many of the sites are saturated with advertisements, indicating that they were likely designed to generate revenue from programmatic ads.”

One of these sites, for example, claimed that US President Joe Biden had died and Vice President Kamala Harris had taken the helm. Another of these websites managed to garner 124,000 followers on its Facebook Page.

Unpicking The Adtech Lock

It’s often said that no one wants to see how the sausage is made. But, with programmatic advertising, much as with sausages, it’s in the making of the sausage where you will find what separates the best from the worst.

A person deeply involved in the fight against programmatic ad-funded misinformation told B&T that the industry created, propagated and continues to benefit from a “black hole.”

Creatives and ad spend go in, the creatives get shuffled around through demand-side platforms (DSPs), then onto supply-side platforms (SSPs), and get spat back out onto a website. Money flows backwards in the opposite direction.

In fact, they said that the reason that misinformation continues to exist and spread online is not, as some adtech players told us, because it is intrinsic to the internet and human nature but it is because the DSPs and SSPs continue to benefit from the Wizard of Oz-esque cloak that they hide their tech behind.

At this point, we would love to bring in some insight from the well-known ad platforms but none of them wanted to talk to B&T for this article.

From the outside looking in, it seems as though much of the problem stems from a simple lack of transparency.

Most big ad exchanges publish what is known as a “sellers.json” file. In theory, these should show who an ad exchange is selling adverts to. Ads.txt, meanwhile, is its publisher counterpart, allowing you to see who placed which ad on a particular website. Ads.txt files are easy to find, simply choose a website and put “/ads.txt” on the end. Here’s the Daily Mail’s, for instance. You won’t find one for B&T because we do not run programmatic ads.

Google was a big advocate for ads.txt and helped to co-develop it as well as promote its adoption. However, with sellers.json, Google was more reluctant. You can find it here but, in all the time that B&T has been researching for this piece, the page has crashed every single time. In fact, PubMatic released its sellers.json file a year before Google did and it works. We asked Google whether it had an up-to-date version of its sellers.json file and it was not able to provide one.

Perhaps more concerning is that, while Google knows where to send the money from its ad operations, it will not tell advertisers who is getting that money.

According to research from Jounce Media back in 2020 (which we would love to update but, as we mentioned, Google’s sellers.json doesn’t want to play ball) 15.42 per cent of the websites and apps that Google authorises as a seller has no matching ID. This means that advertisers cannot see what domains Google is working with. What’s more, Jounce said that, unlike its rivals, these accounts are active and it sees its clients bidding into these supply paths despite the obvious opacity.

But, there’s more. When Jounce Media published, more than half of the accounts in Google’s sellers.json file were confidential, meaning that Google has entirely redacted the information about these sellers. This makes it near impossible for advertisers to know where their ads – and their money – is going. By contrast, two per cent of Xandr accounts were confidential and, for all other ad exchanges with sellers.json files, this was less than one per cent.

Google is not alone though. In 2020, the Incorporated Society of British Advertisers (ISBA), working alongside PwC, unearthed some staggering statistics about the programmatic supply chain.

Just 12 per cent of the 1.3 billion impressions it analysed over 18 months were able to be matched from a DSP to an SSP. Plus, 15 per cent of all programmatic ad spend — representing around a third of supply chain costs — was unattributable. In fact, such was the lack of transparency, ISBA could not say what this missing 15 per cent even represented.

“It could reflect a combination of: limitations in data sets, necessitating occasional estimations; DSP or SSP fees that aren’t visible in the study data; post-auction bid shading; post-auction financing arrangements or other trading deals; foreign exchange translations; inventory reselling between tech vendors; or other unknown factors,” read the trade association’s report.

In January, ISBA offered an updated version of the findings that showed a big improvement. Of 1.3 billion impressions analysed, 58 per cent were able to be matched end-to-end and unattributable spend had dropped to three per cent. But, in 2020, it spent 18 months analysing the supply chain. In 2022, it spent half the time and analysed the same number of impressions.

Then there was the small matter of the recent Google TrueView scandal. A report by Adalytics alleged that “for years significant quantities of TrueView skippable in-stream ads, purchased by many different brands and media agencies” were not viewed or were hosted in content that simply played ads with no substance.

As a result, ads were missold to the likes of the US federal government, the European Parliament, HP, Johnson&Johnson, Bayer, American Express, Samsung and Mercedes-Benz. None were any the wiser about their ads being placed in phoney environments — with the sites simply lifting money from their ad budgets. All the major agencies were affected to one degree or another. Google rejected the findings and said that third parties verified the ads and that Adalytics’ methodology was faulty.

Lawyers are apparently investigating whether a class action lawsuit can be filed against Google.

As the programmatic ad market speeds and becomes increasingly becomes the default method of buying for any digital assets, the percentages will become smaller but the overall size of the programmatic pie will get even larger, and so will the losses.

Suppose you were a media buyer shopping for some newspaper ad spots. But, rather than calling up the publication’s sales team and asking them how much a quarter-page ad would be and haggling on price, you had to ring another company.

That other company then asked you what your ad would be about to see if it is suitable, then asked you how much you are prepared to pay, then it puts you into an auction that completes in seconds and then it puts your ad on a website that you might never have heard of before and sends you back a report on how many people – again, unbeknown to you – looked at it or clicked on it.

This black hole of online advertising has led to the rise of other companies who work to verify the placement of and engagement with digital adverts. We will talk about them next time.