New research from the DoubleVerify’s Fraud Lab highlights an 86 per cent year-over-year increase in general invalid traffic (GIVT) rates during the second half of 2024 – with the fourth quarter of 2024 becoming the first time in history that monthly GIVT volumes exceeded 2 billion ad requests.
The rapid rise in GIVT is linked to the proliferation of AI-powered crawlers and scrapers designed to collect and analyse web content. Whilst GIVT is not inherently malicious like sophisticated invalid traffic (SIVT), and industry standards and regulations offer some protection to advertisers against wasted ad spend from GIVT, the results can still have a harmful impact on accurate measurement.
This means advertisers must now measure and avoid both GIVT and SIVT to improve campaign performance and maximise budget allocation.
The accelerated adoption of generative AI tools in recent years has led to a notable rise in AI-driven crawlers and scrapers. In fact, DV data showed that a record 16 percent of GIVT from known-bot impressions in 2024 were generated by those that are associated with AI scrapers, such as GPTBot, ClaudeBot and AppleBot.
The following chart shows the relative portion of major crawlers and scrapers that drove GIVT in December 2024, measured on known-bot impressions:
Key Findings from DoubleVerify’s Research:
- AI crawlers and scrapers have emerged as a leading contributor to GIVT, inflating traffic metrics across digital platforms.
- If left unchecked, GIVT can distort campaign metrics, inflate impression counts and raise concerns about discrepancies.
- DV’s advanced AI-powered detection tools have been enhanced to mitigate this growing threat, ensuring advertiser investments are safeguarded.
