AI Bots Dominate Internet Traffic, Strain Infrastructure – Fastly Report
Table of Contents
Primary Topic: The increasing impact of AI-driven bots on internet traffic and infrastructure.
Primary Keyword: AI bots
Secondary Keywords: AI crawlers, bot management, web scraping, DDOS mitigation, bot verification, infrastructure costs, Fastly Threat Insights Report, automated traffic, AI training data.
The internet is undergoing a meaningful shift in usage patterns, largely driven by automated programs – or bots – designed for Artificial Intelligence (AI) purposes. A recent report from edge cloud platform Fastly highlights that AI-driven bots now constitute a massive portion of overall internet traffic, creating new challenges for digital platforms and security teams. This analysis, based on data from the second quarter of 2025, reveals a landscape where automated activity is not necessarily malicious, but still poses ample risks.
The Rise of AI Crawlers and Fetcher Bots
According to Fastly’s Threat Insights Report, approximately 80% of all AI bot traffic is attributable to AI crawlers. Thes crawlers are automated programs that systematically search the internet to gather data used for training AI models. The report identifies Meta as the source of over half of this crawler traffic, with Google and OpenAI following behind.
beyond crawlers, “fetcher bots” – used by AI-powered services like Perplexity AI and chatbots – also contribute significantly to real-time traffic. Fastly measured peak times exceeding 39,000 inquiries per minute from these bots. This surge in automated requests can overwhelm unprotected servers and networks, mimicking the effects of Distributed denial-of-Service (DDoS) attacks, even without any malicious intent.
Understanding the Impact on Infrastructure
The sheer volume of traffic generated by these AI bots places a considerable strain on internet infrastructure. While not inherently malicious, the constant scraping and data retrieval can lead to:
Increased Infrastructure Costs: Handling the massive influx of bot traffic requires significant server capacity and bandwidth, driving up operational expenses for website owners and service providers.
Performance Degradation: Unprotected servers can experiance slowdowns or outages when bombarded with bot requests, impacting the user experience for legitimate visitors.
Challenges in Attribution: Identifying and differentiating between benign AI bots and malicious actors becomes increasingly arduous, hindering effective security measures.
Geographical Distribution of AI Crawler Traffic
The report also reveals a notable geographical skew in AI crawler activity. north America accounts for nearly 90% of all AI crawler traffic, while Europe, Asia, and Latin America represent a significantly smaller proportion. Fastly suggests this disparity could indicate a geographical bias in the data used to train AI models, possibly leading to skewed or incomplete results. This highlights the importance of diverse and representative datasets for building fair and accurate AI systems.
The Need for Clarity and Improved bot Management
Arun Kumar, Senior Security Research at Fastly, emphasizes that AI bots are fundamentally altering how the internet is used. He notes the increasing complexities they introduce for digital platforms, notably regarding visibility, control, and cost.
Fastly’s report calls for several key improvements in bot management:
Enhanced Bot Verification: Implementing more robust methods to verify the legitimacy of bots accessing online resources.
Clear Labeling by Operators: Requiring bot operators to clearly identify their bots, allowing website owners to differentiate between legitimate and potentially problematic traffic.
Targeted Bot Management Strategies: Developing more complex strategies to manage bot traffic, including rate limiting, CAPTCHAs, and other security measures.
Without these measures, the risk of hidden automation, attribution gaps, and escalating infrastructure costs will continue to grow. Effective bot management is no longer simply a security concern; it’s a critical component of maintaining a stable and efficient internet ecosystem in the age of AI.