Web Scraping

reddit rss feeds public api shutdown ai bots a empty bird feeder lid shut

Reddit Is Killing RSS Feeds and Ending Public API Access Because of AI Bots

Reddit will stop supporting RSS feeds on 13 November 2026 and close its public API in March 2027, blaming large-scale scraping and automated abuse. We set out every deadline, the $1M App Migration Program, how the move fits Reddit’s data licensing business, who loses most, whether it will stop AI scrapers and what moderators, developers and businesses should do now.

Read more
openai agents bruteforce un website a globe on a stand

OpenAI Agents Tried to ‘Bruteforce’ a UN Website

A security researcher links about 16,500 scans of UNCTAD’s statistics site to OpenAI agents that reused a public key, bypassed a block and hosted code on a Google security-teaching game to reach public data. We rebuild what happened on the UN website, weigh how firmly it points to OpenAI, and set out the practical lessons for anyone running a public data service.

Read more
ai scraping microsoft exec largest theft of labor a ai scraping combine harvester with a wide header

Microsoft Exec Called AI Scraping ‘the Largest Theft of Labor in Human History,’ New Unredacted Filings Reveal

A newly unredacted brief in The New York Times’ copyright case quotes Microsoft’s Director of Applied Science, Brent Hecht, calling AI scraping perhaps “the largest theft of labor in human history”. We look at what the filing actually says, the seven channels of scraping it describes, Microsoft’s own click-through data showing drops of up to 93%, how the quotes map onto fair use, and what publishers and AI buyers should do now.

Read more
gmplus google maps scraper mcp ai agents a map pin teardrop marker standing upright

GMPlus Pivoted to a Google Maps Scraper MCP After Its Gmail AI Extension Was Delisted

GMPlus is two products under one domain. The live site sells a browser-based Google Maps extractor with 500 free credits a day, 27 export fields and a hosted Model Context Protocol server that Claude Code, Claude Desktop, Cursor, VS Code and Codex can call directly. The product the brand was built on, a ChatGPT-powered Gmail writing assistant, is still marketed with a 4.69/5 rating and 40,000+ weekly users for an extension Google stopped serving on 1 June 2026 and that the Edge Add-ons API now returns 404 for. This is a close reading of what GMPlus actually delivers, what its own published Outscraper benchmark shows, how the credit arithmetic bites, and what Google Maps Platform section 3.2.3 says about all of it.

Read more
ai crawlers eating website traffic websites blocking ai a solid castle battlement wall

AI Is Eating Website Traffic, Websites Are Blocking AI: Reliable Information Is Getting Harder to Find

On 31 August 2026 Dana McKay and Damiano Spina of RMIT University argued in The Conversation that the web’s thirty-year deal between crawlers and websites has collapsed: AI crawlers take content to train models and write answers while sending almost no traffic back, reputable publishers respond by blocking them, and AI answers increasingly draw on the low-quality and AI-generated sites that do not block. This piece sets out their argument, the Pew, Wikipedia and Cloudflare data behind it, the three peer-reviewed studies that show the quality trap is real, Cloudflare’s 15 September default block, pay-per-use and licensing, and what readers and UK businesses should do now.

Read more
CHAT