Reddit Blocks Search Engines from Accessing Recent Posts Without Payment
If you’re used to searching for Reddit posts on search engines like Bing or DuckDuckGo, you might notice a change: recent Reddit content won’t show up unless you use Google. This shift comes after Reddit decided to block major search engines and their AI web crawlers from accessing its posts unless they pay for the privilege.
The New Search Landscape
Tim Rathschmidt, a spokesperson for Reddit, “We have been in discussions with multiple search engines. We have been unable to reach agreements with all of them, since some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including their use for AI,” Rathschmidt said.
As a result, only Google has secured a deal to display recent Reddit posts, while other search engines like Bing will still show older content but not the latest updates.
User Impact
For many users, Reddit is a go-to source for personal and human answers to various questions, whether it’s finding tips for a video game, skincare recommendations, or general advice. Typically, adding ‘site
.com’ or ‘Reddit’ at the end of a search query would bring up the best results. Now, doing this on search engines other than Google will likely yield outdated Reddit posts, leaving users frustrated.
Google’s $60 Million Deal with Reddit
Earlier this year, Google struck a $60 million deal with Reddit to allow its AI bots to access Reddit’s content. This deal followed a Reddit blackout last year, a protest against API changes, which significantly impacted Google’s access to various subreddits. However, Rathschmidt clarified that the recent move to block search engines is “not at all related to our recent partnership with Google.”
Reddit’s Firm Stance on Data Scraping
Reddit’s updated robots.txt file now prevents web crawlers from accessing its data without permission, emphasizing that Reddit data isn’t free for the taking. This move is a part of Reddit’s broader strategy to control its content and ensure it isn’t used by AI models without proper compensation.
Microsoft, for its part, has acknowledged Reddit’s stance. Caitlin Roulston, a spokesperson for Microsoft, said, “Microsoft respects the robots.txt standard, and we honor the directions provided by websites that do not want content on their pages to be used with our generative AI models.”
Conclusion
Reddit’s bold moves aim to set clear boundaries about the use of its data, especially concerning AI. For now, if you’re searching for recent Reddit advice, you’ll need to use Google. Other search engines might eventually strike deals with Reddit, but until then, they’ll only provide older Reddit content.
