Navigating AI Crawlers: The Atlantic’s Strategic Approach
The Atlantic has developed an innovative scorecard for AI crawlers, determining which bots bring value versus those that don’t. This proactive strategy helps block ineffective crawlers while allowing those driving traffic and subscribers to thrive.
Key Insights:
- Selective Blocking: The Atlantic only allows AI crawlers that generate referral traffic or new subscriptions.
- Collaborative Partnership: A licensing deal with OpenAI enables them to maintain access for AI that adds value.
- Monitoring Trends: Weekly assessments track AI bot activity, focusing on traffic generation and subscriber conversion.
- Challenges: Different AI platforms, such as Google and Amazon, lead to complex decisions on access without diminishing search traffic.
- Future Leverage: The Atlantic aims to enhance negotiation power through transparent communication in its robots.txt file.
The question remains: Will AI platforms evolve to provide meaningful traffic? Engage with us and let’s explore the impact of AI on the publishing landscape. Share your thoughts!
