Cloudflare has introduced a new feature called “Markdown for Agents,” which improves efficiency by converting HTML web pages to Markdown on-the-fly, reducing token usage by up to 80%. While large language models (LLMs) can read HTML, it produces unnecessary tokens due to added formatting and scripts. For instance, a typical HTML page might generate 16,180 tokens, whereas Markdown only requires 3,150 tokens, greatly lowering inference costs.
When AI agents request pages, Cloudflare facilitates real-time HTML-to-Markdown conversion at the edge, enhancing access for many tools like Claude Code. The feature allows for better machine-readability, exposing token counts via an x-markdown-tokens header. Site owners can activate this from the Cloudflare dashboard without altering their CMS setups. This program marks a significant move towards optimizing web content for AI, underscoring Markdown’s emerging role as a standard format for AI agents.
Source link
