Do AI crawlers render JavaScript? Why server-rendered HTML matters
GPTBot, ClaudeBot and PerplexityBot read the raw HTML response and don't execute JavaScript. If your content only appears after rendering, answer engines can't retrieve it or cite it.
Most don't. GPTBot, ClaudeBot and PerplexityBot fetch the raw HTML response and move on without running JavaScript. If your content only appears after rendering, it isn't there for them, and an answer engine can only cite what its crawler could read. Googlebot is the exception that misleads people. It queues pages for rendering and eventually executes JavaScript, so a client-rendered page can rank in Google while staying invisible to the systems that generate AI answers.
Test the raw response, not the browser view. Fetch your page without rendering and search the response for a sentence from your main content. If the sentence is missing, no amount of AI-visibility work downstream will help.
Which AI crawlers run JavaScript?
Of the major crawlers, only Googlebot does.
| Crawler | Runs JavaScript | Sees a client-rendered page |
|---|---|---|
| Googlebot | Yes, on a render queue | Eventually |
| GPTBot | No | Empty shell |
| ClaudeBot | No | Empty shell |
| PerplexityBot | No | Empty shell |
A client-rendered page gives a non-rendering crawler a root element, a script tag and no text to retrieve. The page looks fine to you and returns nothing usable to them.
Why does this matter more in 2026?
Because more of your traffic is machines, and more of the machine traffic is answering live questions. On Cloudflare's network, humans made 39% of requests in September 2026. HTML was 68.9% of what AI bots fetched that month, and search and answer-retrieval crawling grew from 2.6% of AI-bot requests in January to 10.6% in September. Those fetches happen while a person waits for an answer, with no render queue to fall back on.
Googlebot has a limit of its own. Since 6 February 2026 it downloads at most 2MB of uncompressed data per URL, and anything past the cutoff is not indexed. Dave Smart estimated that fewer than 0.01% of sites hit it, but a page that inlines a large script bundle ahead of its body text risks losing the text.
How is this blog built for retrieval?
Every post here is stored as structured content in Postgres and rendered on the server into complete HTML. Pages are cached at the edge and revalidated on a timer, so crawlers get the full content on the first request.
- Body text ships in the initial HTML response, with no client-side fetch and no JavaScript dependency.
- Each post has its own canonical URL, title and meta description.
- Article JSON-LD is generated from the same fields, so structured data cannot drift from the visible page.
- Question-style headings break each post into sections that still make sense when retrieved as a single passage.
How do you check whether AI crawlers can read your pages?
Fetch the page the way a non-rendering crawler does and look for your text.
- Fetch the raw HTML without rendering, for example with
curl -A "GPTBot" https://example.com/page, and search the response for a sentence from your main content. - Confirm robots.txt does not block the AI crawlers you want citations from.
- Verify that the title, canonical and structured data sit in the initial response rather than being injected by a script.
- Check that the HTML document stays well under 2MB uncompressed, with the body text near the top.
- Repeat the first check on your highest-value pages after every front-end deploy.
Is your business in the answer?
One report, four AI platforms, the questions your customers actually ask.
