Can AI crawlers read JavaScript?

AI crawlers cannot read JavaScript, in the sense that matters: they fetch the HTML your server returns and they do not run the scripts inside it. Vercel and MERJ's analysis of AI crawler behaviour across the Vercel network, published in December 2024, found no evidence of client-side JavaScript execution by any of the major AI crawlers, including OpenAI's GPTBot and OAI-SearchBot, Anthropic's ClaudeBot, PerplexityBot and ByteDance's Bytespider.

Several of them download the JavaScript anyway. The same study reported that 11.50% of ChatGPT crawler fetches and 23.84% of Claude fetches were JavaScript files, which the crawlers retrieved and did not execute. What those crawlers extract is whatever text was present in the initial markup, before a framework had a chance to build the page.

The volume involved is not marginal. Vercel recorded 569 million requests from GPTBot and 370 million from Claude across its network in a single month, against 4.5 billion from Googlebot in the same period, so those two crawlers on their own fetched about a fifth as much as Google did. Adding AppleBot and PerplexityBot takes the AI crawlers to nearly 1.3 billion fetches, which is the 28% of Googlebot's volume Search Engine Journal led with when it covered the study. Traffic at that scale is reading a strictly poorer version of every page it touches.

The practical consequence is a split between two audiences that used to be one. Googlebot renders JavaScript, so a client-side application can rank perfectly well in classic search while returning almost nothing to the systems that write AI answers. A page can hold position one and be, to ChatGPT, an empty shell with a navigation bar.

Which crawlers render JavaScript and which do not?

Crawlers that render JavaScript are the exception rather than the rule. Google is the significant one: Googlebot has run a full rendering pipeline for years, and Google's AI surfaces, AI Overviews and AI Mode, are fed by that same infrastructure, which is why Gemini answers can include content that only exists after hydration. Apple's crawler has also been described as browser-based.

Crawler	Feeds	Executes JavaScript	What it sees on a client-rendered page
Googlebot	Google Search, AI Overviews, AI Mode	Yes	The rendered page, after a queue delay
GPTBot / OAI-SearchBot	ChatGPT and its search index	No, on the Vercel and MERJ data	Initial HTML only
ClaudeBot	Anthropic	No, on the Vercel and MERJ data	Initial HTML only
PerplexityBot	Perplexity's own index	No, on the Vercel and MERJ data	Initial HTML only
Bingbot	Bing and Microsoft Copilot	Partially, on Microsoft's own documentation	Varies by page and budget

Treat the table as the current state rather than a permanent one. Crawler behaviour is not published by the engines themselves, so every line in it comes from server-log analysis by people watching their own traffic, and any of these operators could add rendering next quarter without announcing it. What has not changed since the first measurements is the direction: the dedicated AI crawlers are built for volume and cheap fetches, and running a browser for every URL is neither.

One more line in the same dataset is worth reading. The Vercel study found that 34.82% of ChatGPT crawler fetches and 34.16% of Claude fetches returned 404, against 8.22% for Googlebot, which says these crawlers are working from stale and second-hand URL lists rather than from a carefully maintained index. A crawler that spends a third of its budget on dead URLs is not one to hand an extra rendering step to.

What does a client-side rendered page look like to ChatGPT?

A client-side rendered page looks to ChatGPT like whatever your server sent before any script ran, which for a typical single page app is a title, some meta tags, a navigation shell and an empty div. The headline, the product description, the specification table and the FAQ all arrive later, in the browser, and the crawler has already moved on.

The failure is silent, which is what makes it expensive. Nothing in your analytics reports it. The crawler fetched a page and got a 200 response, your server logs look healthy, and your Google rankings are unaffected because Googlebot renders. The only visible symptom is an absence: your competitors get named in AI answers about your category and you do not.

Partial versions of the problem are more common than the total one. Sites built on a modern framework often render the marketing pages on the server and leave the parts that matter most to commercial questions, pricing tables, product filters, review widgets, specification tabs and anything behind an accordion loaded on interaction, in client-side code. The page is half visible, and the half that is missing is the half a buyer asked about.

Two mitigations that feel similar are not. Content inside a collapsed accordion is usually present in the HTML and merely hidden with CSS, which crawlers can still read. Content fetched by JavaScript when the accordion opens does not exist until a browser runs the script, and no AI crawler will. The distinction is invisible on screen and decisive in the fetch.

How do you check whether your own site has this problem?

You check your own site by fetching a page the way a crawler does and reading what comes back. Open a terminal and request the URL with curl, then search the response for a sentence you know appears on the page. If the sentence is missing from the raw HTML, no AI crawler can quote it, whatever the page looks like in your browser.

The browser version of the same test takes about a minute. Disable JavaScript in your browser settings, reload the page, and look at what remains. Anything that vanishes is content an AI engine cannot use. A second pass through view-source rather than the inspector matters, because the inspector shows the rendered DOM and hides the exact difference you are testing for.

Then check the parts a marketing page test misses. Run the same fetch against a product or service page, a pricing page and one page that answers a specific buyer question, since those are the pages that get retrieved for commercial prompts. Check that the headings and the answer paragraphs are present, rather than only the navigation and the footer.

Finally, look at server logs for the crawler user agents themselves: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Googlebot. Whether they arrive at all, which URLs they request, and what status codes they get is the ground truth that no third-party tool can give you. Our longer walkthrough on how to check if AI can read your website covers the robots.txt and access side of the same audit.

How do you fix a JavaScript site without rebuilding it?

Fixing a JavaScript site usually means changing where the HTML is assembled, not which framework you use. Server-side rendering and static generation both send a complete document to the crawler, and every major framework supports one or the other without a rewrite. For a marketing site, prerendering the routes that answer buyer questions is often a configuration change and a deploy.

If the application genuinely has to run on the client, the cheaper fix is to stop putting the answer inside it. A documentation page, a specification page or a question-and-answer page served as plain server-rendered HTML gives the engines something to cite, and the interactive product can stay as it is. Engines cite pages, not applications.

Avoid the two shortcuts that look like fixes. Serving a different, simpler page to crawlers than to humans is cloaking, and the risk is not worth the saving when server rendering is available. Pushing content into JSON-LD while leaving the visible page empty does not work either, because structured data annotates content that exists rather than substituting for it, a point our page on whether schema markup helps AI citations goes through with the evidence.

Order the work by what gates everything else. A page the crawler cannot read is not a ranking problem or a content problem, it is an eligibility problem, and no amount of vocabulary work or citation building moves a page that arrives empty. Get the answer into the initial HTML first, then worry about whether it is the right answer, which is the sequence our AI citation checklist follows.

Does this mean a single page app can never get cited?

A single page app can get cited, but only for content that exists before its JavaScript runs. Plenty of applications ship a server-rendered marketing layer and a client-side product, and the marketing layer is what answer engines read. The citation goes to the URL that returned the text, so the question is never whether your stack is modern but whether the answer is in the response.

Google is the exception that causes most of the confusion. Because Googlebot renders and feeds AI Overviews and AI Mode, a client-rendered page can appear in Google's AI surfaces and be absent everywhere else. Teams read that as proof the site is fine for AI, when what it proves is that one engine out of several has a rendering pipeline the others do not.

Remember also that rendering is only the first gate. A page that renders on the server still has to be reachable, extractable and worded the way buyers ask, and the Discovered Labs 2026 analysis of more than 2 million citations found vocabulary alignment between page and query to be the only page-level signal that survived controls for domain authority, at an effect size of 0.37. Our methodology page sets out the weights and where each one comes from.

The useful way to hold all this is that JavaScript rendering decides whether you are eligible, and the rest of the work decides whether you win. Eligibility is binary and cheap to test, which is why it belongs at the front of any audit, ahead of the more interesting arguments about content.

What else do people ask about AI crawlers and JavaScript?