Why JavaScript Sites Are Invisible to AI Crawlers
A site can score 85/100 on a traditional SEO tool and still be a blank page to ChatGPT. Here is how that happens.
Of every problem this tool finds, this is the one that costs the most and gets noticed the least. It is invisible in a browser, invisible to most SEO checkers, and total in its effect: the crawler arrives, receives nothing, and leaves.
A real example
While benchmarking this engine against a well-known SEO analyser, we audited the same page with both. The other tool returned 85/100 — a healthy score. Our engine returned 62/100 and one sentence explaining why: the served HTML contained zero words inside an empty <div id="root"></div>.
The site was a React single-page application. In a browser it looked complete — headings, paragraphs, a full navigation. But all of it was assembled by JavaScript after the page loaded. The HTML the server actually sent was a shell.
The other tool wasn't broken. It measured title tags, meta description, canonical, schema — all present and correct in the shell. It simply never asked the question that mattered: is there any content in here?
Why this breaks AI specifically
Googlebot has run JavaScript for years. It queues pages for a second rendering pass, and eventually it sees what you see. That is why a client-rendered site can still rank in Google.
AI crawlers largely do not. GPTBot, CCBot, ClaudeBot and PerplexityBot fetch HTML and parse it. There is no rendering queue, no headless browser, no second pass. What your server returns in that first response is the entire universe as far as they are concerned.
So a client-rendered site ends up in a strange position: visible in traditional search, absent from AI answers, with no error anywhere to explain the difference.
How to check in ten seconds
Ignore your browser — it runs JavaScript, so it will always tell you everything is fine. Ask the server directly:
curl -s https://yoursite.com | wc -wThat counts the words in the raw HTML. A content page should return hundreds. If it returns a handful, your content is not in the response.
To see the shape of the problem:
curl -s https://yoursite.com | grep -o '<div id="[^"]*"></div>'An empty mount node — id="root", id="app", id="__next" — with nothing inside it is the signature. The other quick test needs no terminal at all: disable JavaScript in your browser settings and reload. Whatever disappears is what AI crawlers never had.
The fix, in order of effort
Server-side rendering
The framework generates HTML on the server and sends a complete document; JavaScript then makes it interactive. Next.js, Nuxt, SvelteKit and Remix all do this by default — the trap is opting out, usually by marking a component client-only or fetching content in a browser-side effect.
Static generation
Better still where content doesn't change per request. Pages are built once into real HTML files and served as-is. Fastest for users, perfect for crawlers, and cheapest to host.
Prerendering
If rewriting isn't realistic, a prerender service renders your pages in a headless browser and serves the resulting HTML to bots. It's a retrofit rather than a cure, but it works.
What not to do
Do not serve different content to crawlers than to humans on the assumption nobody will check. Serving a text-only version to bots and a JavaScript app to people is cloaking, and it is a policy violation with real consequences. Prerendering is acceptable because the content is the same; the delivery differs. Substituting different content is not.
One caveat
Not every JavaScript-heavy page has this problem, and our check doesn't assume it does. A page with a React bundle and 1,400 words of server-rendered text passes — the framework is present but the HTML is complete. The failure is specific: an empty mount node plus almost no text. That combination means the content exists only after hydration, and only a browser will ever see it.
Run a free audit to see whether your own pages arrive with their content attached.