Free checker › Guides
Why AI crawlers cannot read JavaScript-only pages
Updated 2 November 2026
A page can look complete in your browser and be nearly empty to an AI crawler. The difference is JavaScript.
What the research found
Vercel analysed AI crawler traffic and reported in The rise of the AI crawler that the major AI crawlers it measured, including GPTBot, ClaudeBot and PerplexityBot, do not render JavaScript. They may download JavaScript files, but they do not execute them, so client-side rendered content is invisible to them. Google's Gemini and Apple's crawler were the exceptions it noted.
What to do about it
- Server-render the content that matters. Main text, headings, product details, prices, metadata and navigation should be in the first HTML response.
- Use client-side rendering for extras. View counters, chat widgets and social feeds can stay client-side.
- Keep URLs clean. Vercel noted that AI crawlers hit many 404 errors, so keep redirects tidy, sitemaps current and URL patterns consistent.
A one-minute test
- Open your page and choose View page source (not Inspect). This shows the first HTML response.
- Search for a distinctive sentence from the middle of the page.
- If the sentence is there, crawlers that do not run JavaScript can read it. If it is missing, they cannot.
The free checker automates this: it compares the visible text in the initial HTML with what a normal page should contain and warns when the page looks thin. You can also paste a sentence from your page to test for it directly.
Common causes
Single-page apps without server rendering, content loaded by tabs or accordions after a click, and pages built entirely by a page-builder script are the usual causes. Most frameworks offer server-side rendering or static generation, and WordPress themes normally render on the server already.