JavaScript SEO: why ChatGPT may not see your site

Reading time: 14 minutes

JavaScript SEO means making sure that content built by JavaScript reaches Google and the AI crawlers that index and cite pages. Google runs the JavaScript before indexing, but the AI crawlers that Vercel and MERJ tested in 2024 read only the initial HTML. That is why this guide covers the process, the tests and the fixes.

Does Google render JavaScript pages?

Yes. The Understand JavaScript SEO Basics guide from Google describes three phases: crawling, rendering and indexing. Every page with a 200 status goes into a rendering queue, unless a robots meta tag or header asks not to index it. Then a headless Chromium runs the code, and Google indexes the HTML that comes out of that run.

Googlebot reads robots.txt, downloads the page, collects the links from href attributes and puts the page in the queue, where it waits seconds or longer. As soon as rendering finishes, the page goes through another round of link collection and into the index.

On the measured site, the queue was generally short. Vercel and MERJ timed more than 37,000 Googlebot renders on nextjs.org in April 2024. The median time between crawling and rendering was 10 seconds, but the 99th percentile reached about 18 hours. Vercel sells hosting for Next.js sites, and MERJ is an SEO consultancy.

Even so, Google says in its JavaScript SEO guide that server-side rendering or pre-rendering is still a good idea. After all, not all crawlers execute JavaScript.

What trips up JavaScript SEO on Google?

Even with rendering, some problems leave content out of the rendered HTML or out of the index. The most common range from blocked or heavy resources to script errors, wrong HTTP status codes and routes with #.

• Resources blocked in robots.txt. According to its JavaScript SEO guide, Google doesn’t render JavaScript from blocked files, so the page may end up incomplete.
• Files that are too large. According to a Google post from March 2026, Googlebot fetches up to 2 MB of each URL. In addition, each script and each CSS file has its own limit, and Google ignores anything beyond it.
• Slow resources. Google doesn’t publish a fixed rendering deadline. However, the Rich Results Test help flags a resource that didn’t load “in a reasonable amount of time.”
• Errors and unsupported features. A script error can break the page, so Google’s JavaScript troubleshooting guide recommends logging errors. The renderer also clears cookies and local storage between pages and doesn’t use WebSockets.
• Content that depends on interaction. Google doesn’t interact with the page, according to its page on lazy loading. So it may miss whatever loads only after a user clicks or scrolls.
• Status other than 200 and noindex. Google may not render error pages. Likewise, with noindex in the initial HTML, Google may skip rendering, so removing the tag by script may not work.
• Soft 404 in SPAs. When a single-page application handles errors in the browser, the nonexistent route usually responds 200 and may end up indexed.
• Fragments with #. In its JavaScript SEO guide, Google advises against routes like /#/products, because Googlebot can’t reliably resolve those URLs.

Do AI crawlers execute JavaScript?

There is no official answer, because the pages about OpenAI’s, Anthropic’s and Perplexity’s crawlers don’t mention JavaScript. In addition, the most cited public measurement, by Vercel with MERJ in December 2024, saw none of the AI crawlers it tested execute JavaScript. On the other hand, a September 2026 report suggests a change in OpenAI’s crawlers, still unconfirmed.

We checked OpenAI’s documentation, Anthropic’s page and Perplexity’s documentation. They describe what each crawler does, but they don’t cover rendering.

In the study published on December 17, 2024, Vercel and MERJ used data from the preceding months on nextjs.org, the Vercel network and two job sites. None of the major AI crawlers tested rendered JavaScript. The ChatGPT and Claude crawlers downloaded scripts, in 11.50% and 23.84% of requests, but didn’t execute them. Note that Vercel, which sells hosting for Next.js, has a commercial interest in the topic.

The study adds two caveats. First, crawlers can still read data that already comes in the HTML, such as embedded JSON. Second, Gemini uses Googlebot’s infrastructure, so it renders JavaScript. Likewise, AI Overviews and AI Mode cite pages from the Search index, which Google builds from the rendered HTML.

“Brands must ensure that critical information is server-side rendered.”

Ryan Siddle, Managing Director of MERJ (2024)

Crawler	Executes JavaScript?	Basis
Googlebot	Yes, in a rendering queue	Google documentation
Gemini (via Googlebot)	Yes, according to the study	Vercel and MERJ, 2024
OAI-SearchBot and GPTBot	Not in 2024; report of a change in 2026	Vercel and MERJ; unconfirmed report
ChatGPT-User	No, in the test	Vercel and MERJ, 2024
ClaudeBot	No, in the test	Vercel and MERJ, 2024
PerplexityBot	No, in the test	Vercel and MERJ, 2024
Claude-SearchBot, Claude-User and Perplexity-User	Not tested in 2024	List in the Vercel study

The September 2026 report

On September 30, the owner of a Japanese marketplace built on Next.js published an analysis of his own logs on DEV Community, with the IPs checked against OpenAI’s lists. According to him, since September 25, OAI-SearchBot and GPTBot have been triggering link prefetches on his sites that only happen when the page renders. However, he found no announcement from OpenAI and thinks the change applies only to some sites.

In addition, the Internet Archive shows no change in the crawler text on OpenAI’s page since the September 17 copy. So treat the report as a signal to watch in your logs. For JavaScript SEO, the practical rule still holds: whatever crawlers need to read must be in the initial HTML. The technical foundation is the first step of SEO for the AI era.

The role of each crawler is in the crawler table in the guide to what GEO is. For ChatGPT answers that use search, for example, the crawler that counts is OAI-SearchBot, as the guide on how to show up in ChatGPT shows.

What content disappears from pages without JavaScript (text, links, prices)?

Everything the browser builds after loading the page disappears. AI crawlers that read only the initial HTML miss text fetched from an API and links without href. Prices, reviews, meta tags and structured data inserted by scripts also disappear.

Content	How it is usually built	What the crawler loses
Main text	API call after loading	The whole page
Internal links	Button or onclick without href	The linked pages
Price and stock	Script that queries the catalog	The offer data
Reviews	Third-party widget	Rating and comments
Title, description and canonical	Set by script	Summary and preferred URL
Structured data	JSON-LD injected by script or by GTM	Product, company, author
Images	Swapping data-src for src	The image address
Long lists	Infinite scroll	Items beyond the first screen

According to Google’s documentation on links, Google generally only crawls <a> elements with an href attribute. As a result, a menu made of buttons can hide internal pages even from Googlebot.

For online stores, Google also warns that Product markup generated with JavaScript can make Shopping crawls less frequent and less reliable.

How do you test JavaScript SEO on your site?

Compare three versions of the page: the HTML the server delivers to each crawler, the DOM after JavaScript and what Google indexed. The checklist below has five tests, each with the symptom, how to check and the fix.

1. curl with each crawler’s user agent

Symptom: the page shows up on Google, but ChatGPT and Perplexity don’t cite it.

How to check: download the HTML with each crawler’s official user agent, then look for a sentence from the text and for the links. Anthropic publishes only the crawler name, but it works for the test. Googlebot serves as a comparison: if only Googlebot receives the text, the server changes its response based on the user agent.

URL="https://www.example.com/product/blue-running-shoe"
PHRASE="Blue running shoe"

# In the Googlebot user agent, W.X.Y.Z is how Google marks the Chrome version
for UA in \
  "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" \
  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)" \
  "ClaudeBot" "Claude-SearchBot" "Claude-User"
do
  BOT=$(echo "$UA" | grep -oE "Googlebot|OAI-SearchBot|GPTBot|ChatGPT-User|PerplexityBot|Perplexity-User|Claude[A-Za-z-]*" | head -1)
  STATUS=$(curl -sL -A "$UA" -o page.html -w "%{http_code}" "$URL")
  echo "$BOT | status $STATUS | phrase: $(grep -oF "$PHRASE" page.html | wc -l) | links: $(grep -oi '<a [^>]*href=' page.html | wc -l)"
done

The test doesn’t reproduce the crawler’s IP, so a firewall that checks both user agent and IP, as Perplexity recommends, may respond differently. The guide on how to show up in Gemini, Perplexity and Claude also covers the Perplexity and Claude AI crawlers.

Fix: if the phrase or the links don’t show up, the content depends on JavaScript and needs to move into the server HTML. A 403 or 503 status calls for a review of the firewall and the CDN before the code.

2. Page source versus rendered DOM

Symptom: the browser shows the text, but the page source has little beyond a <div id="root"> and scripts.

How to check: compare the page source (Ctrl+U) with the Elements panel in the developer tools. On the command line, headless Chrome with the --dump-dom option prints the DOM after running the JavaScript.

URL="https://www.example.com/product/blue-running-shoe"
PHRASE="Blue running shoe"

curl -sL "$URL" > source.html
# On Windows and macOS, replace google-chrome with the path to your installed Chrome
google-chrome --headless --disable-gpu --dump-dom "$URL" > rendered.html

for f in source.html rendered.html; do
  echo "$f | phrase: $(grep -oF "$PHRASE" "$f" | wc -l) | links: $(grep -oi '<a [^>]*href=' "$f" | wc -l)"
done

Fix: whatever only appears in the rendered file is invisible to crawlers that don’t execute JavaScript. Therefore, move text, price, links and meta tags into the server HTML first.

3. URL Inspection in Search Console

Symptom: the page doesn’t show up on Google, or it shows up with the wrong title, description or canonical.

How to check: in the URL Inspection tool, run the live test and then open “View tested page.” There you’ll find the screenshot, the page code, the loaded resources and the JavaScript console output.

Fix: unblock resources blocked in robots.txt, fix the console errors and move to the server whatever is missing from the screenshot. Finally, test again and request indexing.

4. Rich Results Test

Symptom: the structured data doesn’t produce rich results, or the page is on a domain without Search Console access.

How to check: paste the URL into the Rich Results Test. By default, it loads the page as a smartphone, flags resources that didn’t load and shows the rendered code.

Fix: if the JSON-LD shows up in the test but not in the curl HTML, only crawlers that execute JavaScript see it. Then generate the block on the server.

5. Server logs

Symptom: you don’t know whether AI crawlers visit the site, with which status, or whether they render anything.

How to check: filter the log by crawler names, count the status codes and see whether the crawler requests anything beyond the HTML.

# Apache and Nginx combined format: status in field 9, path in field 7

# HTTP status by crawler
for R in OAI-SearchBot GPTBot ChatGPT-User ClaudeBot Claude-SearchBot Claude-User PerplexityBot Perplexity-User; do
  echo "== $R"
  grep "$R" access.log | awk '{print $9}' | sort | uniq -c | sort -rn
done

# What OAI-SearchBot requests beyond the HTML
grep "OAI-SearchBot" access.log | awk '{print $7}' \
  | grep -E '\.(css|js|png|jpe?g|webp)|/api/|_rsc=' \
  | sort | uniq -c | sort -rn | head -20

A request for a .js file doesn’t prove execution, because in the Vercel study the AI crawlers downloaded JavaScript files without executing them. Stronger signals are the API calls that the script makes and, in Next.js, URLs with _rsc=. On Next.js sites, also see what the framework serves each crawler in the guide to SEO in Next.js. Since anyone can spoof a user agent, check the IP against each company’s lists.

For Google, use the Crawl Stats report in Search Console. This is because Google’s JavaScript troubleshooting guide warns that browser-based analytics may not show this activity in full.

Fix: a 403 or 429 for a crawler you want to allow points to a firewall or a rate limit. Many 404s call for redirects and an updated sitemap. In the Vercel study, for example, 34.82% of ChatGPT’s requests and 34.16% of Claude’s hit 404 pages, compared with 8.22% for Googlebot.

How do you fix JavaScript SEO without rebuilding the site?

In general, the fixes fit into your current code, without switching frameworks. There is only one rule: whatever crawlers need to read, cite or follow must be in the HTML the server delivers. That applies to everything from links and meta tags to images, pagination and HTTP status codes.

Links with href. Keep the router, but render an <a> with the real URL. A click event can stay on the link.

<!-- Avoid: Google may not extract the URL -->
<span onclick="goTo('/shoes')">Shoes</span>
<a onclick="goTo('/shoes')">Shoes</a>

<!-- Crawlable -->
<a href="/shoes">Shoes</a>

One URL per route, with the History API. Replace /#/products with /products and use pushState in the router. The server should respond to each route with its own HTML.

Meta tags in the server HTML. Title, description, canonical and robots should come in the server’s response. In its JavaScript SEO guide, Google accepts a title and description set by script, but asks scripts to keep the original canonical.

<head>
  <title>Blue running shoe | Example Store</title>
  <meta name="description" content="Lightweight shoe for 5 to 21 km runs.">
  <link rel="canonical" href="https://www.example.com/product/blue-running-shoe">
</head>

Structured data in the initial HTML. Google reads JSON-LD injected by JavaScript when it renders, but a crawler that reads only the HTML doesn’t. So generate the block on the server.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Product",
  "name": "Blue running shoe",
  "offers": {
    "@type": "Offer",
    "price": "399.90",
    "priceCurrency": "BRL",
    "availability": "https://schema.org/InStock"
  }
}
</script>

Lazy loading with a fallback in the HTML. The browser’s native lazy loading keeps the src in the initial HTML: without JavaScript, the image is still there. Libraries that swap data-src for src, however, leave crawlers that don’t execute JavaScript without the image.

<!-- The crawler sees the image address -->
<img src="/img/blue-running-shoe.webp" alt="Blue running shoe" loading="lazy" width="800" height="600">

<!-- Without JavaScript, there is no image -->
<img data-src="/img/blue-running-shoe.webp" class="lazyload" alt="Blue running shoe">

Pagination instead of infinite scroll. Give each block of the list a fixed URL, like ?page=2, with sequential links, and update the URL with the History API as the user scrolls. As a result, the crawler finds every item.

<!-- On the /shoes?page=2 page -->
<nav aria-label="Pagination">
  <a href="/shoes?page=1">Previous</a>
  <a href="/shoes?page=3">Next</a>
</nav>

Correct HTTP status codes. A nonexistent route should respond 404 on the server. If the SPA doesn’t allow it, Google accepts a script redirect to a URL that returns 404 or a noindex on the error page. For crawlers that don’t execute JavaScript, however, only the server status counts.

// Product route in a SPA (adapted from Google's example)
fetch(`/api/products/${id}`)
  .then((response) => response.json())
  .then((product) => {
    if (!product.exists) {
      window.location.href = '/not-found'; // the server responds 404 at this URL
    }
  });

When are JavaScript SEO fixes not enough?

If all of the main content depends on the browser, these fixes aren’t enough. Then the solution is to change the rendering mode, with server-side rendering, static generation or pre-rendering. To choose between these modes, see the comparison between rendering on the server and in the browser. The same applies to sites built on Lovable and other tools that build pages in the browser. On React sites, the guide to React SEO shows the SSR, pre-rendering and Next.js options.

Frequently asked questions

Main sources

• Google Search Central, Understand JavaScript SEO Basics (2026).
• Vercel and MERJ, study on how Google renders JavaScript (2024).
• Vercel and MERJ, The rise of the AI crawler, on AI crawlers and JavaScript (2024).
• OpenAI, Overview of OpenAI Crawlers (2026).
• Anthropic, Anthropic’s crawlers and how to control them (2026).
• Perplexity, Perplexity Crawlers (2026).