What Is an XML Sitemap and Why Google Can't Read Yours

By Ugur Saritepe · July 9, 2026

An XML sitemap is a machine-readable list of the pages you want Google to crawl. It is also easy to get silently wrong: yours can be missing, can secretly be an HTML page, or can list pages a crawler can't actually read. One curlcommand tells you which case you're in.

The 30-second self-diagnosis

Ask your site for its sitemap and look at what comes back:

curl -s https://yourdomain.com/sitemap.xml | head -3

The first lines of output sort you into one of three cases:

Starts with <?xml and a <urlset> or <sitemapindex> tag. You have a real sitemap. Your remaining risks are the quieter ones: nothing referencing it, listed pages that render client-side, or a list full of thin pages. Causes 3 to 5 below.
Starts with <!DOCTYPE html> or <html>. Your “sitemap” is an HTML page in disguise. Google's parser can't read it, so you effectively have no sitemap. Cause 2 below.
Nothing, or an error page. Run curl -sI https://yourdomain.com/sitemap.xml | head -1 and if the status is 404, there is no sitemap at the standard location. Cause 1 below.

What an XML sitemap actually is

An XML sitemap is a plain text file, written in a fixed XML format, that lists the URLs on your site you want search engines to know about. A minimal valid one looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yourdomain.com/pricing</loc>
    <lastmod>2026-07-01</lastmod>
  </url>
</urlset>

Crawlers discover pages by following links from pages they already know. A sitemap short-circuits that: it hands Google the full list up front, which matters most for new pages, deep pages with few internal links, and new sites that nothing links to yet. It is a hint, not a command. Google's sitemap documentation (as of July 2026) says directly that a sitemap “doesn't guarantee that all the items in your sitemap will be crawled and indexed.”

Do you actually need one?

Google's honest answer: not always. Its docs say a site of roughly 500 pages or fewer, where every page is reachable through internal links, might not need a sitemap at all. The cases where a sitemap earns its keep are large sites, new sites with few external links pointing at them, and sites with lots of media or news content.

In practice the question rarely matters, because generating one costs nothing. Most CMS platforms and frameworks do it automatically; in Next.js it is a single app/sitemap.tsfile. Our own traffictriage.com sitemap is generated that way and lists 12 URLs. Tiny, but the site is new with few external links, which is exactly the profile Google says benefits. The real question is not “do I need a sitemap” but “can Google read the one I have” and that is what the rest of this article checks.

Five ways your sitemap can be invisible

1. There is no sitemap anywhere Google looks

Crawlers find sitemaps in three places: the conventional /sitemap.xml path, Sitemap: lines in your robots.txt, and direct submission in Google Search Console. Some platforms use other paths by convention (WordPress SEO plugins often serve /sitemap_index.xml), which is fine as long as something references the real location.

30-second check: the diagnosis curl returned 404, and curl -s https://yourdomain.com/robots.txt | grep -i sitemap prints nothing. Google can still find your pages through links, so a missing sitemap alone won't deindex you. But on a new site with few backlinks it means discovery runs at its slowest possible speed.

2. Your “sitemap” is an HTML page

The sneakiest version, common on single-page apps. The same catch-all rewrite that causes soft 404s answers every path with your app shell, including /sitemap.xml when no real file exists there. The URL returns 200, so everything looks fine, but the body is HTML. Google's parser needs a <urlset> or <sitemapindex>document; feed it HTML and Search Console reports the sitemap as “Couldn't fetch” or “could not be read.”

30-second check: curl -s https://yourdomain.com/sitemap.xml | head -1. Good output starts with <?xml. Bad output starts with <!DOCTYPE html>. The status code cannot tell you this; only the body can.

3. Nothing points to it

A sitemap at the standard /sitemap.xml path can be found by convention. One at any other path is invisible unless you advertise it: a Sitemap: line in robots.txt (with the full absolute URL) or a submission in Search Console. The robots.txt line is the one every crawler reads, not just Google, so add it even if you also submit in Search Console:

Sitemap: https://yourdomain.com/sitemap.xml

30-second check: curl -s https://yourdomain.com/robots.txt | grep -i sitemap. Good output is one line per sitemap with an absolute URL. Nothing is a fixable gap, not an emergency: our scan scores it Monitor, not Critical.

4. The trap: listed pages render client-side

This is the failure the other checks can't see. A URL can sit in a perfectly valid sitemap, return 200, and still be invisible, because the HTML the server sends contains an empty <div id="root"> and some script tags, and the actual content only appears after JavaScript runs in a browser. The sitemap got the crawler to the door; the page answered with a blank room.

Google can render JavaScript, but rendering is a separate, queued step, and if it fails or times out the empty shell is what gets evaluated. Most other crawlers, including the ones behind AI search products, don't execute JavaScript at all. For them a client-side rendered page has no content, full stop.

30-second check: strip the tags from the raw HTML of a page in your sitemap and count what a crawler actually gets:

curl -s https://yourdomain.com/pricing | sed 's/<[^>]*>//g' | wc -w

Good output is roughly the word count of the visible page. Bad output is a few dozen words on a page that clearly shows more in a browser. The fix is server-side rendering or prerendering, covered in our indexing triage for JavaScript sites.

5. It faithfully lists thousands of thin pages

A sitemap amplifies whatever your site says. If your site generates a page per keyword, per city, or per database row, the sitemap hands Google the complete list of them. In scans we ran in early July 2026, one site's sitemap listed 21,656 URLs; another listed 2,479 of which more than 2,000 carried under 200 words of content each. None of that is an XML error, but it tells Google your site is mostly near-empty pages, and Google rates sites partly on the quality of what they ask to have indexed.

30-second check: curl -s https://yourdomain.com/sitemap.xml | grep -c "<loc>" counts the URLs. If the number is far beyond the pages you could name, open a few of the listed URLs and judge them as a stranger: would this page deserve a Google result on its own?

How to fix your sitemap

Generate a real XML sitemap.Use your framework's built-in route (Next.js: app/sitemap.ts) or your CMS plugin rather than a hand-maintained file that drifts out of date.
List only canonical, absolute, live URLs. Google's sitemap guidelines call for fully-qualified absolute URLs, one preferred version per page. No redirects, no 404s, no noindex pages, no http:// leftovers.
Reference it in robots.txt with a Sitemap: line carrying the absolute URL, so every crawler can find it.
Submit it in Google Search Consoleunder Indexing → Sitemaps. This also gets you the fetch status and the discovered-URL count for free.
Keep it honest. Prune or noindex thin programmatic pages instead of listing them, and only update lastmodwhen the content meaningfully changes. Google's docs single out cosmetic updates like a copyright-date change as not worth a new lastmod.

How to confirm the fix worked

Re-run the diagnosis curl: the body must start with <?xml and contain a <urlset>, and the robots.txt grep must print your Sitemap:line. Then check Search Console's Sitemaps report: after Google refetches (on its own schedule, typically within days), the status should read Success with a discovered-URL count matching what you published. That count is the proof Google parsed the file; the curl output is your instant proof it can.

How TrafficTriage runs this check

The sitemap is check #2 of the eight in our free Triage Report, and it automates exactly the diagnosis above. We look for a sitemap at /sitemap.xml, at every Sitemap: line in your robots.txt, and at /sitemap_index.xml. Nothing found scores Critical. Found but not a <urlset>/<sitemapindex> document scores Critical too. Then we sample up to three listed URLs and fetch each one the way a crawler does, without running JavaScript: a page that answers 200 but carries almost no words in its raw HTML gets flagged as client-side rendered, a missing robots.txt reference scores Monitor, and a sitemap past 300 URLs gets a thin-pages flag to review by hand.

If your report says “No sitemap found” or “Sitemap is not valid XML,” that is an afternoon of work with an outsized payoff: it restores the entire discovery channel at once. If it says some sitemap URLs render client-side, prioritize that instead. A sitemap pointing at pages crawlers can't read is a map to empty rooms.

From experience: 9 of 29 sites flagged

Between July 4 and July 9, 2026 we ran this check on 29 sites: 20 scored Healthy, 7 Monitor, and 2 Critical. One Critical simply had no sitemap at any of the three locations. The other was a hacked small-business site whose attackers had injected their own sitemaps listing 1,146 spam pages, marked “updated hourly” so Google would keep crawling them. A sitemap is a megaphone: it amplifies whatever your site says, including things you didn't say.

The most common flag was size: 5 of the 7 Monitor results were sitemaps past the 300-URL threshold, ranging from 322 to 21,656 URLs, mostly programmatically generated pages. The remaining two were a sitemap nothing referenced and a site whose bot protection blocked our crawler from fetching the listed pages at all, which means it likely blocks other crawlers too. Explanations of the other checks from the same report live in Checks Explained.

FAQ

Do I need an XML sitemap for a small website?

Probably yes, even though Google says you can skip it. Google's own guidance is that a site of roughly 500 pages or fewer, with every page reachable through internal links, might not need one. But the sites that fit that description are usually also new sites with few external links, and that is exactly the case where a sitemap helps Google discover pages it would otherwise miss. Most frameworks and CMS platforms generate one automatically, so the practical answer is: keep it, and make sure it works.

Why does Google Search Console say it couldn't fetch my sitemap?

The three usual causes are: the URL is wrong and returns 404, the URL returns an HTML page instead of XML (common when a single-page app serves its shell for every path), or something in front of your site, like a firewall or bot protection, blocks Google's crawler. Open the sitemap URL in your browser and view the source. If you see HTML instead of a <urlset> document, that is your cause.

Does submitting a sitemap guarantee Google will index my pages?

No. Google's documentation is explicit that a sitemap does not guarantee the listed URLs get crawled or indexed. A sitemap is a discovery hint: it tells Google which URLs exist and which you consider canonical. Each page still has to be reachable, readable without JavaScript, and worth indexing on its own.

How many URLs can one XML sitemap contain?

50,000 URLs or 50MB uncompressed, whichever comes first. Past either limit you split the list into multiple sitemap files and reference them all from one sitemap index file (a <sitemapindex> document), then submit only the index.

Not sure whether Google can read your sitemap?

Get a free Triage Report: eight checks, including this one, run from your URL alone. No Search Console access needed, each check scored Critical / Monitor / Healthy, ending with the one thing to fix this week. It renders on your screen in about half a minute, with no signup and no email required.

Request your free Triage Report →