Skip to main content
Request access

CRAWLING, RENDERING & INDEXING

Noindex: what it means and how it works

A meta tag or HTTP header that asks search engines to keep a page out of results.

Check it on your site
All glossary terms
  • Beginner level
  • 2 primary sources
  • Reviewed 29 October 2026
At a glanceSEO glossary
  • TopicCrawling, rendering & indexingAlso in Technical SEO, On-page SEO & content
  • LevelBeginnerNo background needed
  • Sources2 primary sourcesGoogle Search Central
  • Reviewed29 October 2026Re-checked every 12 months
On this page
  1. Quick definition
  2. Why it matters
  3. How it works
  4. How to check it
  5. Common mistakes
  6. Related concepts
  7. Check it on your site
  8. Sources and review

Quick definition

Noindex is a rule, set in a robots meta tag or an X-Robots-Tag HTTP header, that tells search engines not to show a page in their results. The search engine has to crawl the page to see the rule, so a noindexed page must not also be blocked in robots.txt.

Why it matters

Noindex is the reliable way to keep a crawlable page out of search: thank-you pages, internal search results, staging copies, thin archives. Used by mistake, for example left over from a staging site, it removes important pages from search.

How it works

<meta name="robots" content="noindex">
In the page’s <head>: applies to all crawlers that support it.
X-Robots-Tag: noindex
As a response header: works for PDFs, images and other non-HTML files.

You can target one crawler with its name, such as <meta name="googlebot" content="noindex">. When Google next crawls the page and sees the rule, it drops the page from results. Google may crawl long-term noindexed pages less often.

GoalUse
Keep a page out of resultsnoindex (page must be crawlable)
Stop crawlers fetching URLsrobots.txt Disallow
Consolidate duplicates into one URLrel=canonical or a redirect
Keep content privateAuthentication, not noindex

How to check it

  1. Check both the HTML <head> and the HTTP response headers; an X-Robots-Tag set by a server or CDN is easy to miss.
  2. Look at the rendered HTML too: JavaScript can add or change a robots meta tag after load.
  3. In Search Console, the Page indexing report lists URLs "Excluded by ‘noindex’ tag". Check that none of them are pages you want found.
  4. After launching a site, confirm the staging "discourage search engines" setting was switched off.

Common mistakes and misconceptions

  • Myth Robots.txt and noindex do the same thing.

    In fact Robots.txt controls crawling; noindex controls indexing. Blocking a noindexed page in robots.txt hides the noindex from the crawler.

  • Myth Noindex and canonical together send a stronger signal.

    In fact They conflict: one says drop this page, the other says this page duplicates another. Pick the one that matches your intent.

Check it on your site

Free tools, no sign-up

In the NdovuSEOLabs platform

Run the same checks on every page of your site, with your own data.

Sources and review

Written and maintained by Ndovu SEO Labs. First published 29 October 2026. Facts last checked against the sources above on 29 October 2026. How we write and review the glossary.