This is the noindex I miss most often, because it lives in the HTTP header and never appears in the HTML. The page looks perfectly indexable in view-source while quietly being told to disappear. A noindex delivered via HTTP header (not visible in HTML).
Why does x-robots-tag noindex in http header matter for AI search?
Easily overlooked because it’s not in the markup, yet fully effective at deindexing, a common hidden cause of lost visibility.
How do you fix it?
Inspect response headers; remove the header at server/CDN level.
How do you find it on your site?
- Run curl -I https://yourdomain.com/your-page and read every X-Robots-Tag line.
- Look for noindex or none in the header value.
- Confirm in Search Console URL Inspection, which reports header directives.
- Trace the header to your server, CDN or framework config and remove it.
Which ranking and citation factors does it touch?
A header noindex is as absolute as an HTML one. Because it is invisible in the markup, it can suppress a page for months before anyone checks the headers.
What is the impact?
Severe/blocking. Direct; especially insidious because invisible in source.
What is the evidence?
X-Robots-Tag applies the same directives via HTTP header. Google Search Central, Robots Meta Tags Specifications; Google Search Central, Block Search indexing with noindex
Sources
Frequently asked questions about x-robots-tag noindex in http header
What does "X-Robots-Tag noindex in HTTP header" mean?
A noindex delivered via HTTP header (not visible in HTML).
How do you fix x-Robots-Tag noindex in HTTP header?
Inspect response headers; remove the header at server/CDN level.