← Glossary
SEO Glossary
Robots meta tag
Definition
Robots meta tags are HTML directives placed in the
section of web pages that provide specific instructions to search engine crawlers about how to handle individual pages.Explained
Robots meta tags are HTML directives placed in the <head> section of a web page that tell search engine crawlers how to treat that specific page. Unlike robots.txt, which works at the site level, robots meta tags provide granular, page-level control over indexing, link following, and snippet display.
The standard syntax is <meta name="robots" content="directive1, directive2">. Common directives include:
- noindex — prevents the page from appearing in search results
- nofollow — tells crawlers not to follow links on the page
- nosnippet — blocks text snippets in search results
- noarchive — prevents cached copies of the page
- max-snippet — limits the length of displayed snippets
When used strategically, robots meta tags help optimize crawl budget, prevent duplicate content and cannibalization, protect competitive and privacy-sensitive content, and align a site with AI-driven and privacy-first search trends.
Example
E-commerce brands use "noindex, follow" on filtered product pages to prevent duplicate content issues while still passing link equity to important product pages.