Course contentsShow
← Web Development
Lesson 1875 of 2,98234. Metadata and SEOFree lesson

Linking Sitemap in Robots.txt

Reference your XML sitemap in robots.txt and submit it to search engine webmaster tools.

Linking Sitemap in Robots.txt

What you'll learn: How to inform search engines where your sitemap lives by declaring it in your robots.txt file.

The Core Idea

Think of robots.txt as the front desk receptionist and your sitemap as a building directory. While search engines can discover your sitemap through other means, explicitly telling them where it is in your robots.txt file ensures they find it immediately when they first visit your site.

By adding a Sitemap directive to your robots.txt file, you're giving crawlers a direct pointer to your XML sitemap, making it easier for them to understand your site's complete structure right from their first interaction.

How to Reference Your Sitemap

Add this line to your robots.txt file (typically at the top or bottom):

User-agent: *
Disallow: /admin/

Sitemap: https://www.example.com/sitemap.xml

Key points:

  • Use the absolute URL (full path including https://)
  • The directive is not case-sensitive but conventionally capitalized
  • You can list multiple sitemaps if needed:
Sitemap: https://www.example.com/sitemap.xml
Sitemap: https://www.example.com/sitemap-images.xml
Sitemap: https://www.example.com/sitemap-index.xml
  • The Sitemap directive is independent of User-agent rules—it applies globally

Beyond Robots.txt

While adding your sitemap to robots.txt is helpful, also submit it directly through:

  • Google Search Console
  • Bing Webmaster Tools
  • Other search engine webmaster platforms

This double approach ensures maximum visibility and gives you monitoring capabilities.

Key Takeaway: Always reference your XML sitemap location in robots.txt using absolute URLs, and also submit it directly to search engine webmaster tools for optimal discoverability and monitoring.