Robots.txt Tester | Google Webmaster Tools Guide

In this video we will take a look at robots.txt Tester in Google Webmaster Tools.

What is robots.txt Tester?

Robots.txt is a text file that you can use to communicate with web robots how they should behave on your site.

For example:

  • You can use it to grant access to certain robots and not others.
  • You can grant certain sections of your site.
  • You can tell where your Sitemaps are located so these robots can find them easily.

Google Webmaster Tools’ robots.txt tester can help us see if the text file is created correctly.

If you changed your robots.txt recently, you can also notify Google using this tool to ensure Google is aware of the latest version.


Watch other Google Search Console guide videos

Back to Google Webmaster Tools Guide Home Page

Watch This Video - Easy To Rank Keyword Database

Easy To Rank Keywords for Niche Sites

DRM robot Learn More >>

Leave a Reply