GWMT icon

Robots.txt Tester | Google Webmaster Tools Guide

In this video we will take a look at robots.txt Tester in Google Webmaster Tools.

What is robots.txt Tester?

Robots.txt is a text file that you can use to communicate with web robots how they should behave on your site.

For example:

  • You can use it to grant access to certain robots and not others.
  • You can grant certain sections of your site.
  • You can tell where your Sitemaps are located so these robots can find them easily.

Google Webmaster Tools’ robots.txt tester can help us see if the text file is created correctly.

If you changed your robots.txt recently, you can also notify Google using this tool to ensure Google is aware of the latest version.

 

Watch other Google Search Console guide videos

Back to Google Webmaster Tools Guide Home Page
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *