The "robots.txt" file allows you to specify pages in your site that search engines should ignore. If you just don't want to see these error/warning messages any more, create an empty text file called robots.txt and upload it to the root folder of the web site. If you do want to cause search engines to skip over certain portions of your site using the robots.txt file, see the following page for more information:
http://www.robotstxt.org/
In particular, please view the information found on this page:
http://www.robotstxt.org/wc/norobots.html
Please note, Web Easy will not create robots.txt files automatically - you'll have to do it manually with a text editor like Notepad, then upload the file via FTP
I was told I should have a robots.txt file added to my site. What is this and how do I include one? (KB005195)
Modified on: Tue, Jun 28, 2016 at 12:15 PM
Did you find it helpful? Yes No
Can you please tell us how we can improve this article?