What is robots.txt?
Robots.txt is a text file. It is through this file. It instructs search engine crawlers about indexing and caching of a webpage, file of a website or directory, domain.
If you like this question & answer and want to contribute, then write your question & answer and email to freewebmentor[@]gmail.com. Your question and answer will appear on FreeWebMentor.com and help other developers.
