Disable Search Engine Indexing
Search engines use web crawler (spider/robot) which browses WWW in an automated manner to provide up-to-date data. But what if one doesn’t want to get listed? Yes, “robots.txt” comes into play when you have to set your own trend whether to allow bots or prevent your site from being crawled!
Place robots.txt in your root directory [...]