For getting index fast in various kind of search engine it’s very important to allow them in your website. Today we are going to learn how to allow them with just few lines of code.Lets get started-
Open a robot.txt file and write followint code. This code allow only google, yahoo, msn bots crawling of your web contents. You need to put this file inside public_html folder(If you are using linxu server).
User-agent: * Disallow: / User-agent: Googlebot Allow: / User-agent: Yahoo-slurp Allow: / User-agent: Msnbot Allow: /
Was this information useful? What other tips would you like to read about in the future? Share your comments, feedback and experiences with us by commenting below!