Reply With QuoteHi, with the Robots.txt it works normally on, only the agent must also keep to the robots.txt. In general, Google does this, most of the other bots are not.
Senior Member
Website should not be listed in Google Hey people, I have a webserver on it and always store my websites before I move them to the server of each client. Now I will, however, that all my websites, files, etc., etc. on my web space can not be found on Google or other search engines. Until now I have robot.txt file just tried this. However, this does not work. Is that somehow different? Thank you.
Senior Member
Hi, with the Robots.txt it works normally on, only the agent must also keep to the robots.txt. In general, Google does this, most of the other bots are not.
Senior Member
I have now removed some old projects and all projects that I've uploaded the robots.txt after it seems to be working. But I will probably still install an additional htaccess. As long as it keep Google, Yahoo and Bing, it's okay. Thank you
Senior Member
If you are the robots.txt änderst / initial upload, then of course you have to wait until the crawler time pass so this affects the search results. You can also, after you've uploaded the robots.txt to request the deletion from the index.
Posting Permissions
Copyright Techfuels
Bookmarks