robots.txt for test MODx site?

I’m testing the new modx-site located in the same web-directory as existing old site. I don’t want to place test modx-site in different folder because of image-paths in documents. So, is it enough to use robots.txt (see code below) to prevent indexing test modx-documents by search boots? I’m using RewriteEngine On.

I think that should do the trick. But I could be wrong.

How about:

Disallow: *

I have reread your post and apparently I misunderstood your request. Disallow: * will disallow the entire site. If using robots.txt you would want to Disallow all Modx directories and files individually. So the following is not all inclusive but provides an example:

User-agent: *
Disallow: /index.php
Disallow: /manager/

If you wish to use the header method then the noindex method works as well. You could also use this module: http://modxcms.com/MetaTagsExtra-1.5m-932.html which provides a toggle for this.

I usually include a chunk in the <header> part of all my website templates to control how i wish the search engine’s to treat my content. Maybe it is also a suitable alternative for your website.

When developing i put this in my chunk:

When the site is finished and filled with content i change it to:


I once had the same situation as you have and i kept the old website which started with index.html which seems to get a higher priority then index.php on apache based webservers. Beside changing the meta tag i then additionally deleted this index.html file and made the other old files html redirect to the new website’s root.

I have not read any clear comment from a reliable source on how many/which search engines support this tag but from what I experienced at least google seems to respect it.