Where are my robots?

I've relaunched my site. I torched the old production site, setup a new one and launched yesterday. But, for some reason I have a robots.txt blocking search engines and I can't find any way of changing this.

this url returns the robots.txt file which the engines find http://www.touchretail.co.uk/robots.txt

There is no reference to this within manager/files and if I ssh to the www folder, it's not there. If I upload one, it's ignored and still returns the disallow one.

I've looked in modxcloud admin and can't see any way to change it here, but I can only assume it's this that is serving up the file. Any ideas anyone?

thanks in advance
Stu

Look at the .htaccess file in the MODX root and see if there's a rewrite rule that specifically mentions robots. If there is and you didn't put it there, it's possible that your site has been hacked.

Quote from: BobRay at Apr 06, 2014, 09:23 PM

Look at the .htaccess file in the MODX root and see if there's a rewrite rule that specifically mentions robots. If there is and you didn't put it there, it's possible that your site has been hacked.

Thanks for the reply Bob,

Nope, nothing in there that mentions robots. I only did this on Friday night, it's on the latest modx version, so very much doubt it's been hacked.

I took a snapshot of the site when it was in a dev, I then created a production cloud using the dev snapshot. I'm wondering if it brought something across with it as dev's automatically block robots I believe.

Checkout the attachment, this tab mentions robots but nothing is listed for me to change any settings - could that be related?

I didn't notice you were on MODX Cloud. I think that setting is probably the issue.

Thanks Bob. I've raised a ticket with modxcloud support, see if they can resolve. Cheers

Hi there, I have the same problem. Any solution to this issue yet? Can't find my robots.txt in modxcloud.

Definitely something wrong there. Here's what I have on my production account

This means, your site is crawled or not?

Quote from: sottwell at Jun 15, 2014, 01:38 AM

Definitely something wrong there. Here's what I have on my production account

Where can I find this setting? My site is hosted in the MODXcloud by my agency, so I can only access the manager for the site not the Cloud

That's in the Cloud dashboard. If you don't have access to the Cloud, then there's not much you can do about it. Supposedly a robots.txt file put in your site root will override whatever the Cloud settings are. You could create one and use the File browser to upload it.

Thanks for your help. Now it works.