We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 19873
    • 67 Posts
    I just ran a link integrity check (XENU) on my site and I see that the /manager lgin page get’s crawled. Why is that? I am not linking to it anywhere...

      • 33968
      • 863 Posts
      Is that Googlebot or some other robot? They don’t always play by the rules, but you can advise robots which pages should not be crawled by adding the robots.txt file to your site root (see Google Webmaster Guidelines).

      By default it should look something like this for modx:
      # Default modx exclusions
      User-agent: *
      Disallow: /assets/cache/
      Disallow: /assets/docs/
      Disallow: /assets/export/
      Disallow: /assets/import/
      Disallow: /assets/modules/
      Disallow: /assets/plugins/
      Disallow: /assets/snippets/
      Disallow: /install/
      Disallow: /manager/


      Also, check to see that
      <meta name="robots" content="noindex, nofollow" />

      appears in your page header...