We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
MODX Community Forums Archive

Answered Disallowing some MODX folders in robots.txt - good idea or not?

    • 38783
    • 571 Posts
    Andy Tough Reply #1, 9 years, 2 months ago
    I wonder if anyone has any thoughts on this?

    It is sensible to list folders I don't want indexed by search engine bots in robots.txt using 

    User-agent: *
Disallow: /foldername/


    I was recently advised that I should include folders like /manager in this list.

    Or if I have secured a MODX installation, to include the new manager folder name. ie:

    User-agent: *
Disallow: /secretManagerfoderName/


    My view is that if a bot finds my manager folder then the .htaccess file inside it will make the contents inaccessible so it won't get indexed anyway.

    But if I list the manager folder it in the robots file I provide hackers with an easy way of finding out what CMS I am using.

    Or if I have changed the name of the manager folder I have told them what the 'secret' manager folder name is.

    Any thoughts?

    This question has been answered by lkfranklin. See the first response.

      If I help you out on these forums I would be very grateful if you would consider rating me on Trustpilot: https://uk.trustpilot.com/review/andytough.com

      email: [email protected] | website: https://andytough.com
    • discuss.answer
      • 17301
      • 932 Posts
      LK Reply #2, 9 years, 2 months ago
      Personally I wouldn't bother blocking non accessible folders from the robots file, like you say you're just giving hackers an easy way of finding it out. If you go through the steps of hardening MODX which it sounds like you have done already then I really don't see what it could be indexed for anyhow. You could go really over the top and change the login buttons to say something else, but I wouldn't worry about that too much smiley

      For everything else though the SEO Tab extra is really good and easy way of stating if a resource should be indexed or not.
        ■ email: [email protected] | ■ website: https://alienbuild.uk

        The greatest compliment you can give back to us, is to spend a few seconds leaving a rating at our trustpilot: https://uk.trustpilot.com/review/alienbuild.uk about the service we provided. We always drop mention of services offered by businesses we've worked with in the past to those of interest.
        • 38783
        • 571 Posts
        Andy Tough Reply #3, 9 years, 2 months ago
        Thanks Liam
          If I help you out on these forums I would be very grateful if you would consider rating me on Trustpilot: https://uk.trustpilot.com/review/andytough.com

          email: [email protected] | website: https://andytough.com