We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 30319
    • 406 Posts
    Using contexts, if a new context other than 'web' points to a sub-directory, should one use a robots.txt file to disallow robots/crawling that context's site??

    Or should the robots directives be in the meta tags??

    Thank you, Tom
    • I don't recommend configuring Contexts into subdirectories, since the folder could then conflict with requests to the main Context, especially if something had the same name as the folder. This also complicates the rewrite rules necessary for FURLs. That said, putting the robots.txt into the subfolder for the other Context should work assuming the requests are being made directly to the subfolder.
        • 30319
        • 406 Posts
        The contexts are into subdirectories because the subdirectories must contain site-specific files.
        If this is not a recommended method, some ways to deal with this are:

        1) fix contexts so it's easier for each context to have its own folder for site-specific content unrelated to the 'web' context (such as site specific images, files, css, javascripts) -- or document the best way to do this without making things more complicated in the MODx documentation

        2) document contexts best practices in the documentation??

        It's impractical to run separate MODx installs for each site, so contexts are a great idea.

        Nevertheless each context's website has its unique requirements for file storage, website names, etc., and somehow proper information should be disseminated to facilitate meeting these requirements appropriately within MODx's design constraints.

        I am still figuring out how to do what we need and the available documentation -- 3 different websites and techniques -- is confusing.

        On top of that none of it that I've found so far applies well to localhost development for testing and development purposes.

        On the confusion aspect, I think it's up to the developer and site managers/editors to prevent your good suggestion about naming conflicts.

        Thank you for reading this, I will try both approaches -- robots.txt and meta tags for one context's website, which must be "private," with a login for frontend access and the other context is "public" anyway.

        Thank you, Tom