We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 36416
    • 589 Posts
    Quote from: woundedbeer at Jan 10, 2009, 12:35 AM

    Then, with the last mod, I only write the contentType array for the docs that aren’t text/html (only 1 line for me).

    This also works in 0.9.6.3 with a single WHERE addition in CacheSync code, good work!
    75k of code gone from cache, not bad... wink
      • 4749
      • 623 Posts
      Yeah, I tried that snippet that Jason wrote and I couldn’t get it to work. I tried it out on eform and the form disappeared. I couldn’t get it to work. I even wrote the whole file address in to be sure.
        The MODx has you...
        Utah Web Design
        • 4749
        • 623 Posts
        How did you get Wayfinder, Ditto, and Eform to work with that include snippet? Every time I try it doesn’t work. Is there a special work around for snippets that pull information from the database?

        I’m really hoping to make this happen. My sitecache file would go from 14k lines down to 5k if I could figure out how to get the snippets and plugins to work as includes. Thanks.
          The MODx has you...
          Utah Web Design
          • 4310
          • 2,310 Posts
          I can’t seem to get the [!Include!] snippet to work with eForm either.
            • 7253
            • 97 Posts
            Seems like adding the WHERE clause into the CacheSync code is a plan with no drawbacks. It’s a simple change, makes the cache file smaller per document, giving more benefit on larger sites where it’s most needed. By the sound of it - and I could be wrong - it doesn’t have any negative side effects.

            Would there be any support for merging this bit into the MODx core?
              • 22303 MODX Staff
              • 10,725 Posts
              Quote from: dflock at Jan 13, 2009, 04:03 AM

              Would there be any support for merging this bit into the MODx core?
              Yes, of course. Is there a Jira ticket associated with this BTW? If not, I’ll get this committed for the Evolution 1.0 release as soon as someone gets the ticket entered. wink
                • 33372
                • 1,611 Posts
                Just two quick comments on this discussion:

                First, I’ve seen the site cache issue mentioned elsewhere as the "5000 document limitation" or some similar description. As you folks are obviously aware (but others may not be), there’s no fixed limit, and the actual number of documents at which you might start to have trouble varies dramatically (it could be much more than 5000, but it could be less). If you have a fast server with a lot of CPU to spare, then even parsing a very large cache file on every page load may not overload it (depending upon how many hits you’re getting), whereas if your web server is already strained or limited even a small cache file may be too much.

                Second, if you save your snippets as include files, make sure that your code is not vulnerable to XSS attacks. In most cases right now this will not be a major threat (since hackers will not know the names and location of your custom files), but if this starts to become a widely-used technique then proper attention will need to be paid to these files. Having register_globals set to OFF will protect you from XSS attacks, but it might also be a good idea to use an .htaccess rule to prohibit direct access to your include files (for those of us using Apache servers) or otherwise treat these files as a little more toxic than the rest of the MODx installation (which has been tested and vetted many times).

                Given that AjaxSearch and Ditto can both deal with content in custom tables easily now, offloading bulk content (e.g., directory listings) into other tables may be the easiest way to reduce the cache file size without modifying the core at all.
                  "Things are not what they appear to be; nor are they otherwise." - Buddha

                  "Well, gee, Buddha - that wasn't very helpful..." - ZAP

                  Useful MODx links: documentation | wiki | forum guidelines | bugs & requests | info you should include with your post | commercial support options
                  • 7253
                  • 97 Posts
                  Ok, I’ve added this as an issue to JIRA:

                  http://svn.modxcms.com/jira/browse/MODX-592
                    • 22303 MODX Staff
                    • 10,725 Posts
                    Quote from: ZAP at Jan 13, 2009, 09:55 AM

                    First, I’ve seen the site cache issue mentioned elsewhere as the "5000 document limitation" or some similar description. As you folks are obviously aware (but others may not be), there’s no fixed limit, and the actual number of documents at which you might start to have trouble varies dramatically (it could be much more than 5000, but it could be less). If you have a fast server with a lot of CPU to spare, then even parsing a very large cache file on every page load may not overload it (depending upon how many hits you’re getting), whereas if your web server is already strained or limited even a small cache file may be too much.
                    Amen ZAP!

                    Quote from: ZAP at Jan 13, 2009, 09:55 AM

                    Second, if you save your snippets as include files, make sure that your code is not vulnerable to XSS attacks. In most cases right now this will not be a major threat (since hackers will not know the names and location of your custom files), but if this starts to become a widely-used technique then proper attention will need to be paid to these files. Having register_globals set to OFF will protect you from XSS attacks, but it might also be a good idea to use an .htaccess rule to prohibit direct access to your include files (for those of us using Apache servers) or otherwise treat these files as a little more toxic than the rest of the MODx installation (which has been tested and vetted many times).
                    Having register_globals set to OFF may prevent a majority of common XSS attacks, it certainly doesn’t protect you from those that can occur from improperly scrubbed user input. If you have a location that is not accessible by the web server, but that PHP can read from, you might put your snippet code in this location, so it can only be included from local scripts and never executed directly via URL. Or simply leave off the php extension (or use an alternative one that will not reveal the source code, i.e. don’t make it .txt), again so it can’t be executed via URL directly as a PHP script.

                    Quote from: ZAP at Jan 13, 2009, 09:55 AM

                    Given that AjaxSearch and Ditto can both deal with content in custom tables easily now, offloading bulk content (e.g., directory listings) into other tables may be the easiest way to reduce the cache file size without modifying the core at all.
                    For serious, scalable web applications, this kind of organization and separation is critical to long-term success IMO. I’m a firm believer that Documents AND TV’s are simply parts of the View (i.e. the presentation) and should never represent part of the actual Model (i.e. the data).

                    Quote from: dflock at Jan 13, 2009, 10:05 AM

                    Ok, I’ve added this as an issue to JIRA:

                    http://svn.modxcms.com/jira/browse/MODX-592
                    Thanks kindly dflock; your efforts are much appreciated.
                      • 33372
                      • 1,611 Posts
                      Quote from: OpenGeek at Jan 13, 2009, 10:13 AM

                      Having register_globals set to OFF may prevent a majority of common XSS attacks, it certainly doesn’t protect you from those that can occur from improperly scrubbed user input.
                      Very true, although this would mostly likely be the case whether the snippet were an include file or not.

                      Quote from: OpenGeek at Jan 13, 2009, 10:13 AM
                      If you have a location that is not accessible by the web server, but that PHP can read from, you might put your snippet code in this location, so it can only be included from local scripts and never executed directly via URL.
                      This could also work with an .htaccess rule (say in /assets/snippets/includes), which would help to standardize the location but wouldn’t work for those who don’t use Apache.

                      Quote from: OpenGeek at Jan 13, 2009, 10:13 AM
                      Or simply leave off the php extension (or use an alternative one that will not reveal the source code, i.e. don’t make it .txt), again so it can’t be executed via URL directly as a PHP script.
                      That may be the best idea yet imho. It’s relatively foolproof in terms of security and would allow for standardization. I think that’s what I’ll start to do from now on.
                        "Things are not what they appear to be; nor are they otherwise." - Buddha

                        "Well, gee, Buddha - that wasn't very helpful..." - ZAP

                        Useful MODx links: documentation | wiki | forum guidelines | bugs & requests | info you should include with your post | commercial support options