We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 14883 ☆ A M B ☆
    • 450 Posts
    Okay, now that I know HTTP headers are not the source of my site's slowness, and have a slightly better understanding of how caching works in general in MODX, I'm hoping I can get some help on speeding up my very slow website.

    I'll begin with some very brief overview of how my site is set up. Please don't judge too harshly.

    My MODX templates basically all look like this:
    [[setTemplatePlaceholders?&outputChunk=`html`]]


    "setTemplatePlaceholders" is a big, fat master snippet that determines which banner, menus & other custom content to swap in, and basically sets a bunch of placeholders - then returns the chunk 'html' (or whichever chunk you specify).

    One of the main things the snippet does is find a "Site Definition Resource" in the current' resource's ancestor hierarchy. This is a resource that uses a particular template, with TVs that basically define a sub-site. Which banner to use, the title of the sub-site, certain layout options, etc. All of the child descendents of such a resource need to display those same "site definition" properties, so the snippet finds the "Site Definition Resource" and sets a bunch of placeholders accordingly.

    Each template also has some custom properties; My 'html' chunk calls other chunks like 'css', 'js', 'divStructure', 'innerContentStructure', etc., which can be overridden by properties in the template.

    For whatever reason (file under, 'it seemed like a good idea at the time'), over the years I've moved away from having direct calls to things like getResources, Wayfinder, etc. in my chunks, instead moving towards calling all those things from within "setTemplatePlaceholders" and setting placeholders in my chunks. (I guess my thought was, the fewer separate snippet calls, the better. I honestly have no idea whether this is a legit theory, but it is the direction I've gone in.)

    So here's the problem. Uncached, this snippet is taking roughly 2.25 to 2.5 seconds to load - even on pages that barely have anything going on content-wise.

    I know, I know. All I can say is, given the environment here, page load performance really hasn't come up as an issue until recently. I always figured I'd eventually take a look at the numbers and address the issues I found. And now is that time.

    So.

    There are several possible issues that I already know I can/should look in to to begin taming this beast and getting my site load time down to acceptable levels. And there are probably more issues I'm not yet aware of, that hopefully some of you can help me out with.


    1. Is my "run all of the snippets from one master snippet & set placeholders" approach a worse approach than putting the individual snippet calls directly into my template chunks?
    2. Is my "store all the details about which sub-site a page belongs to in the TVs of some resource in the parent hierarchy, and retrieve them each time" approach an incredibly inefficient/stupid one? I've thought of maybe writing a plugin that saves some of this stuff locally to a resource when it is saved (at least the id of the "site definition resource"). Would that be better?
    3. Is there a good way to use 'getCache' to cache this snippet, on a per-resource basis, in a custom cache partition so that it doesn't get overwritten every time the site cache gets cleared?

    I definitely think #3 (which I just became aware of today) has potential. But if #1 and/or #2 would significantly speed up the processing of this snippet (or the total load time of a page, if I bust-up this snippet), I'd definitely want to do that as well. I've already been experimenting with flattening-out my "chunk hierarchy"; not having so many granular layers of chunk calls built in to my template logic. This seems to be helping a bit - although I haven't done a significant amount of data collection thus far.

    Any ideas or thoughts would be highly appreciated. I'm happy to share more about what my code is doing if that might help as well.
      • 38290
      • 712 Posts
      One of the main things the snippet does is find a "Site Definition Resource" in the current' resource's ancestor hierarchy. This is a resource that uses a particular template, with TVs that basically define a sub-site.

      When you say subsite I think of contexts. Using multiple contexts may not necessarily makes senses for your data model and site structure, but if they are actually subsites (be they sub domains or sub directories) contexts could be a saving grace. Reason being each context can have context settings which override System Settings. They are very quick to look up and could be used instead of the ancestor resource + TV approach.

      I think #2 is the culprit.
        jpdevries
        • 14883 ☆ A M B ☆
        • 450 Posts
        I've realized something rather important about MODX caching and the way we do business in this shop.

        First, I have a lot of MODX manager users - content editors - probably about 2 dozen or so different people that edit a page at least once per month, and probably at least 6-8 different folks editing at least one page per day on weekdays.

        This means that my site cache is getting completely cleared out several times per day.

        Second, these sites are relatively low-traffic. We have a few pages that get 500 or so hits per day, but the vast majority of our pages get less than 20 hits per day.

        If I understand correctly, this combination of 1) frequent site-wide cache clearing, and 2) relatively low traffic, means it is highly likely that a sizeable percentage of our total page hits (and our hits on any given page) are coming in a completely uncached scenario.

        Basically, the higher your ratio of site traffic to site edits, the more efficient MODX's default caching is. The lower the ratio, the less efficient it is. And I'm working with a very low ratio.

        Would anyone agree/disagree with that premise?
          • 38290
          • 712 Posts
          That is a safe assumption. I have some pointers on caching here:
          http://devries.jp/blog/2013/03/23/cash-in-big-with-modx-cacheing/

          If your server supports Cron you can use opengeek's script to crawl the site and cache all cacheable resources every so often. I run it every 10min on heartroasters.com.
          https://gist.github.com/opengeek/ec3f885cefe17e470aa1

          Of course optimizing your site more to begin with will help reduce uncached loads. Stay away from looking up TVs on other resources as much as possible. TVs of the current resource are always loaded and quick to look up.
            jpdevries
            • 14883 ☆ A M B ☆
            • 450 Posts
            Thanks for the help JP. I was reading your blog post, among other things, this morning before I posted this question.

            I will have to look in to the context approach, to have my "site definition" data available as settings rather than as TVs on a parent resource. Most of what I am calling 'subsites' are all in different sub-folders of the same domain name. They are basically different department sites, each with its own banner & internal navigation structure. They relate to each other through the overall branding, the breadcrumb hierarchy, etc.

            Before I just jump in to re-doing everything in a context-based approach, though, I think I'm going to do a little more research in to A) which aspects of my snippet are causing the most time-suck, and B) using getCache and the RegenCache script to make the cache much more useful than it currently is.
              • 18373 ☆ A M B ☆
              • 3,141 Posts
              I reckon it would help if you would share some of your snippet code wink
                Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

                Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
                • 38290
                • 712 Posts
                Sounds like an applicable use of contexts to me! Although I have been known to be a little more context-happy than Mark wink
                  jpdevries
                  • 14883 ☆ A M B ☆
                  • 450 Posts
                  Alright Mark, here you go. As with most of my code, the following is a proprietary blend of genius and stupidity, packaged with an extra heaping helping of bad organization and non-standard everything:
                  <?php
                  // PAGE-SPECIFIC STUFF
                  
                  $customcontent = (isset($scriptProperties['customcontent'])) ? $scriptProperties['customcontent'] : '';
                  $modx->setPlaceholder('customcontent', $customcontent);
                  
                  //deal with in-page widgets
                  $x = $modx->getObject('modTemplateVar', array('name'=>'widgets'));
                  if ($widgetsList = $x->getValue($modx->resource->get('id'))) {
                    $widgets = explode('||', $widgetsList);
                    $wOut = '';
                    foreach($widgets as $w) {
                      if (trim($w) != '') {
                        if ($ps = $modx->getObject('modPropertySet', $w)) {
                  	    $psName = $ps->get('name');
                  	    $properties = $ps->getProperties();
                  	    $chunkName = $properties['widget.element'];
                  	    if ($chunk = $modx->parser->getElement('modChunk', $chunkName)) {
                  	      $wOut .= $chunk->process($properties);
                  	    }
                        }
                      }
                    }
                    $modx->setPlaceholder('widgets',$wOut);
                  }
                  
                  // SITE-SPECIFIC STUFF
                  $siteID = $modx->runSnippet('getSite');
                  $modx->setPlaceholder('site', $siteID);
                  $siteRes = $modx->getObject('modResource', $siteID);
                  $siteMenuTitle= $siteRes->get('menutitle');
                  $modx->setPlaceholder('siteMenuTitle', $siteMenuTitle);
                  
                    $TVobjs = array();
                    $rawTVs = array('bannerImage', 
                    			'siteType',
                  		  'siteTitle',
                  		  'siteJavascript', 
                  		  'siteParent',
                  		  'siteUltimateParent',
                  		  'siteMegaDropdown',
                  		  'siteSearchPage', 
                  		  'siteFileRepository',
                  		  'site_url',
                  		  'searchResultsContainer',
                  		  'siteRestrictionUserGroup',
                  		  'sitePrecontent'
                  		 ); 
                  	foreach ($rawTVs as $var) {
                      	if ($tv = $modx->getObject('modTemplateVar', array('name'=>$var))) {
                      		if ($val = $tv->getValue($siteID)) {
                  				$TVobjs[$var] = $val;	
                        		}
                      	}
                    	}
                  	$modx->setPlaceholders($TVobjs);
                  
                  switch ($TVobjs['siteType']) {
                    case 'orphan':
                              $mega = FALSE;
                              $redtab = FALSE;
                              $tabmenu = TRUE;
                              break;
                    case 'parent':
                              $redtab = $TVobjs['siteTitle'];
                              $mega = (isset($TVobjs['siteMegaDropdown'])) ? $TVobjs['siteMegaDropdown'] : FALSE;
                  	    	$tabmenu = ($mega && ($mega !='none')) ? FALSE : TRUE;
                  	    	break;
                    case 'child':
                  	   		$ultID = $TVobjs['siteUltimateParent'];
                  	    	$ultRes = $modx->getObject('modResource', $ultID);
                    			$modx->log(modX::LOG_LEVEL_ERROR, 'ultID: ' . $ultID );   	   
                  	    	$tv = $modx->getObject('modTemplateVar', array('name'=>'siteMegaDropdown'));
                   	    	$mega = ($val = $tv->getValue($ultID)) ? $val : FALSE;
                    			$modx->log(modX::LOG_LEVEL_ERROR, 'mega: ' . (string) $mega );   
                  	    	$tv = $modx->getObject('modTemplateVar', array('name'=>'siteTitle'));
                  	    	$redtab = ($val = $tv->getValue($ultID)) ? $val : FALSE;
                  	    	$tabmenu = TRUE;
                              break;
                  }
                  
                  if ((isset($mega)) && $mega != 'none') {     
                                 $megaPH = $mega;
                  			   
                                 $modx->setPlaceholder('jquery_megadropdown',$modx->getChunk('jquery_megadropdown'));
                  }
                  
                      if ($tabmenu) {
                        $modx->setPlaceholder('tabmenu', $modx->runSnippet('Wayfinder', array('startId'=>$siteID,
                  									    'level'=>'1',
                  									    'outerTpl'=>'tabLinksTpl',
                  									    'outerClass'=>'menu_dept',
                  									    'hereTpl'=>'tabLinksActiveRowTpl',
                  									    'activeParentRowTpl'=>'tabLinksActiveRowTpl'
                  									   )
                  							)
                  			   );
                        $modx->setPlaceholder('css_tabs', $modx->getChunk('css_tabs'));
                        if ($megaPH) {
                          $modx->setPlaceholder('upperMegaD', $megaPH);
                          $modx->setPlaceholder('css_mega', $modx->getChunk('css_mega'));
                        }
                        if ($redtab) {
                  	  	$modx->setPlaceholder('supertitle', $redtab);
                  		$modx->setPlaceholder('supersitelink', $modx->makeUrl($TVobjs[siteUltimateParent]));
                          $modx->setPlaceholder('upperSuperSiteTitle', $modx->getChunk('titletab', array('titletabclass'=>'top')));
                        }
                      }
                      else { // if $tabmenu is FALSE, this is a PARENT site
                        if ($megaPH) {
                          $modx->setPlaceholder('lowerMegaD', $megaPH);
                          $modx->setPlaceholder('css_mega', $modx->getChunk('css_mega'));
                        }
                        if ($redtab) {
                  	$modx->setPlaceholder('supertitle', $redtab);
                  	$modx->setPlaceholder('supersitelink', $modx->makeUrl($TVobjs[siteUltimateParent]));
                          $modx->setPlaceholder('lowerSuperSiteTitle', $modx->getChunk('titletab',
                                                 array('titletabclass'=>'bottom')));
                        }
                      }
                    $js_includes = $modx->runSnippet('js_includes');
                  
                  $modx->setPlaceholder('js_includes', $js_includes);
                  $modx->setPlaceholder('uwlogin', $modx->getChunk('UWlogin') );


                  One thing that definitely needs to be rewritten is the whole switch statement section and the part that immediately follows it. These are 3 main header layouts for a site, determined by a TV in the "Site Definition Resource". Instead of all this convoluted logic, it could be much more simply done by just having a chunk that corresponds to each layout, instead of a weird "one-size-fits-all" chunk that fills various placeholders to achieve different markup combinations. That was a stupid approach.

                    • 18373 ☆ A M B ☆
                    • 3,141 Posts
                    Suggestions.

                    Line 49-55:
                        foreach ($rawTVs as $var) {
                                    $TVobjs[$var] = $siteRes->getTVValue($var);
                        }

                    getTVValue uses element caching to fetch the TV. Same optimisation would be possible along lines 70-79.

                    Line 105
                    $modx->makeUrl($TVobjs[siteUltimateParent])

                    should be
                    $modx->makeUrl($TVobjs['siteUltimateParent'])


                    Other than that (and not understanding how this is easier to maintain than regular tags tongue) I don't see a lot of problems. Anything interesting in the "html" chunk?
                      Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

                      Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
                      • 14883 ☆ A M B ☆
                      • 450 Posts

                      Quote from: markh at May 01, 2013, 01:00 PM

                      Line 49-55:
                          foreach ($rawTVs as $var) {
                                      $TVobjs[$var] = $siteRes->getTVValue($var);
                          }

                      getTVValue uses element caching to fetch the TV.

                      Alright, how long has getVVValue been around, and why didn't anyone tell me about it before now? I've been repeatedly frustrated by the clunkiness of what I thought was the only API method for getting these values.

                      Quote from: markh at May 01, 2013, 01:00 PM
                      Suggestions.
                      Line 105
                      $modx->makeUrl($TVobjs[siteUltimateParent])

                      should be
                      $modx->makeUrl($TVobjs['siteUltimateParent'])

                      I think you just found a pesky source of makeUrl errors in my error log.

                      I should show people my crappy code more often.

                      Quote from: markh at May 01, 2013, 01:00 PM

                      Other than that (and not understanding how this is easier to maintain than regular tags tongue) I don't see a lot of problems. Anything interesting in the "html" chunk?

                      Nothing terribly interesting. There are a few snippet calls that build menus and such. Originally this snippet had even more things going on in it, running other snippets and setting the results as placeholders, but I moved some of it out.

                      I don't understand how this is easier to maintain than regular tags either. I think it was simply a development concept that outlived its usefulness and got taken to an extreme. Sometimes it isn't easy to stop and make time to revisit past assumptions and decisions. But sometimes, like now, it is necessary.