We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 36834
    • 25 Posts
    Here is a summary of how I am caching a set of search results (and their elements to speed up further cache creation). I am caching to the MODx resource cache, so that the cache is cleared when making any changes to other elements in the site like menus ($cache_options).

    $cache_options = array(
    	xPDO::OPT_CACHE_KEY => 'resource/web/propertymanager/properties',
    	xPDO::OPT_CACHE_HANDLER => 'xPDOFileCache',
    	xPDO::OPT_CACHE_EXPIRES => 0
    );
    $cache_page_key = md5($_SERVER['REQUEST_URI']);
    if ($cache = $modx->cacheManager->get($cache_page_key, $cache_options)) {
    	return $cache;
    }
    
    ... logic here ...
    
    foreach ($properties as $property) {
    	$cache_element_key = $property->id;
    	if ($cache = $modx->cacheManager->get($cache_element_key, $cache_options)) {
    		$properties_list .= $cache;
    	} else {
    		$output = $property_manager->getChunk('propertieslistitem', $property->toArray());
    		$properties_list .= $output;
    		$modx->cacheManager->set($cache_element_key, $output, 0, $cache_options);
    	}
    }
    $placeholders['properties_list'] = $properties_list;
    
    ... logic here ...
    
    $output = $property_manager->getChunk('propertieslist', $placeholders);
    $modx->cacheManager->set($cache_page_key, $output, 0, $cache_options);
    return $output;


    Because I am using custom resources I am also clearing the cache in my CMP processors

    $modx->cacheManager->refresh(array('resource' => array()));


    This really makes a massive difference. Another thing that I've found is that if things are a little slow, it's usually 'cos I've not been as efficient as I could have been with my code.

    Hope this helps ...
      • 3749
      • 24,544 Posts
      It's hard to know what you're doing without seeing the code, but it sounds like you're calling getChunk() 220 times. If you're using getChunk() to retrieve Tpl chunks to format data rows, it will be *way* faster if you just get the chunks once and reuse them by replacing the placeholders with str_replace().


      ---------------------------------------------------------------------------------------------------------------
      PLEASE, PLEASE specify the version of MODX you are using . . . PLEASE!
      MODX info for everyone: http://bobsguides.com/modx.html
        Did I help you? Buy me a beer
        Get my Book: MODX:The Official Guide
        MODX info for everyone: http://bobsguides.com/modx.html
        My MODX Extras
        Bob's Guides is now hosted at A2 MODX Hosting
        • 33968
        • 863 Posts
        @bob: Isn't that basically how getChunk works within a loop now? See this post by opengeek: http://forums.modx.com/thread/32070/xpdo---more-efficent-way-to-query-one-to-many-table#dis-post-173377
          • 3749
          • 24,544 Posts
          @Lucas, I had forgotten that mod. Thanks for reminding me, though I suspect that getting the chunk outside the loop would still be slightly faster -- I could be wrong.


          ---------------------------------------------------------------------------------------------------------------
          PLEASE, PLEASE specify the version of MODX you are using . . . PLEASE!
          MODX info for everyone: http://bobsguides.com/modx.html
            Did I help you? Buy me a beer
            Get my Book: MODX:The Official Guide
            MODX info for everyone: http://bobsguides.com/modx.html
            My MODX Extras
            Bob's Guides is now hosted at A2 MODX Hosting
            • 3749
            • 24,544 Posts
            I did a little testing, and the results are fairly dramatic.

            In a 100-iteration loop, I got these results on MODX 2.2.0:

            Inside the loop: 4.285 sec.
            Outside the loop: 1.028 sec.

            The results are similar whether the snippet is called cached or uncached and whether the database cache is on or off. Reversing the two tests made the 'outside the loop' version even more efficient. In every case, the 'outside the loop' version was at least 4 times as fast.

            Maybe there's something wrong with my test code. Here's the code I used:

                $output = '<p>Testing getChunk Times </p>';
                $replace = array('npx.fieldName' => 'This is the FieldName');
                $x = '';
            
                $output .= '<p>Outside the loop:</p>';
                /* set start time */
                $mtime = microtime();
                $mtime = explode(" ", $mtime);
                $mtime = $mtime[1] + $mtime[0];
                $tstart = $mtime;
                set_time_limit(0);
                
                $chunk = $modx->getChunk('npTextAreaTpl');
                for ($i = 1; $i < 100; $i++) {
                   $x .= str_replace('[[+npx.fieldName]]', 'This is the field name', $chunk);
                   if ($i ==1) {
                      $output .= $x;
                   }
                }
                
                /* report how long it took */
                $mtime= microtime();
                $mtime= explode(" ", $mtime);
                $mtime= $mtime[1] + $mtime[0];
                $tend= $mtime;
                $totalTime= ($tend - $tstart);
                $totalTime= sprintf("%2.4f s", $totalTime);
                $output .= $totalTime;
            
                $x='';    
                $output .= '<p>inside the loop:</p>';
                /* set start time */
                $mtime = microtime();
                $mtime = explode(" ", $mtime);
                $mtime = $mtime[1] + $mtime[0];
                $tstart = $mtime;
                set_time_limit(0);
                
                for ($i = 1; $i < 100; $i++) {
                   $x .= $modx->getChunk('npTextAreaTpl', $replace); 
                   if ($i ==1) {
                      $output .= $x;
                   }
                }
                
                /* report how long it took */
                $mtime= microtime();
                $mtime= explode(" ", $mtime);
                $mtime= $mtime[1] + $mtime[0];
                $tend= $mtime;
                $totalTime= ($tend - $tstart);
                $totalTime= sprintf("%2.4f s", $totalTime);
                $output .= $totalTime;
            
                
               
                return $output;

              Did I help you? Buy me a beer
              Get my Book: MODX:The Official Guide
              MODX info for everyone: http://bobsguides.com/modx.html
              My MODX Extras
              Bob's Guides is now hosted at A2 MODX Hosting
              • 33968
              • 863 Posts
              Bob - your code looks pretty good to me and yeah, that is a pretty dramatic difference. Would be keen to hear Jason's thoughts on this smiley
                • 36834
                • 25 Posts
                Bob's experiment reflects my findings also, having run a similar series of tests. One other observation that might be helpful to someone is that empty placeholders have a negative impact even in cache files. Inspecting your cache files will let you see whats getting through and then you can improve performance by setting them to be empty rather than letting MODx parse them out.

                If you can optimise your code and cache everything, then there isn't really any issue once the cache files have been generated. I too would interested in an opinion from the core team.
                  • 34004
                  • 125 Posts
                  simonp, your comment abouve ^^^^^ is very important, if you have a look in my thread, at my last post http://forums.modx.com/thread/32070/xpdo---more-efficent-way-to-query-one-to-many-table#dis-post-173381 I ask the same question.

                  On one page I could possibly have a thousand placeholders with nothing assigned, and the page would time out, even with the limit set to 3+ mins. Set those placeholders to "" and we are back to speedy 2 second loads. I was stumped for a long time...

                  I look forward to the core team opinion also!
                  • Before 2.1, iterating over a single Chunk can be done more efficiently like this:
                    <?php
                    $chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
                    $chunk->setCacheable(false);
                    for ($i = 0; $i < 1000; $i++) {
                        $chunk->process();
                    }
                    

                    This avoids having to load the Chunk object instance on each iteration, which is the major bottleneck you are exploring. This does not however, use the sourceCache and will generate a database query each time it is called, even if that chunk has been used before in the same request.

                    After 2.1, there is a parser method for working with single instances of any Element which directly uses an internal cache which is loaded by cached Resources without any db hits, and reused by the parser anytime the same element is called within the same request:
                    <?php
                    $chunk = $modx->getParser()->getElement('modChunk', 'testSpeed');
                    for ($i = 0; $i < 1000; $i++) {
                        $chunk->process();
                    }
                    

                    Note that getChunk also uses the sourceCache, but does not avoid object instantiation, and thus is not good to use iteratively like this.

                    Future work on MODX will be focused on reducing the overhead of class loading, object instantiation, unnecessary parsing passes, and total code so that these bottlenecks can be more easily avoided, and using them is as efficient as including a simple file and parsing it.

                    Here are some additional benchmarks for comparison:
                    [2012-02-28 08:17:13] (INFO) Processed 1000 chunks using getObject — 0.3189 s
                    
                    [2012-02-28 08:17:14] (INFO) Processed 1000 chunks using modParser::getElement — 0.2116 s
                    
                    [2012-02-28 08:17:17] (INFO) Processed 1000 chunks using getChunk — 2.8757 s
                    
                    [2012-02-28 08:17:17] (INFO) Processed 1000 iterations on included content using modParser::processElementTags — 0.0323 s


                    These benchmarks used the following script:
                    <?php
                    include 'config.core.php';
                    include MODX_CORE_PATH . 'model/modx/modx.class.php';
                    $modx = new modX();
                    $modx->initialize('web');
                    
                    $modx->setLogTarget(XPDO_CLI_MODE ? 'ECHO' : 'HTML');
                    $modx->setLogLevel(xPDO::LOG_LEVEL_INFO);
                    
                    $tstart = $modx->getMicroTime();
                    $chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
                    $chunk->setCacheable(false);
                    for ($i = 0; $i < 1000; $i++) {
                        $chunk->process();
                    }
                    $tend = $modx->getMicroTime();
                    $modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getObject — %2.4f s\n", $tend - $tstart));
                    
                    $tstart = $modx->getMicroTime();
                    $chunk = $modx->getParser()->getElement('modChunk', 'testSpeed');
                    for ($i = 0; $i < 1000; $i++) {
                        $chunk->process();
                    }
                    $tend = $modx->getMicroTime();
                    $modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using modParser::getElement — %2.4f s\n", $tend - $tstart));
                    
                    $tstart = $modx->getMicroTime();
                    for ($i = 0; $i < 1000; $i++) {
                        $modx->getChunk('testSpeed');
                    }
                    $tend = $modx->getMicroTime();
                    $modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getChunk — %2.4f s\n", $tend - $tstart));
                    
                    $tstart = $modx->getMicroTime();
                    $content = include 'test-content.php';
                    for ($i = 0; $i < 1000; $i++) {
                        $modx->getParser()->processElementTags('', $content);
                    }
                    $tend = $modx->getMicroTime();
                    $modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 iterations on included content using modParser::processElementTags — %2.4f s\n", $tend - $tstart));
                    exit();
                    
                      • 33968
                      • 863 Posts
                      That was a really informative post - thanks so much!

                      I've bookmarked this but it might be helpful for others if some of that could be included in the docs for getChunk or getParser... is that possible?