New Community Forums are coming. Watch this space for news.
Subscribe: RSS
  • Same topics as:
    - PHx not running when content is bigger than a certain limit?
    - Extended placeholders not interpreted

    AjaxSearch use Phx to set up the display of results. Unfortunately when the number of results is important the placeholders are broken and the display wrong ... Like this:
    [+as.noResults:is=`1`:then=`
    `:else=`
    Since several months i have discovered this issue and I didn’t understand why ...

    Now I have the proof of the limitation of the Modx Parser. This limitation comes from the PHP "preg_match_all" order.

    Here is a simple test (extracted from the ajaxResults template):

    [+as.noResults:is=`1`:then=`
    <div class="[+as.noResultClass+]">[+as.noResultText+]</div>
    `:else=`
    <p class="AS_ajax_resultsInfo">[+as.resultInfoText+]</p>
    [+as.listGrpResults+]
    `+]
    This is a very simple test. If noResults=0 you display the results (as.resultsInfoText and aslistGrpResults)

    On my laptop (XP, Php 5.3, php memory_limit = 32Mb) this test is correct until the size of the results is < 32kb 64 Kb (255 lines of 128 characters * 2 bytes).
    If the results reach the size of 32 kb (256 lines of 128 characters), the placeholders are broken.

    Find enclosed a small code to demonstrate this limitation. Look at the code. This test is done by calling the MODx parser (assets/plugin/phx class).
    <?php
    // Test of Phx by Coroico (coroico@wangba.fr)
    
    Define('LINE128','xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx');
    
    $tpl = <<<EOD
    [+as.noResults:is=`1`:then=`
    <div class="[+as.noResultClass+]">[+as.noResultText+]</div>
    `:else=`
    <p class="AS_ajax_resultsInfo">[+as.resultInfoText+]</p>
    [+as.listGrpResults+]
    `+]
    EOD;
    
    // Setup the MODx API
    define('MODX_API_MODE', true);
    include_once('index.php');
    $modx->db->connect();
    $modx->getSettings();
    
    global $modx;
    
    include_once('assets/plugins/phx/phx.parser.class.inc.php');
    ?>
    <div >
    <FORM method='POST'>
    Nb lines: <INPUT type='text' name="nbr" value="" />
      
    <INPUT type="submit" name="sub" value="GO" />
    </FORM>
    </div>
    <?php
    
    if ($_POST['nbr']) {
    
        $nbr = (int) $_POST['nbr'];  // nb of results
        $nbc = $nbr*128*2;    // nb bytes
        $size = $nbc/1024; // in Kb
        $size = $size."kb";
    
        if (class_exists('PHxParser')) {
    
            $varResults = array();
    
            for($i=0;$i<$nbr;$i++) $listGrpResults .= LINE128 ."\n\r";
    
            $phx = new PHxParser($tpl);
    
            $phx->setPHxVariable('as.listGrpResults',$listGrpResults);  //[+as.listGrpResults+]
            
            $resultInfoText = $nbr.' lines ('.$nbc.' characters) displayed. size='.$size;
            $phx->setPHxVariable('as.resultInfoText',$resultInfoText);  // [+as.resultInfoText+]
            
            $phx->setPHxVariable('as.noResults',0);     // [+as.noResults+]
            
            $output = $phx->parse($tpl) . "\n";
    
            echo $output;
            return;
        }
    }
    ?>

    Copy/paste the above code in a testPhx.php file and execute this file at the root of your Modx environment.

    Then enter a value of 255 and see the results (255 lines of xxx should be displayed).
    Do the same with 256 lines and see how the placerholders are broken.

    May be the limits could be different on your environment. On my latop the limit is between 255 and 256.

    I found this article about a similar issue. I haven’t still tried to increase the pcre.recursion_limit (see http://docs.php.net/manual/en/pcre.configuration.php). This solution seems a bit dangerous.
    So i think that I will simply reorganise the display of AjaxSearch differently in order to avoid to use the parser to concatenate the results to the layout template laugh

    Hope this post help to explain some issues with phx and large volume of data.




    • Just of out curiosity, in my php_init file,I have doubled the value of the pcre.bactrack_limit. From 100000 to 200000.

      ;PCRE library backtracking limit.
      pcre.backtrack_limit=200000
      With this value, now I could display 512 lignes instead of 256 wink

      As one UTF-8 character is encoded on 2 bytes, the correct size limit with the default value of 100000 for the pcre.backtrack_limit is 256*128*2 = 64kb rather than 32kb.

      About pcre.backtrack_limit : http://bugs.php.net/bug.php?id=40846
      • Nice find coroico.

        Now we know whats happening behind the scenes.

        Thanks
          Zaigham R - MODX Professional | Skype | Email | Twitter

          Digging the interwebs for #MODX gems and bringing it to you. modx.link
        • Just noticed this thread by chance. I too have come across this problem and ended up having to increase the backtrack limit size when parsing multilingual document templates within YAMS. (These can be approx. n times the size of normal document templates, where n is the number of languages.) I was wondering how big I would need to make the backtrack limit to be sure to avoid problems. I ended up using:
          ini_set('pcre.backtrack_limit', '16000000');
          

          within the code. This is approximately the maximum number of single byte characters that are allowed in the MODx content field. Although the number is very large I haven’t encountered any negative side-effects.
            YAMS: Yet Another Multilingual Solution for MODx
            YAMS Forums | Latest: YAMS 1.1.9 | YAMS Documentation
            Please consider donating if you appreciate the time and effort spent developing and supporting YAMS.
          • ini_set('pcre.backtrack_limit', '16000000');
            Do you know the corresponding size allowed with this value ? If my assumptions are correct = 64 * 16000000 / 100000 / 1024 = 10 Mb !

            With this value, you haven’t memory bottleneck during the script execution ?
            • No. As I understand it the backtrack_limit setting sets the maximum number of bytes preg is allowed to search over before it gives up. The strings I’m searching are never as big as 10MB however. Most document templates/web pages are smaller than 1 MB. This setting just guarantees that preg will search the whole string before giving up - which is what I wanted in this case.
                YAMS: Yet Another Multilingual Solution for MODx
                YAMS Forums | Latest: YAMS 1.1.9 | YAMS Documentation
                Please consider donating if you appreciate the time and effort spent developing and supporting YAMS.
              • Great find, coroico! Thank you for your input, PMS. This definitely helps clarify things for me on my end. Will try increasing the limit on some currently impacted sites and post-back with the results. cool
                  Mike Reid - www.pixelchutes.com
                  MODx Ambassador / Contributor
                  [Module] MultiMedia Manager / [Module] SiteSearch / [Snippet] DocPassword / [Plugin] EditArea / We support FoxyCart
                  ________________________________
                  Where every pixel matters.
                • Quote from: pixelchutes at Nov 24, 2009, 06:53 AM

                  Great find, coroico! Thank you for your input, PMS. This definitely helps clarify things for me on my end. Will try increasing the limit on some currently impacted sites and post-back with the results. cool

                  So awesome! This totally fixed any PHx limitation I was previously running into. By updating the PCRE backlimit configuration in the php.ini, all of my sites that were previously impacted have inherently been fixed. Plenty to be thankful for this Thanksgiving wink
                    Mike Reid - www.pixelchutes.com
                    MODx Ambassador / Contributor
                    [Module] MultiMedia Manager / [Module] SiteSearch / [Snippet] DocPassword / [Plugin] EditArea / We support FoxyCart
                    ________________________________
                    Where every pixel matters.
                  • Thanks Mike. I spend some hours with Xdebug to undertsand and identify where the problem is. But I am happy with the results now.
                    • Sweet! smiley
                        @hawproductions | http://mrhaw.com/

                        Infograph: MODX Advanced Install in 7 steps:
                        http://forums.modx.com/thread/96954/infograph-modx-advanced-install-in-7-steps

                        Recap: Portland, OR (PDX) MODX CMS Meetup, Oct 6, 2015. US Bancorp Tower
                        http://mrhaw.com/modx_portland_oregon_pdx_modx_cms_meetup_oct_2015_us_bancorp_tower