We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 39897
    • 5 Posts
    I've just recently started playing around with MODX, and today I wrote a snippet that successfully integrates the Python Pygments syntax highlighter. This is actually the first MODX snippet I've written, and frankly...I'm not 100% sure how or why it works. But nevertheless, I want to share it here and allow others to comment on it, edit it, use it, etc.

    This snippet is a server-side implementation of syntax highlighting. For a client-side implementation, see the SyntaxHighlighter MODX package by Bob Ray. This snippet also assumes you have python and the Pygmentize script installed on your system.

    Beyond Syntax Highlighting:

    One interesting thing I realized about this snippet is that it can be rewritten to connect MODX to any server script in any language, provided the script can be executed via the command line.

    Credit / Resources used:

    Pygments syntax highlighter
    Bob's SyntaxHighlighter Tutorial

    Disclaimer:

    This snippet was written by a casual hacker with virtually no professional experience or knowledge of any of the technologies or languages contained herein. This is copy+paste programming--at no time did I fully understand what the hell I was doing. Use at your own risk.

    The Code:

    <?php
    
    /**
    
    DIRECTIONS:
    
    1. Edit $codePath to point to a folder where you've kept source code files to 
    apply syntax highlighting to.
    2. Create a file called "test.html" and place it in the $codePath folder.
    3. Save this as a snippet called "Pygmentize".
    4. Insert the following into a template: [[Pygmentize? &file=`somefile.ext`]] 
    where "somefile.ext" is either test.html or some other file you want to highlight.
    
    **/
    
    
    
    // Define the absolute path to the file we want to use as our source code to 
    // highlight.
    
    // Assumes you have created a folder in assets called "code"
    $codePath = '/PATH/TO/assets/code/';
    
    // Assumes you have created a test file called "test.html" and placed it in 
    // your /assets/code directory. 
    // Also assumes you will call this snippet and provide a file name like this:  
    // [[Pygmentize? &file=`yourfile.ext`]]
    $fileName = empty($scriptProperties['file']) ? 'test.html' : $scriptProperties['file'];
    
    $filePath = $codePath . $fileName;
    
    
    
    // The next lines of code set command line options (flags), based on whether or 
    // not those options were specified in the snippet call. In other words, we're 
    // converting MODX snippet call parameters into system command line parameters 
    // to use with the "pygmentize" CLI command.
    // 
    // See also: http://pygments.org/docs/cmdline/
    
    // If a lexer wasn't specified, leave this empty so Pygments can choose one 
    // based on the file extension.
    $lexer = empty($scriptProperties['l']) ? ('') : (' -l ' . $scriptProperties['l']);
    
    // If a formatter wasn't specified, default to HTML.
    $formatter = empty($scriptProperties['f']) ? (' -f html') : (' -f ' . $scriptProperties['f']);
    
    // If options were not specified, leave it blank. Otherwise, wrap in quotes.
    // NOTE: has not been tested with multiple options
    $options = empty($scriptProperties['O']) ? ('') : (' -O "' . $scriptProperties['O'] . '"');
    
    
    
    // If CSS output has not been disabled, register it for the current MODX resource.
    // To disable CSS output, add this to the MODX snippet tag: &S=`off`
    if (($scriptProperties['S'] != 'none') && ($scriptProperties['S'] != 'off')) {
    
        ob_start();
    
        // If a style was specified, use it for the CSS output.
        $style = empty($scriptProperties['S']) ? (' -S default') : (' -S ' . $scriptProperties['S']);
    
        // If an "a" value was specified, use it for the CSS output class prefix. 
        // Otherwise, default to the prefix ".sh".
        $a = empty($scriptProperties['a']) ? (' -a .highlight') : (' -a ' . $scriptProperties['a']);
    
        $cssCommand = 'pygmentize' . $formatter . $style . $a;
    
        system(escapeshellcmd($cssCommand));
    
        $css = ob_get_clean();
    
        $modx->regClientCSS('    <style type="text/css">' . "\n" . $css . "\n" . '    </style>');
    }
    
    
    
    $pygCommand = 'pygmentize' . $lexer . $formatter . $options . ' ' . $filePath;
    
    system(escapeshellcmd($pygCommand));
    
    
      • 39897
      • 5 Posts
      Forgot to Mention:

      I should also note that I intentionally chose Pygments because it allows for highlighting of individual files (versus pasting the code directly into a MODX resource's HTML code). This also allows for easy downloading/saving of code files, since you can direct users directly to the raw code files. To make sure these raw code files are served as plain text, I added the following to my server directives in my nginx.conf file:

      location /assets/code/ {
          types  { }
          default_type  text/plain;
      }


      How to Improve This Snippet:

      This snippet is essentially executing the Pygmentize python script every time a page (containing the snippet tag) is built. It doesn't benefit from caching the snippet's PHP code, because the PHP code just calls the python script. It would be nice to include some logic that caches the python script's output and only executes the python script again when the source code file (e.g. /assets/code/test.html) changes.
        • 39897
        • 5 Posts
        How to Install Python Pygments:

        There are general installation instructions here.

        I'm running Debian 6 "Squeeze", so installation was quite simple via SSH command line:

        apt-get install python-pygments
          • 39897
          • 5 Posts
          Speed Test: Pygmentize vs. SyntaxHighlighter:

          Bob Ray asked me how the Pygmentize snippet compares against his SyntaxHighlighter snippet, in terms of page load times...so I ran a basic test to compare them. These results aren't exactly conclusive, as I don't have the resources to create a tightly-controlled test environment, but they're good enough to give us a rough idea of how well Pygmentize performs. (Spoiler: Pygmentize is surprisingly fast.)

          Setup

          I created 2 test pages. Both use a bare-bones MODX template that contains only 3 dynamic tags:

          [[*longtitle]]
          [[*pagetitle]]
          [[*content]]


          These tags are, of course, placed in a normal HTML page template that contains no additional embedded content or resources. In other words, there are no unrelated images, .js, or .css files loaded.

          shtest.html

          For the first page, the content was a small piece of JavaScript code (~50 lines of code), wrapped in <pre> tags and preceded by the SyntaxHighlighter snippet tag like this:

          [[SyntaxHighlighter? &brushes=`JScript`]]
          <pre class="brush: js">
              ~50 lines of code
          </pre>


          pytest.html

          For the second page, the content was the Pygmentize snippet tag like this:

          [[Pygmentize? &S=`friendly` &file=`test.js`]]


          ...where test.js is a file I saved to "/assets/code/" and which contains the same ~50 lines of code that were in the <pre> tags of the first page.

          Tool + Settings

          WebPageTest

          Location: Kansas City, MO
          Browser: Chrome
          Connection Speed: FIOS
          Runs: 10

          Process

          I ran both pages twice (for a total of 20 individual test runs) and averaged the results. I will show the data here for each set (of 10) and the averages.

          Test Results

                           SyntaxHighlighter
          
                       Load Time    First Byte    Start Render
          Run  1-10    0.335s       0.093s        0.297s
          Run 11-20    0.310s       0.093s        0.337s
          -----------------------------------------------------
          Average      0.323s       0.093s        0.317s
          
          
          
                              Pygmentize
          
                       Load Time    First Byte    Start Render
          Run  1-10    0.219s       0.091s        0.191s
          Run 11-20    0.228s       0.095s        0.204s
          -----------------------------------------------------
          Average      0.224s       0.093s        0.198s
          


          Conclusion

          To the non-technical eye, these results may suggest that Pygmentize is the clear winner, but that's not necessarily the case. In reality, these two snippets work differently--with many variables to consider--and this test targeted just one possible scenario and reported on one small set of test data (i.e., page load times). Here are some additional points to consider...

          Pygmentize uses your server's processor and memory to parse the code snippet and wrap each token in the appropriate HTML tags before returning it to MODX. How intensive is this use of server resources? I don't know. But Pygmentize's results aren't cached, so in its current form, this snippet causes the Pygmentize python script to execute on every page load. Scaling might quickly become an issue here.

          SyntaxHighlighter, on the other hand, is mostly a client-side program. The MODX snippet's main function is to provide a convenient mechanism for pairing code snippets with their corresponding SyntaxHighlighter brushes. The main difference here is that SyntaxHighlighter uses the CPU and memory resources of the Users' computer, as it's all handled via JavaScript.

          So...while the test results above show Pygmentize as being faster, keep in mind that this test was run on a very low-traffic server. Additionally, keep in mind that this test was for "out of the box" performance--SyntaxHighlighter could have been optimized, according to the tips provided in Bob's tutorial under the heading, Speed Considerations.