We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 3908
    • 6 Posts
    Hi guys!

    I'm not posting a question here actually as our problem has already been resolved, fortunately. But I wanted to share our experiences that might help someone eventually...

    So, our web site case specs:

    • AJAX driven web site installed on dedicated host
    • MODX Revolution 2.2.11 (traditional) - we used MODX Revo 2.2.11 but as it seems like this issue relates to ImageMagick more then to MODX and so I think it might affect latest MODX releases as well
    • phpThumbOf extra for generating image thumbnails (configured via manager to use convert utility)
    • 23,000+ resources in the database
    • 2,500,000 pageviews per month

    Host specs:

    • Debian 6 Squeeze (16 cores, 12GB RAM)
    • PHP 5.3.3-7 (300MB memory limit, APC 3.1.12 enabled)
    • Apache + Lighttpd (for serving static content like images, stylesheets, javascript)
    • MySQL 5.0.51a-24
    • ImageMagick 6.6.0-4

    The site had been working perfect during development but we ran into serious problems after going officially live. It seemed like high-traffic gradually consumed whole server RAM and finally led to server crash.

    I had just very basic (almost none) server debug info as server was maintained by outsourcing company, so I did the following:

    These things helped to get significantly better web site performance but in fact didn't stopped server crashes.

    Finally I got some better server debug info and realized that not the Apache and PHP but process convert was consuming server resources.

    I did some further invertigations and found out that:

    • there were some really big images (up to 14,500 x 8,000 pixels, copied from former web site resources) sent to phpThumbOf
    • there were non-image files (FLASH .swf) sent by editors to phpThumbOf
    • there was old version of ImageMagick installed on server (version 6.6.0-4 2012-05-02 Q16) - problems with ImageMagick crashing server by sending non-image file has been reported here: http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=18789

    I think normally sending non-image file to phpThumbOf should lead to just logging unsuccessful thumbnail creation attempt, that's all. But together with outdated ImageMagick installed on server and with the fact that unsuccessful attempts to send .swf files to phpThumbOf were processed very frequently - all resulted in server crashes.

    We finally solved our issue by installing latest version of ImageMagick (currently release version 6.8.8). And just for me to sleep well I also modified phpThumbOf snippet so that it rejects input files other then *.jpg, *.jpeg, *.png or *.gif.

    Well knowing all the above I encourage you guys to update the MODX server requirements page (http://rtfm.modx.com/revolution/2.x/getting-started/server-requirements) accordingly. Currently there is ImageMagick noted there without any specific version or without "knowing issues" related with it. [ed. note: borys last edited this post 12 years, 6 months ago.]
      • 35150 ☆ A M B ☆
      • 191 Posts
      Yes, phpThumbOf has a few face-palm-worthy moments when it comes to performance.

      Because of its dumb attempt at cache cleaning, the more images in your cache the slower phpThumbOf gets. On sites with a few thousand cached thumbnails this can add several seconds to the generation time for each page, even if phpThumbOf doesn't need to create any new thumbnails. Every time the snippet runs it tries to clean its cache. (And the comical irony is it never even deletes any files, just wastes time.)
      Also if you use the same image on multiple pages—for instance a banner image on each page in a given section—phpThumbOf will create a separate thumbnail for each page. This exacerbates the cache cleaning problem, plus it means visitors have to download the same image—just with a different name—each page they visit.

      Have you looked at pThumb? It's my effort to create a faster and more robust phpThumbOf. On most sites you can simply uninstall phpThumbOf, install pThumb and be done, with no changes to templates or chunks needed. I think you will notice the difference in speed.

      Although resizing a 116 megapixel image is going to be slow no matter what. That is a massive image.

      Also, your server would benefit from a newer version of PHP. 5.3 is reaching end of life, and newer versions improve performance and memory usage considerably.

      If you don't want to mess with anything new, at least go to System Settings, core > phpThumb and set phpThumb Max Cache* (3 settings) to 0. That will cut phpThumbOf's cache cleaning attempts short.
        Extras :: pThumb • Resizer • imageSlim • setPlaceholders
        • 3908
        • 6 Posts
        Hi, thanks a lot for the tip about phpThumb Max Cache settings, I've adjusted those accordingly.

        As for generating different image thumbnails for each page - yes, you're right. I noticed that and solved that by adjusting phpThumbOf settings via manager:

        • phpthumbof.hash_thumbnail_names - true
        • phpthumbof.postfix_property_hash - true

        As a result phpThumbOf generates full hash thumbnail filenames (e.g.: 84cc3d2069e615234bd8cc1902fcbaf8.403c40b2350129b432759647faa61941.jpg) and in that case uses one image thumbnail even on different pages.

        And yes, I've looked at pThumb extra and played with it on my local MODX installation for a while. It looked really promissing but unfortunately I misconfigured it somehow via manager and ended up with all images deleted - even those that belong to default manager theme. I know it's my fault and I did it by trying to setup pThumb paths to correspond our real server folder structure. But I didn't want to risk some other unexpected behavior on live server so just stuck with phpThumbOf. Currently - as clearing phpThumbOf cache is disabled and there are thousands of image thumbnails already generated in its cache - just about 10 new image thumbnails are generated per day so that's just fine. [ed. note: borys last edited this post 12 years, 6 months ago.]
          • 35150 ☆ A M B ☆
          • 191 Posts
          As a result phpThumbOf generates full hash thumbnail filenames (e.g.: 84cc3d2069e615234bd8cc1902fcbaf8.403c40b2350129b432759647faa61941.jpg) and in that case uses one image thumbnail even on different pages.

          Yes, if you're not worried about having human- and search-engine-readable image names, then setting those two options to true does sidestep a couple phpThumbOf bugs.

          About files getting deleted, if you change pthumb.clean_level to 2, pthumb.ptcache_location to / and clear the site cache, then it would remove *.jpe?g, *.gif and *.png from all subdirectories of MODX_BASE_PATH. I'll add a more selective regex filter in the next version, one that will only select images which have an 8-character hash in their filename ( '/.+\.[0-9a-f]{8}\.(jpg|png|gif)$/' ). I hadn't considered someone would do this, but I agree there should be better safeguards in place and probably a note too, saying / is generally not a good choice of location for the pThumb cache. (I do like having the option of putting the cache at the top level of the web root though; I usually set it to image-cache, which keeps the URLs nice and concise).

          There's also phpThumbOn which is pretty good, especially if you can read Russian.
            Extras :: pThumb • Resizer • imageSlim • setPlaceholders