We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 3749
    • 24,544 Posts
    RefreshCache does what it says, though I'm not particularly proud of it. It's an inelegant, brute-force way of refreshing the cache. It refreshes all cacheable, published resources that are not deleted and not hidden from menus by requesting all of them with cURL (whether they need refreshing or not -- it's not very intelligent). The cache for any cacheable snippets, chunks, or plugins involved will be refreshed as well.

    You can run it when you're finished working on your site so that visitors will get faster page loads.

    It's intentionally slow to avoid stressing the server and/or running afoul of bot-blocking scripts, so it will take a while to run. Expect 4-10 minutes per 100 cacheable pages.

    If you close the browser window during the run, it will abort and the next run will produce multiple progress bars that will go beyond 100%, but no harm will be done and the cache will still be refreshed. Just let it run to completion and subsequent runs should be normal.

    If you have a very large site, you may want to try changing the getCollection() call in the snippet to getIterator() (the arguments are the same) to save memory (execution time will be about double). The current version uses a lot of memory and I'll fix it when I get time. Note that in some versions of Revolution, getIterator() is broken and will request *all* resources on the site.

    Tutorial at: http://bobsguides.com/refreshcache-tutorial.html
      Did I help you? Buy me a beer
      Get my Book: MODX:The Official Guide
      MODX info for everyone: http://bobsguides.com/modx.html
      My MODX Extras
      Bob's Guides is now hosted at A2 MODX Hosting
      • 34178
      • 212 Posts
      Hi Bob,

      in combination with the CacheMaster we can now handle the caching of our site perfectly. The only thing we are missing is to also refresh sites not shown in the menu. Do you think it is possible to add a select-field to separately refresh the sites not included in the menu in a second step after refreshing all the other sites? We have many sites like confirm-sites of fomit-forms which were not shown in the menu but also should be in cache.

      BTW: Exactly at the end of processing by the snippet we always get an error-message in the error logfile of this kind:

      Error HY000 executing statement: 
      Array
      (
          [0] => XYZ
          [1] => XYZ
          [2] => MySQL server has gone away
      )
      


      followed by many codelines.

      Do you have any idea what could be the reason for the MySQL-Server-Issue right at the very end of the snippet and reproducible by every call of the snippet?
        • 34178
        • 212 Posts
        Hi Bob,

        we made some modifications regarding your very useful Snippet. Maybe you want to take over some of the code for one of the next updates.

        We added a variable to exclude resources which are not supposed to be loaded (e.g. 11,422,1555). So the the snippet-call is:
        [[!RefreshCache? &id_exceptions=`11,422,1555`]]


        In the snippet itself right before the submit-button we added the possibility to choose which kind of resources are going to be refreshed (menu-resources or not-menu-resources):
        <!-- Menu-Resources or not Part I (Html) -->
        <p>
        <input type="checkbox" name="menuyes" value="1" checked="checked" >Include menu-resources<br>
        <input type="checkbox" name="menuno" value="0" >Include non-menu-resources <br>
        </p>
        <!-- END Menu-Resources or not Part I -->
        


        To make theses features work it is necessary to add somewhere in the php-part this code:
        // Menu-Resources or not Part II (PHP)
        $id_exceptions = explode(',',$scriptProperties['id_exceptions']);
        if($_POST['menuno'] == 1 and $_POST['menuyes'] == 1){
        $hidemenu = 'hidemenu:!=';
        $menuvalue = 2;
        }
        
        if($_POST['menuno'] == 0 and $_POST['menuyes'] == 1){
        $hidemenu = 'hidemenu';
        $menuvalue = 0;
        }
        
        if($_POST['menuno'] == 1 and $_POST['menuyes'] == 0){
        $hidemenu = 'hidemenu';
        $menuvalue = 1;
        }
        // END Menu-Resources or not Part II
        


        Finally the definition of $c must be changed into:
            $c = array(
                    'published' => '1',
                    'deleted' => '0',
                    'cacheable' => '1',
                    $hidemenu => $menuvalue,
                    'class_key' => 'modDocument',
                    'id:NOT IN' => $id_exceptions,
            );
        


        Hope I did no mistakes when transferring the code. And please excuse the kind of coding. Probably you will find many points which could be solved much better, but at least in our settings it works.

        Thanks a lot again for this snippet.
          • 36788
          • 75 Posts
          Hi Bob,
          Can you please help me to solve an error with RefreshCache?

          It works perfectly on the same site on the local machine under wamp, but returns error on the shared hosting.
          On the shared hosting it refreshes from 10 to 40 documents (depends on refreshcache_ajax_delay setting) and then returns: "Unexpected token <: ERROR The requested URL could not be retrieved, Returned empty string (zero length), Squid didn't get any data".

          I've tested various settings of refreshcache_ajax_delay up to 1000 and refreshcache_curl_delay up to 5 sec the only difference is in number of refreshed documents. Though in all cases the error appears right in the beginning of the operation on 10 or 40 document.

          Modx error log says is empty
          The server error log is empty

          RefreshCache version 1.1.0-pl
          MODX Revolution 2.2.8-pl (traditional)
          PHP- Shared hosting: 5.4.16 / Local hosting: 5.3.13 [ed. note: sladdals last edited this post 10 years, 9 months ago.]
            • 16430
            • 217 Posts
            Hi, it would be great feature to add option which parent folder to refresh a leave the rest.
            Is it possible to run refreshcache as a cron job?
              • 19328
              • 433 Posts
              Hi Bob Ray, I'm using your refreshCache snippet, thanks for writing it! I have a problem though and I think it occurs since I started using revoSSL to make some pages https. I get the error: cURL error: 3 - <url> malformed for every URL except the homepage. The urls work fine. Do you have any idea what could be the problem? Thanks for any help!
                • 3749
                • 24,544 Posts
                @kudykam - I've never tried it, but I can't think of any reason that it wouldn't work.

                @michelle - It could well be revoSSL that's causing the trouble. Is the base href tag in your templates called uncached? If not try that:

                <base href="[[!++site_url]]">


                Also, do you have a rewrite rule in .htaccess that could be affecting this.

                I don't have the code for revoSSL, so I'm not sure how it works. RefreshCache uses this code to generate the URLs:

                $url = $modx->makeUrl($pageId, '', '', 'full');


                So makeUrl() needs to know what URL scheme is appropriate for each resource. It can be set in the call, but I don't know how RefreshCache could know which scheme to use (or if it would solve the problem if it did).

                Unfortunately, I couldn't find any decent information on what conditions can cause that cURL error. Could you paste one of the URLs that triggers the error here?
                  Did I help you? Buy me a beer
                  Get my Book: MODX:The Official Guide
                  MODX info for everyone: http://bobsguides.com/modx.html
                  My MODX Extras
                  Bob's Guides is now hosted at A2 MODX Hosting
                  • 19328
                  • 433 Posts
                  Hi Bob, that might be it. The base href is called uncached, but we changed it to //domain.nl (without the 'http' part). This was necessary to get the http/https switching to work correct (also for the included css / js files).

                  The rewrite rules concerning the SSL switching are:

                  RewriteCond %{HTTP_HOST} .
                  RewriteCond %{HTTPS} on
                  RewriteCond %{HTTP_HOST} !^www\.domain\.nl [NC]
                  RewriteRule (.*) https://www.domain.nl/$1 [R=301,L]
                  
                  RewriteCond %{HTTP_HOST} .
                  RewriteCond %{HTTPS} off
                  RewriteCond %{HTTP_HOST} !^www\.domain\.nl [NC]
                  RewriteRule (.*) http://www.domain.nl/$1 [R=301,L]
                  


                  The code of the RevoSSL plugin is:

                  /*
                   * Title:           RevoSSL
                   * Author:          Joshua Gulledge
                   * Created:         5/5/2011
                   * Updated:         4/11/2013
                   * Version:         1.0.1pl
                   * Description:     Allow users to make a page SSL and the Manager SSL
                   * 
                   * Based on Evo plugin:
                   * Title:      SSL
                   * Author:     Mark Croxton, [email protected]
                   * Desc:        Plugin triggered OnWebPageInit
                              Manages switching between https:// and http:// pages
                              Sets site-wide custom placeholders for secure/insecure server paths and base href
                  */
                  $makeSSL = false; 
                  switch($modx->event->name) {
                      /* on the manager load */
                      case 'OnBeforeManagerPageInit':
                      case 'OnManagerPageInit':
                      case 'OnManagerLoginFormPrerender':
                      //case 'OnManagerLoginFormRender':
                          /**
                           * Other events: 
                           * OnManagerPageInit
                           * OnManagerPageInit
                           * OnManagerLoginFormPrerender
                           * OnManagerLoginFormRender
                           * OnBeforeManagerLogin
                           * OnManagerAuthentication
                           * OnManagerLogin
                           */
                          // get a system setting - revoSSL.manager
                          $makeSSL = $modx->getOption('revoSSL.manager',$scriptProperties,false);
                          if ( $makeSSL == 'Yes' || $makeSSL == 'Y' ) {
                              $makeSSL = true;
                          }
                          break;
                      /* On web page loads */
                      case 'OnWebPagePrerender':
                          // if a property set is defined:
                          $makeSSL = $modx->getOption('makeSSL', $scriptProperties, 0);
                          if ( $makeSSL == '1' || $makeSSL == 1 ) {
                              $makeSSL = true;
                          }
                          $id = $modx->resourceIdentifier;
                          if ( $id > 0 ){
                              $resource = $modx->getObject('modResource', array('id' => $id ) );
                              // now if a TV is defined:
                              if (is_object($resource) ){
                                  // http://forums.modx.com/thread/83816/revossl---pretty-severe-bug
                                  if ($resource->get('published') == false || $resource->get('deleted')) {
                                      if ( !$modx->hasPermission('view_unpublished') ) {
                                          return false;
                                      }
                                  }
                                  // END
                                  $tmpSSL = $resource->getTVValue('makeSSL');
                                  if ( empty($tmpSSL)){
                                      
                                  } elseif ( $tmpSSL == '1' || $tmpSSL == 1 || $tmpSSL ) {
                                      $makeSSL = true;
                                  } else if ( $tmpSSL == 'No' || $tmpSSL == 'N' || !$tmpSSL ) {
                                      $makeSSL = false;
                                  }
                              }
                          }
                              
                          break;
                  }
                  // is the current page in SSL?
                  if( $_SERVER['HTTPS'] == 1 || $_SERVER['HTTPS'] == 'on' || $_SERVER['SERVER_PORT'] == 443) {
                      $ssl = true;
                  } else {
                      $ssl = false;
                  }
                  $force_redirect = false;
                  $host = $_SERVER['HTTP_HOST'];
                  // force www or no www
                  $force_www = $modx->getOption('revoSSL.forceWWW', $scriptProperties, false);
                  if ( strpos($_SERVER['HTTP_HOST'], 'www.') === false && $force_www ) {
                      // no WWW
                       $host = 'www.'.$_SERVER['HTTP_HOST'];
                       $force_redirect = true;
                  }
                  
                  // the current URL of the page
                  $url = $host.$_SERVER['REQUEST_URI'];//$_SERVER['PHP_SELF'];
                  // http://rtfm.modx.com/display/revolution20/modX.sendRedirect
                  
                  // switch between http / https if necessary
                  if ( $makeSSL && !$ssl ) {
                      // if SSL off and we are about to access a secure page then redirect
                      $modx->sendRedirect('https://'.$url);
                  } elseif (!$makeSSL && $ssl ) {
                      // if SSL is on and we are about to acccess an unsecure page then redirect
                      $modx->sendRedirect('http://'.$url);
                  } elseif ( $force_redirect ) {
                      // if SSL is on and we are about to acccess an unsecure page then redirect
                      if ( $makeSSL ) {
                          $modx->sendRedirect('https://'.$url);
                      } else {
                          $modx->sendRedirect('http://'.$url);
                      }
                  }
                  
                  return;


                  In short, the plugin looks for a TV 'makeSSL' to determine if the page should be http or https and redirects if necessary.

                  Based on the RefreshCache log it looks like the error occurred on all pages besides the homepage of << www. ebenhaezer . nl >>
                    • 3749
                    • 24,544 Posts
                    The base href is called uncached, but we changed it to //domain.nl (without the 'http' part).

                    I'm afraid I don't understand this. If the base href tag looks like the one I posted above, it should automatically include the correct scheme without the need to change anything, but then I'm not really qualified to discuss the intricacies of url management in a site with mixed http and https pages.

                    I think stuff like this is why many people choose to make the whole site use https.
                      Did I help you? Buy me a beer
                      Get my Book: MODX:The Official Guide
                      MODX info for everyone: http://bobsguides.com/modx.html
                      My MODX Extras
                      Bob's Guides is now hosted at A2 MODX Hosting
                      • 12410
                      • 353 Posts
                      Hi,
                      So is refresh cache supposed to populate public_html/core/cache/resource/web/resources
                      It runs visually 0-100 for me on Revo 2.3.3 but doesnt populate above folder? Folders are 777
                      Thanks

                      Added attached ... it goes beyond 100%...
                      [ed. note: howster last edited this post 9 years, 1 month ago.]