We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 19031
    • 1 Posts
    Hello,

    Can you please reference the code where the context.cache.php is generated?

    I noticed that when posting or refreshing the manager, it calls
    http://mysite.com/connectors/resource/index.php?action=getNodes&id=web_0&type=modContext

    Thanks
      • 14915
      • 43 Posts
      We have many thousands of documents (50,000+). When the context cache is being re-built, the context cache file is deleted from disk, and remains gone for at least a minute or two. In the mean time, the site comes to a stand-still and the manager becomes un-usable (as the file tree disappears). Is there a way to change how the context cache is re-generated so that:

      - The context cache is first rebuilt in memory
      - The old context file is deleted
      - The new context file written

      Also - when a document is saved, the entire context cache is rebuilt. This results in tens (or hundreds) of thousands of queries being run. This happens every time a document is saved. Is there a way to just look up what’s changed for the document being saved, and just merge those changes in the context cache array, before it is saved to disk? This could mean running 15 - 20 queries instead of 100,000 queries. I don’t expect a full solution here, but a pointer in the right direction (in terms of which code to edit) would be greatly appreciated.

      Thank you!
        • 22303 MODX Staff
        • 10,725 Posts
        The cache clearing mechanism clears the cache files. The first request to the site that needs that information rebuilds it. You should never have that many Resources in a single Context.
          • 14915
          • 43 Posts
          Hello,

          the actual generation of the context cache file seems to be here:

          core/model/modx/modcachemanager.class.php -> generateContext()

          /**
               * Generates a cache entry for a MODx site Context.
               *
               * Context cache entries can override site configuration settings and are responsible for
               * loading the various lisitings and maps in the modX class, including documentListing,
               * documentMap, and eventMap.  It can also be used to setup or transform any other modX
               * properties.
               *
               * @todo Refactor the generation of documentMap, aliasMap, and
               * resourceListing so it uses less memory/file size.
               *
               * @param modContext $obj  The modContext instance to be cached.
               * @param array $options Options for system settings generation.
               * @return array An array containing all the context variable values.
               */
              public function generateContext($key, array $options = array()) {
                  $results = array();
                  $obj= $this->modx->getObject('modContext', $key, true);
                  if (is_object($obj) && $obj instanceof modContext && $obj->get('key')) {
                      $contextConfig= $this->modx->config;
          
                      /* generate the ContextSettings */
                      $results['config']= array();
                      if ($settings= $obj->getMany('ContextSettings')) {
                          foreach ($settings as $setting) {
                              $k= $setting->get('key');
                              $v= $setting->get('value');
                              $matches = array();
                              if (preg_match_all('~\{(.*?)\}~', $v, $matches, PREG_SET_ORDER)) {
                                  $matchValue= '';
                                  foreach ($matches as $match) {
                                      if (isset ($this->modx->config["{$match[1]}"])) {
                                          $matchValue= $this->modx->config["{$match[1]}"];
                                      } else {
                                          $matchValue= '';
                                      }
                                      $v= str_replace($match[0], $matchValue, $v);
                                  }
                              }
                              $results['config'][$k]= $v;
                              $contextConfig[$k]= $v;
                          }
                      }
          
                      /* generate the documentMap, aliasMap, and resourceListing */
                      $tblResource= $this->modx->getTableName('modResource');
                      $tblContextResource= $this->modx->getTableName('modContextResource');
                      $resourceFields= 'id,parent,alias,isfolder,content_type';
                      if (isset ($contextConfig['cache_context_resourceFields']) && $contextConfig['cache_context_resourceFields']) {
                          $resourceFields= $contextConfig['cache_context_resourceFields'];
                      }
                      $resourceCols= $this->modx->getSelectColumns('modResource', 'r', '', explode(',', $resourceFields));
                      $bindings= array (
                          ':context_key1' => array('value' => $obj->get('key'), 'type' => PDO::PARAM_STR)
                          ,':context_key2' => array('value' => $obj->get('key'), 'type' => PDO::PARAM_STR)
                      );
                      $criteria= new xPDOCriteria($this->modx, "SELECT {$resourceCols} FROM {$tblResource} `r` LEFT JOIN {$tblContextResource} `cr` ON `cr`.`context_key` = :context_key1 AND `r`.`id` = `cr`.`resource` WHERE `r`.`id` != `r`.`parent` AND (`r`.`context_key` = :context_key2 OR `cr`.`context_key` IS NOT NULL) AND `r`.`deleted` = 0 GROUP BY `r`.`id` ORDER BY `r`.`parent` ASC, `r`.`menuindex` ASC", $bindings, false);
                      if (!$collContentTypes= $this->modx->getCollection('modContentType')) {
                          $htmlContentType= $this->modx->newObject('modContentType');
                          $htmlContentType->set('name', 'HTML');
                          $htmlContentType->set('description', 'HTML content');
                          $htmlContentType->set('mime_type', 'text/html');
                          $htmlContentType->set('file_extensions', 'html,htm');
                          $collContentTypes['1']= $htmlContentType;
                      }
                      $collResources= null;
                      if ($criteria->prepare() && $criteria->stmt->execute()) {
                          $collResources= & $criteria->stmt;
                      }
                      if ($collResources) {
                          $results['resourceMap']= array ();
                          $results['resourceListing']= array ();
                          $results['aliasMap']= array ();
                          $results['documentMap']= array ();
                          $containerSuffix= isset ($contextConfig['container_suffix']) ? $contextConfig['container_suffix'] : '';
                          while ($r = $collResources->fetch(PDO::FETCH_OBJ)) {
                              $parentId= isset($r->parent) ? strval($r->parent) : "0";
                              $results['documentMap'][]= array("{$parentId}" => (string) $r->id);
                              $results['resourceMap']["{$parentId}"][] = (string) $r->id;
                              $resourceValues= get_object_vars($r);
                              $results['resourceListing'][(string) $r->id]= $resourceValues;
                              $resAlias= '';
                              $resPath= '';
                              $contentType= isset ($collContentTypes[$r->content_type]) ? $collContentTypes[$r->content_type] : $collContentTypes['1'];
                              if ((isset ($obj->config['friendly_urls']) && $obj->config['friendly_urls']) || $contextConfig['friendly_urls']) {
                                  if ((isset ($obj->config['friendly_alias_urls']) && $obj->config['friendly_alias_urls']) || $contextConfig['friendly_alias_urls']) {
                                      $resAlias= $r->alias;
                                      if (empty ($resAlias)) $resAlias= $r->id;
                                      $parentResource= '';
                                      if ((isset ($obj->config['use_alias_path']) && $obj->config['use_alias_path'] == 1) || $contextConfig['use_alias_path']) {
                                          $pathParentId= $parentId;
                                          $parentResources= array ();
                                          $currResource= $r;
                                          $parentSql= "SELECT {$resourceCols} FROM {$tblResource} `r` WHERE `r`.`id` = :parent AND `r`.`id` != `r`.`parent` LIMIT 1";
                                          $hasParent= (boolean) $pathParentId;
                                          if ($hasParent) {
                                              if ($parentStmt= $this->modx->prepare($parentSql)) {
                                                  $parentStmt->bindParam(':parent', $pathParentId);
                                                  if ($parentStmt->execute()) {
                                                      while ($hasParent && $currResource= $parentStmt->fetch(PDO::FETCH_OBJ)) {
                                                          $parentAlias= $currResource->alias;
                                                          if (empty ($parentAlias))
                                                              $parentAlias= "{$pathParentId}";
                                                          $parentResources[]= "{$parentAlias}";
                                                          $pathParentId= $currResource->parent;
                                                          $hasParent= ($pathParentId > 0 && $parentStmt->execute());
                                                      }
                                                  }
                                              }
                                          }
                                          $resPath= !empty ($parentResources) ? implode('/', array_reverse($parentResources)) : '';
                                      }
                                  } else {
                                      $resAlias= $r->id;
                                  }
                                  if (!empty($containerSuffix) && $r->isfolder) {
                                      $resourceExt= $containerSuffix;
                                  } else {
                                      $resourceExt= $contentType->getExtension();
                                  }
                                  if (!empty($resourceExt)) {
                                      $resAlias .= $resourceExt;
                                  }
                              } else {
                                  $resAlias= $r->id;
                              }
                              $results['resourceListing'][(string) $r->id]['path']= $resPath;
                              if (!empty ($resPath)) {
                                  $resPath .= '/';
                              }
                              if (isset ($results['aliasMap'][$resPath . $resAlias])) {
                                  $this->modx->log(xPDO::LOG_LEVEL_ERROR, "Resource alias {$resPath}{$resAlias} already exists for resource id = {$results['aliasMap'][$resPath . $resAlias]}; skipping duplicate resource alias for resource id = {$r->id}");
                                  continue;
                              }
                              $results['aliasMap'][$resPath . $resAlias]= $r->id;
                              $results['aliasListing']["{$resPath}{$resAlias}"]= (string) $r->id;
                          }
                      }
          
                      /* generate the eventMap and pluginCache */
                      $results['eventMap'] = array();
                      $results['pluginCache'] = array();
                      $eventMap= $this->modx->getEventMap($obj->get('key'));
                      if (is_array ($eventMap) && !empty($eventMap)) {
                          $results['eventMap'] = $eventMap;
                          $pluginIds= array ();
                          $this->modx->loadClass('modScript');
                          foreach ($eventMap as $pluginKeys) {
                              foreach ($pluginKeys as $pluginKey) {
                                  if (isset ($pluginIds[$pluginKey])) {
                                      continue;
                                  }
                                  $pluginIds[$pluginKey]= $pluginKey;
                                  $plugins[$pluginKey]= $this->modx->getObject('modPlugin', intval($pluginKey), true);
                              }
                          }
                          if (!empty ($plugins)) {
                              foreach ($plugins as $pluginId => $plugin) {
                                  if (!is_object($plugin)) {
                                      continue;
                                  }
                                  $results['pluginCache'][(string) $pluginId]= $plugin->toArray('', true);
                              }
                          }
                      }
                      if ($this->getOption('cache_context_settings', $options, true)) {
                          $options[xPDO::OPT_CACHE_KEY] = $this->getOption('cache_context_settings_key', $options, 'default');
                          $options[xPDO::OPT_CACHE_HANDLER] = $this->getOption('cache_context_settings_handler', $options);
                          $lifetime = intval($this->getOption(xPDO::OPT_CACHE_EXPIRES, $options, 0));
                          if (!$this->set($obj->getCacheKey(), $results, $lifetime, $options)) {
                              $this->modx->log(modX::LOG_LEVEL_ERROR, 'Could not cache context settings for ' . $obj->get('key') . '.');
                          }
                      }
                  }
                  return $results;
              }
          


          Can you confirm that this is correct?
            • 14915
            • 43 Posts
            Also, as a side-note, it seems that the need for performance improvements for this section of code has already been documented in the code:

            @todo Refactor the generation of documentMap, aliasMap, and
                 * resourceListing so it uses less memory/file size.