Error Logs out of Control

☆ A M B ☆
2,475 Posts

Everett Reply #1, 10 years, 11 months ago

I'm losing my mind on this. My error logs grow to 40 mb or so in a single day, and all the messages are stupid things from "bad" URLs. I've been digging around in the database for weeks now. Why is this happening? Why can't MODX figure out what's the correct URL? How can this be fixed? I want my error log spotless. I want it empty. If a single thing gets written to it, I want alarm bells going off. Instead I have firehose that can't be turned off.

[2013-06-04 11:18:27] (ERROR @ /connectors/system/settings.php) Resource URI locations/xxxx/raystown-lake-visitor-center.html already exists for resource id = 1256; skipping duplicate resource URI for resource id = 1729
[2013-06-04 11:18:27] (ERROR @ /connectors/system/settings.php) Resource URI locations/yyyy/jc-blair-memorial-hospital.html already exists for resource id = 996; skipping duplicate resource URI for resource id = 1612
[2013-06-04 11:18:27] (ERROR @ /connectors/system/settings.php) Resource URI locations/zzzz/standing-stone-coffee-company.html already exists for resource id = 1356; skipping duplicate resource URI for resource id = 1357

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

1,572 Posts

Paulp Reply #2, 10 years, 11 months ago

Do the resources mentioned in each error have the same URL alias ?, ie 1256 and 1729

☆ A M B ☆
2,475 Posts

Everett Reply #3, 10 years, 11 months ago

Yep. But I can't account for why. I've been in the database several times cleaning it out manually, but it's like I got a leaky plugin or something.

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

MODX Staff
1,535 Posts

Garry Nutting Reply #4, 10 years, 11 months ago

What version of MODX are you using and is this happening after an upgrade from an earlier version?

Garry Nutting
Senior Developer
MODX, LLC

Email: [email protected]
Twitter: @garryn
Web: modx.com

712 Posts

jpdevries Reply #5, 10 years, 11 months ago

What version of MODX?

jpdevries

☆ A M B ☆
2,475 Posts

Everett Reply #6, 10 years, 11 months ago

MODX version 2.2.7.

Nope, this behavior has been going on for a long time, it's not something that happened after an upgrade. Usually it's the phpThumb errors that kill a log (that's another rant: somebody's got to write some unit tests for that), but this is almost entirely URL warnings. This would be a good podcast topic: how to clean yer logs...

I'm gonna start hacking the core to pinpoint this.

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

☆ A M B ☆
2,475 Posts

Everett Reply #7, 10 years, 11 months ago

It's coming from the core/model/modx/modcachemanager.class.php's generateContext() function, line 101:

// ...
if ($this->modx->getOption('friendly_urls', $contextConfig, false) && $this->getOption('cache_alias_map', $options, false)) {
    if (array_key_exists($r->uri, $results['aliasMap'])) {
        $this->modx->log(xPDO::LOG_LEVEL_ERROR, "Resource URI {$r->uri} already exists for resource id = {$results['aliasMap'][$r->uri]}; skipping duplicate resource URI for resource id = {$r->id}");
        continue;
    }
    $results['aliasMap'][$r->uri]= (integer) $r->id;
}

So clearing the cache (for example), will unleash a flurry of errors.

Here's the MySQL query I'm using to identify the problematic resources:

SELECT uri, count(*) as cnt FROM `modx_site_content`
GROUP BY uri
HAVING cnt > 1

Still digging... [ed. note: Everettg_99 last edited this post 10 years, 11 months ago.]

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

☆ A M B ☆
2,475 Posts

Everett Reply #8, 10 years, 11 months ago

My 'global_duplicate_uri_check' system setting is set to "No".

The following files use the 'duplicate_uri_found' lexicon key:

./core/model/modx/processors/resource/update.class.php
./core/model/modx/processors/resource/publish.class.php
./core/model/modx/processors/resource/create.class.php

Of interest here is also the isDuplicateAlias() function inside of core/model/modx/modresource.class.php:

public function isDuplicateAlias($aliasPath = '', $contextKey = '') {
    if (empty($aliasPath)) $aliasPath = $this->getAliasPath($this->get('alias'));
    $criteria = $this->xpdo->newQuery('modResource');
    $where = array(
        'id:!=' => $this->get('id'),
        'uri' => $aliasPath,
        'deleted' => false,
        'published' => true
    );
    if (!empty($contextKey)) {
        $where['context_key'] = $contextKey;
    }
    $criteria->select('id');
    $criteria->where($where);
    $criteria->prepare();
    $duplicate = $this->xpdo->getValue($criteria->stmt);
    return $duplicate > 0 ? (integer) $duplicate : false;
}

In checking a couple of my problem entries, it's clear that in each case only 1 of the duplicates is published... so the isDuplicateAlias() always gives them a clean pass, whereas the cache manager doesn't appear to distinguish between the published and not-published status.

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

☆ A M B ☆
2,475 Posts

Everett Reply #9, 10 years, 11 months ago

SOLVED: in System Settings, I turned off cache_alias_map

Counter-intuitively, the larger the site, the better it is to turn that setting off because PHP doesn't store the data very efficiently. So the getResourceCacheMap() functionality caches resource data by their URI, and if there is a non-unique URI, an error is thrown. This collision happens when 2 resources share the same URI, and one of them is published and the other is not.

http://tracker.modx.com/issues/9963 -- I submitted a patch for this that simply includes that system setting name in the error message. [ed. note: Everettg_99 last edited this post 10 years, 11 months ago.]

Hosting stuff: http://fireproofsocks.com/
Coding stuff: http://craftsmancoding.com/
Blog stuff: http://tipsfor.us/

712 Posts

jpdevries Reply #10, 10 years, 11 months ago

2.2.8 also may fix a related bug

jpdevries