Help with getChunk and MODx speed please - My Forums

Re: Help with getChunk and MODx speed please

exside — Apr 26, 2013, 06:45 PM

yesss! that's it, thank you so much... was missing the

$chunk->_processed = false;

inside the loop...got me a performance boost of around 2 seconds for the same task!

Re: Help with getChunk and MODx speed please

Kristoffer — Apr 24, 2013, 08:14 AM

I managed to edit my test code to actually get it to work. The code below is working and the first two methods takes approximately 0,5 seconds to run on my local XAMPP-installation compared to the last one that takes 1,7 seconds.

initialize('web');
 
$modx->setLogTarget(XPDO_CLI_MODE ? 'ECHO' : 'HTML');
$modx->setLogLevel(xPDO::LOG_LEVEL_INFO);
 
$tstart = $modx->getMicroTime();
$chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
$chunk->setCacheable(false); // This row is needed
for ($i = 0; $i < 1000; $i++) {
    $chunk->_processed = false; // This row is needed
    echo $chunk->process(array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getObject — %2.4f s\n", $tend - $tstart));
 
$tstart = $modx->getMicroTime();
$chunk = $modx->getParser()->getElement('modChunk', 'testSpeed');
$chunk->setCacheable(false); // This row is needed
for ($i = 0; $i < 1000; $i++) {
    $chunk->_processed = false; // This row is needed
    echo $chunk->process(array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using modParser::getElement — %2.4f s\n", $tend - $tstart));
 
$tstart = $modx->getMicroTime();
for ($i = 0; $i < 1000; $i++) {
    echo $modx->getChunk('testSpeed', array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getChunk — %2.4f s\n", $tend - $tstart));
exit();

Re: Help with getChunk and MODx speed please

exside — Apr 10, 2013, 11:15 AM

I also just tried out ALL of the suggested alternatives to getChunk also with different things like seen commented out in the following code

$tpl = 'chunk.name';
$tpl = $this->modx->getObject('modChunk', array('name' => $tpl));
//$tpl = $this->modx->getParser()->getElement('modChunk', $tpl);
//$tpl->setCacheable(false);
//$tpl->cache_type = 1;
//$tpl->_processed = false;
//$this->modx->log(modX::LOG_LEVEL_ERROR, '[' . $this->config['packagename'] . '] Chunk Object ' . print_r($tpl->toArray(),1));
foreach ( $records as $record ) {
	// parse tpl chunk
	if ( $record->getOne('Song') ) {
		$song = $record->getOne('Song')->toArray();
		$record = $record->toArray();
		$data = array_merge($record, $song);
	} else {
		$record = $record->toArray();
		$data = $record;
	}
	
	// parse template chunk and return output, $tpl is a chunk name
	//$output .= $this->modx->getChunk($tpl, array(
	$output .= $tpl->process(array(
		'timestamp'			=> $data['timestamp'],
		'starttime'			=> $data['starttime'],
		'audioartist'		=> $data['artistname'],
		'audiotitle'		=> $data['songtitle']
	));
}

I cannot use Bob's suggested str_replace version because I need output modifiers inside the chunk to be processed.

the scenario is that I have to fetch a large list of songs, think of a 24 hour playlist, around 500 items, and make html out of it...with getChunk this takes loooong (several seconds), so I'm just interested if there was another option, found this thread and sadly it doesn't work, I just get the data of the first fetched record in ALL the playlist items...it's not very urgent or so because I'm caching the generated html anyways (until the next song comes), so the user is not realizing that it takes seconds to generate that playlist from the database, but it kinda bothers me that I'm not able to make use of the nice techniques shown earlier in this thread. Was there maybe something changed in the core or am I just doing it wrong?

Re: Help with getChunk and MODx speed please

ellipsis_89 — Oct 15, 2012, 07:32 AM

Where exactly did you add it? Cause $myChunkAsObject is your chunk variable, the provided code does not seem to fix the same value problem.

The following code outputs the same values for each item:

$chunkie = $modx->getObject('modChunk', array('name' => 'thumbTemplate'));
foreach ($items as $item) {

    $itemArray = $item->toArray();
    $itemArray['idx'] = $idx;

(...)

$output .= $chunkie->process($itemArray);
$idx++;
};

Re: Help with getChunk and MODx speed please

yurkobb — Jun 24, 2012, 02:57 PM

Quote from: simonp98 at Mar 01, 2012, 09:23 PM

What appears to be happening is that the tag is not changing with each new property value and so the result is that the chunk is always retrieved from the source cache.

Hi, after adding

  $myChunkAsObject->_processed = false;

to the loop with the process() call, it worked, and it was MUCH faster than getChunk(). Thank you for raising the issue, btw, don't know how I'd find what's the problem without your tests.

Re: Help with getChunk and MODx speed please

Kristoffer — Jun 08, 2012, 11:31 AM

I'm trying to use the techniques described here to loop over a chunk but can't get it to work properly. Is it possible to output dynamic content via the process method? If I change OpenGeeks benchmark script to echo the results from each iteration, I get 1000 copies of the first record for getObject and modParser::getElement but getChunk works as expected. Here is the code I', using:

initialize('web');

$modx->setLogTarget(XPDO_CLI_MODE ? 'ECHO' : 'HTML');
$modx->setLogLevel(xPDO::LOG_LEVEL_INFO);

$tstart = $modx->getMicroTime();
$chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
$chunk->setCacheable(false);
for ($i = 0; $i < 1000; $i++) {
    echo $chunk->process(array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getObject — %2.4f s\n", $tend - $tstart));

$tstart = $modx->getMicroTime();
$chunk = $modx->getParser()->getElement('modChunk', 'testSpeed');
for ($i = 0; $i < 1000; $i++) {
    echo $chunk->process(array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using modParser::getElement — %2.4f s\n", $tend - $tstart));

$tstart = $modx->getMicroTime();
for ($i = 0; $i < 1000; $i++) {
    echo $modx->getChunk('testSpeed', array('i' => $i));
}
$tend = $modx->getMicroTime();
$modx->log(modX::LOG_LEVEL_INFO, sprintf("Processed 1000 chunks using getChunk — %2.4f s\n", $tend - $tstart));
exit();

Re: Help with getChunk and MODx speed please

simonp98 — Mar 02, 2012, 02:39 AM

It's more of a theoretical thing BobRay. I was trying to understand the methods illustrated by Opengeek in the context of my original issue. This was solved with caching and the site flies, but in an effort to improve my understanding of MODx I wanted to explore these other options and have come unstuck at the first!!

Re: Help with getChunk and MODx speed please

BobRay — Mar 01, 2012, 04:53 PM

Maybe I'm mistunderstanding you, but why do you need a chunk at all? Why not just return the appropriate output from the snippet and use a snippet tag rather than a chunk tag.

---------------------------------------------------------------------------------------------------------------
PLEASE, PLEASE specify the version of MODX you are using . . . PLEASE!
MODX info for everyone: http://bobsguides.com/modx.html

Re: Help with getChunk and MODx speed please

simonp98 — Mar 01, 2012, 03:23 PM

Thank you for your patience. I think I might not be explaining myself very well so I have put together the piece of code that might illustrate the difficulty I am having. My chunk looks like this

[[+status]]

$chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
$chunk->setCacheable(false);
for ($i = 0; $i < $y; $i++) {
	echo "Pass - $i ";
	echo '   Printing chunk output ... ' .$chunk->process(array('status'=>$i));
	echo '   Printing chunk content ... ' . $chunk->getContent();
	echo '   Printing properties ... '; print_r($chunk->_properties);
	echo '   Printing tag ... ' . $chunk->_tag;
	echo '
';
}

This is what I think I'm doing ...
1. retrieving my 'testSpeed' chunk from the DB once.
2. processing my chunk in a loop, each time passing a new value for the 'status' property that increments in the loop.

What appears to be happening is that the tag is not changing with each new property value and so the result is that the chunk is always retrieved from the source cache.

Here is the result of my code

Pass - 0 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 0 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 1 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 1 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 2 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 2 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 3 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 3 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 4 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 4 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 5 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 5 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 6 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 6 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 7 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 7 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 8 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 8 ) Printing tag ... [[$testSpeed?status=`0`]]
Pass - 9 Printing chunk output ... 0 Printing chunk content ... [[+status]] Printing properties ... Array ( [status] => 9 ) Printing tag ... [[$testSpeed?status=`0`]]

Re: Help with getChunk and MODx speed please

opengeek — Mar 01, 2012, 12:36 PM

Quote from: simonp98 at Mar 01, 2012, 05:02 PM

Is it that I can't and shouldn't be trying to use the methods you demonstrated in the way I am trying to is it just that I'm doing it wrong?

I think you're missing the properties parameter of the process() method:

$chunk = $modx->getObject('modChunk', array('name' => 'testSpeed'));
$chunk->setCacheable(false);
for ($i = 0; $i < $y; $i++) {
	$chunk->process(array('i' => $i));
}

If I alter the test to pass in unique properties on each iteration, you can see now the additional processing time for not pulling the already processed output from the elementCache (which is where output is cached, vs. the source of the elements which is in the sourceCache).

My tests now result in:

[2012-03-01 11:31:15] (INFO) Processed 1000 chunks using getObject — 0.3535 s

[2012-03-01 11:31:16] (INFO) Processed 1000 chunks using modParser::getElement — 0.2690 s

[2012-03-01 11:31:18] (INFO) Processed 1000 chunks using getChunk — 2.8828 s

[2012-03-01 11:31:19] (INFO) Processed 1000 iterations on included content using modParser::processElementTags — 0.3932 s

Here is the current benchmark code in a gist...
https://gist.github.com/1952005