getCollection white screen 'o death

If I remember correctly, I've reported this problem before...

I've currently got a database with roughly 11000 rows and I get a white screen of death when using collection. Echoing before and and after hand doesn't show anything so my guess is it cleans the entire page with ob_clean or something... Anyway, I solved the problem with using the getIterator, but again this should not be happening.



ucsAssortment has 9 rows of which 6 are IDs and I've got indexes on all of those ID rows. Finally an active, createdon and editedon row, plus an ID on itself since the table is an xPDOSimpleObject.

So, you need to replace getIterator with getCollection and then it totally fails... How come?

I've cleaned the cache, renamed the cache and created a new folder, no errors in the log in both cases, completely removed and replaced the modx/model and xpdo directory and still nothing... This is pretty problematic.

Hi,

In the past I've had an issue with getCollection if the primary key (PK) of the target table is not included in the select list. Adding the primary key is probably not possible if you're using DISTINCT (although I think MySQL might allow it - try it?), so if you can, sticking with getIterator might be best.

Explanation:

At least in the version of xPDO I'm using in MODX 2.1.3, _loadCollectionInstance uses the PK for the cache key, and trying to get the PK when the original select() doesn't include it triggers an infinite recursion, which will hit a PHP limit and cause the WSOD.

getIterator() eventually uses _loadInstance which doesn't use the PK, and so doesn't cause the error.

Hope this helps...

Ignore this: "(although I think MySQL might allow it - try it?)"
I was thinking of a GROUP BY.

Thanks for the speedy reply mate, I get what you're saying, but I find it hard to find the logic behind it.

What I had, and note that this worked fine before hitting the 10.000 barrier, is the following:


This didn't work even when I depleted the table down to 50 rows or when my query even matched about 50 rows (I'd set all active rows to 0 and then update 50 rows to active 1). Both cases gave me a WSOD...

While the problem is solved and your fix did the trick, can you explain the problem I had in the first place?

Not sure why 10.000 rows would give you a problem, or why deleting back down to 50 wouldn't fix it - unless something else about the table, schema or query had changed.

If you can, enable the php error_log, the reason for the WSOD should appear in there.

Hmmm, the problem has been fixed but I'm gonna get into the mess of it Friday or Monday, so I'll give that baby a try. Fact is, getCollection didn't work with the specific query, but getObject (thus 1, the first result) worked perfectly... Maybe Jason has an idea when he has time to look... ;-)

Thanks again Martin!

You can't load that many rows into memory at once, period. You have x amount of memory and more data than memory. Why would you want to load all of that data into memory at once? Iterating over the result set one row at a time is the only way to work with that much data, and even then the amount of processing time will likely be too much for a typical web request.

I didn't realise that it got pushed into the memory I guess. ;-) That's the mistake I made. I now also realise that VisionCart has that same limitation.

I just used getIterator, but unfortunately you can't get the "count" from the array size. I reckon you'd just have to use one single xPDOQuery object with both getCount and getIterator? Tx guys, epic support!

Also, make sure you are select()'ing at a minimum, the id of the object you are querying. You must select at least the primary key of an object to load it as an xPDOObject instance. In addition, if you try to access any fields of the object you have not select()'ed in the query, additional queries will be fired off to get the values of those columns; this is called lazy-loading, and can cause some serious performance issues if you need access to that data.

You can indeed use the xPDOQuery (before you add the select() and DISTINCT to it) with getCount() before re-using it in the actual query to get the result set.

Alright, excellent advice and I'll keep in mind that I bascially always need the primary key (with xPDOSimpleObject being `id`). Tx Jason.