Out of memory when creating too many objects

I have written a snippet to successfully read a CSV file and add the contents to a custom table in the modx revo database. Looping through every line in the file (~5000 lines), I create a new simpleobject each time and fill in the fields then do an object save. This works very well, but it only built about half the table before it ran out of memory with a 32M limit. I upped the limit to 64M and still got the out of memory error, but luckily, all the data was in the table. Isn’t going over the limit the result of creating 5000 objects without destructing any of them? I couldn’t find any reference to a destructor in the documentation or looking through the XPDO code. Is there any way to release the memory from an object before exiting the script? I am assuming at some point, the memory release is taken care of by the object or some other mechanism. I tried creating the object only once and then reusing it with new values, but (as I expected), the object save just rewrote over the same record (or row) each time. I am sure there is a better way to do this, but I was more or less following the documentation on custom tables in 3PC’s.

Quote from: GeneC at Aug 02, 2010, 04:14 PM

I have written a snippet to successfully read a CSV file and add the contents to a custom table in the modx revo database. Looping through every line in the file (~5000 lines), I create a new simpleobject each time and fill in the fields then do an object save. This works very well, but it only built about half the table before it ran out of memory with a 32M limit. I upped the limit to 64M and still got the out of memory error, but luckily, all the data was in the table. Isn’t going over the limit the result of creating 5000 objects without destructing any of them? I couldn’t find any reference to a destructor in the documentation or looking through the XPDO code. Is there any way to release the memory from an object before exiting the script? I am assuming at some point, the memory release is taken care of by the object or some other mechanism. I tried creating the object only once and then reusing it with new values, but (as I expected), the object save just rewrote over the same record (or row) each time. I am sure there is a better way to do this, but I was more or less following the documentation on custom tables in 3PC’s.

Objects are not intended for use in mass like that. You need to deal with one row and object at a time TBH. To "destruct" an instance, just unset() it.

Thanks. Just couldn’t find the terminology last night. I will rewrite now, but don’t have to do this again for several months. I appreciate the rapid response.

I have a similar problem. Once I update the 1500 facilities.

code:


If removed $res->save();, it works.