2 Pages, same snippet & template, 1 works, 1 doesn't

Got a puzzle here. We have two pages using the same snippet. They also use the same template but are in different language contexts using Babel. The pages are identical except for different chunks of language being called into the template depending on context.
When a page is saved a snippet is fired, looping through the User database firing out emails to users, matching on-page criteria to user criteria. One context works fine, sends the emails.
The other fires, starts looping through but then returns:



Does anyone know what that might signify?

A snippet, or a plugin? You probably need to switch contexts to correspond with the context the resource is in.

This error happens when an script takes too long to execute against fcgi timeouts ( https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html )

If you have a lot of emails to send, you should use a service like MailChimp, Mailgun, or Mandrill.

Mailgun is free and there is a MailgunX class in the Notify package that makes it fairly easy to use Mailgun to send email to a list of users as you loop through them (see the Notify class and properties).

The send will go much faster, your host will appreciate it (since it bypasses the host completely), delivery rates will be higher, and you'll have a nice log to look at that can indicate deliveries, opens, and clicks.

Thanks guys.
So presumably, the snippet run from one page is finding more matches, thus sending more mails and timing out?

We did use Mailgun, but not in this case as many emails were getting caught in spam filters due to the Mailgun IP origin. This was set up in the global Modx 'SMTP' system settings. It's now set to another provider though, so mails aren't being sent by the server I don't think. Resource Watcher was used and any pages published with a specific parent trigger the snippet, which ends up looping through a $modx->getService('mail', 'mail.modPHPMailer'); for each match pulling an email Tpl for the message body. Presumably there's a better way...

Well, after increasing the php.ini 'Memory_Limit', this script is working again in both contexts. But with users increasing, no doubt we'll reach a limit again.

The snippet mentioned above which uses the mailer is actually searching through and matching user information currently held in Extended Fields. We know NOW that extending the ModUSer would be the way to do this, but back then, we didn't and we were new to Profile / Register plug ins Is it still possible to extend ModUSer retrospectively and rewrite a script to pull out the info in extended fields and write to the new fields? And does anyone have experience in adapting Proflie plug in to write to extended ModUser fields?

Presumably we can then do a quicker search through user data.

IIRC, ClassExtender has snippets you can add to the Register and UpdateProfile pages to write the data to the new fields. You may need to modify them to meet your needs.

You could definitely write a utility snippet to copy the data from the extended fields to the new fields.

Once you've done that, you should be able to use the getExtUsers snippet to get your users in a single query rather than looping through all of them. OTOH, it's possible to get the users with a single query now, though it's fairly complicated.

That said, I suspect that you're hitting the timeout because sending the emails takes so long rather than processing the users. That's why I started using MailGun and Mandrill -- then the time gets spent at their server rather than yours. Using ClassExtender should help with the memory limit, if you're hitting that, but it may not solve your timeout problem. It's not that difficult, you just include the appropriate class file (e.g., mailgunX), and send it the Message, and call its addUser() function for each user. Every 25 users or so, you call the function to send the messages and the one that clears the users. It takes a lot less time than sending them individually on your server.

One thing that might work with your current setup would be to call set_time_limit(0) after every nth user (try 25) by setting a counter and resetting it when you call set_time_limit(). You can also get your users in batches by using an offset in your query if memory is a problem.

Beside of converting your Data you could also reduce the memory of your Application.

For Example using $modx->getCollection() creates with lots of Customers lots and lots of Megabyte Memory Usage, while $modx->getIterator() does the same for most Usages, using only an minimum of Memory.
Because getCollection generates xPDO Objects for every Single Data-Row at once, while getIterator creates them row by row freeing up memory.

Also using $modx->getChunk() inside big foreach Data sets blow up your memory, instead you could only load it once, and reuse the object for every row. An Memory Optimized Example would look like this: