Dealing with Big data in MODx / GetResources and other listing snippets

I've got a client that wants to list all their jobs on a Google map. I used gmaps (new one) which appears to be based on getResources
https://github.com/craftsmancoding/gmaps/wiki/Gmarker-Snippet

It works great with 10 items... fine with 20 or 30 .. 100 starts to get slow loading non cached... the full set is 1000 or so.. and is murdering our server.

Is there a way to "chunk" these? I get an error 500 ... presumably because the hamsters died of heart failure.

Anyway .. looking for ideas of dealing with these large data sets.

I'm not sure that 1,000 records qualifies as "Big Data."

That said, The first thing is to try caching everything you can (pages, snippets, chunks, etc.).

Have you tried &suppressLookup=`1` in the snippet tag? If the locations are stored, that might speed things up a lot. It looks like you'd have to save any resource created before the package was installed in order to set the coordinate TVs.

The fact is, though, that using getResources with TVs involved can be very slow with large data sets. If the above stuff doesn't do it, you're probably going to want to go to a custom DB table where each row contains the information about a single job (including its coordinates) and have a single resource that will display any job, rather than a different resource for each job.

A custom snippet (using the classes from one of the mapping extras) should then be able to produce the map fairly quickly.