I marked this as a discussion rather than question because I'm not sure there's a solitary 'right' answer, but I've run into something I'm wondering how others in the community have or would handle. Keep in mind the data being fed into MODX will remain read only, meaning MODX consumes the data but does not edit, create, update or remove it.
I'm starting working on abstracting how I hooked Shopify and MODX together for an e-commerce solution into a MODX extra. The idea is that every so often, MODX hits the authenticated Shopify API which returns XML. What this addon will do is parse that XML and store it in custom tables using a composite relationship (one to many) between Products and Product Variants.
I have two main ideas for how to handle this:
- Recursive Sync
- Torch and Re-Import
Recursive sync being for each item, check if it already exists and if so update it otherwise add a new one. Pretty standard, but also could be processor intensive with thousands of items.
My other idea is simpler. Since this is one-way read only, rather than doing a bunch of fancy smancy processing per item to check if it exists or not, just delete everything, then re add. Way less recursion and looping this way, but is it "bad"? I was thinking I'd need to set MySQL AUTO_INCREMENT back to 1 so indexes don't get way out of control.
If I went with the Torch and Re-Import approach the auto incrementing ids of the rows would be arbitrary and not used, which is fine because everything is keyed off Shopify product ids.
