Handle 40+ fields with TV and MODX

I’m working on a project where.

1) Every night a CSV file will be uploaded to a folder, and then inserted into to the database. This will use cron and sits outside of MODX (mainly because I don’t know if cron can be run inside an instance of MODX).
2) The CSV file has 40 fields.
3) The table of data is 500,000 rows in depth and gets "renewed" every mid-night.

I’m wanting to use MODX to act as the go between, but I’m not sure if TVs are the way to go, or to use a separate table and use the following page/document structure as the go between;

ie:


Where [+page+] is placeholder used to find a relating page in the DB.

But here’s what I was thinking -- could I use TV to hold all 40 fields, or do you recommend using a third party table to hold the relevant fields and then running a snippet that looks at the CSV table?

It is a database of holiday offers.
I agree putting 40+ fields in the template variable is unwieldy, and I think making a snippet which calls a seperate table is the right way to do it.
I do not want to be creating half a million pages in MODX.

Because of the size/scope, my idea was:

1. Cron inserts into a table.
2. A user visits "someaddress.com/holiday/offer/$offer_id" and I grab the $offer_id and put it on the page.
3. A user does a search, which returns a paginated search result on "someaddress.com/holiday/search/".

So, essentially the table with a million rows sits outside of MODX, but I wanted to use 1 page to handle the offers.

I’ve seen it done before, its where you create a page with a placeholder phrase inside the page title.

Does this help?

With that many rows I’d certainly work with a external table.

You can make yourself a snippet that looks at the $_REQUEST array to determine which record(s) should be retrieved and additionally you could get fancy and make some rewrite rules which translate paths to request variables so that your urls are even more search engine friendly.

Ideally your snippet should use templates and placeholders to output the data. This will keep it much more manageable. AjaxSearch can search in external tables so you don’t have to forego on a powerful search function on on your site.

I’m sure you can do a solution where in the document manager you create a page where the placeholder is in the pageTitle.

I’m going to try it and see what happens.

AjaxSearch can search in external tables

Is there any documentation or examples of this?

I found some now:
http://modxcms.com/forums/index.php/topic,27316.0.html

Thanks

*closed*