Data Import Problems - How to Troubleshoot?

3 Posts

tylera Reply #1, 15 years, 7 months ago

Hi All,

I’m new to MODx and I am in the process of migrating a site into MODx, Revo 2.0.8-pl. We are moving from a proprietary db/cms, so we have to import from an XML file. I’ve got the data importing directly into the site_content table via a custom script, and I’ve made sure to match all of the field values in the table, based on a resource that I created directly through the MODx manager. Other than just importing the resource, I am setting a parent value so that the resources will be placed into containers I created with the manager, and setting a template var value in the site_tmplvar_contentvalues table. I’ve turned off the tv part of the script for troubleshooting, but the problem described below still persists, and I’d like to know if setting tv values this way is a problem.

The newly imported resources show up in the doc tree, but when I go to edit one, it forces me out of the manager session, redirects to the manager login page, and then upon logging back in, doesn’t show the content of the resource. I’ve checked the table modx_site_content, and the content field indeed has content, but the manager/web don’t display the content data. The page title, parent, and alias values I’ve set display, but not the content.

I’m working with about 2K records/resources.

My initial questions are a few:

Is there a problem with my method of importing data, i.e. does MODx not allow such imports?
Is there a table that I need to update along with the site_content table, that tricks MODx into thinking that it created the resources?
Does MODx get angry (and force me to log back in) if the publishedon value is older than the superuser’s/MODx install creation date?

I’ll start with that, and hope that I’ve provided enough information to get some advice.

BTW, loving MODx thus far.

3 Posts

tylera Reply #2, 15 years, 6 months ago

Anyone have any ideas? I would really appreciate some sort of advice. Starting points instead of solutions are appreciated too.

Thanks.

☆ A M B ☆
181 Posts

jgulledge19 Reply #3, 15 years, 6 months ago

Are you using xPDO, the MODX way to access the DB? I wrote a script as a snippet earlier this week that creates pages on the fly with some default data and it also assigned a parent.

Here is the revalent code, that would be inside a loop:

<?php
$resource = $modx->newObject('modResource');
/* modx_site_content
 */
$alias = str_replace(' ', '-', preg_replace('/[^A-Za-z0-9_]/', '',$row['name'].' '.$row['lastname']) );
$faculty_profile = 
'<div>
HTML content goes here
</div>
'.$row['profile'];
$resource_data = array(
        'pagetitle' => $row['name'].' '.$row['lastname'],
        'longtitle' => $row['name'].' '.$row['lastname'],
        'description' => $row['name'].' '.$row['lastname'],//(firstname Lastname)
        'alias' => '',
        'published' => 1,
        'parent' => 302,// the profiles folder
        'content' => $faculty_profile,// (actual profile)
        'template' => 2, // default template
        'menuindex' => ++$count,// (order/count)
        'createdby' => 1,//user_id
        'createdon' => time(),// timestamp
        'menutitle' => '',// title
        'hidemenu' => 1,
        'uri' => 'profile/'.$alias// (profile/alias.html)
    );
// add in the profile stuff
$resource->fromArray($resource_data,'',true);
$resource->save();
?>

Also look here http://rtfm.modx.com/display/xPDO20/Home and http://www.devtrench.com/first-impressions-of-xpdo-wordpress-to-modx-migration-tool/.

MODX ebook | My Site

249 Posts

cyclissmo Reply #4, 15 years, 6 months ago

I’m not sure if I completely understand how you are importing data. My first thought would be, is your data migration script utilizing the MODx API, or are you pumping data directly into the database via a separate independent script? The MODx database schema is fairly complex, and importing data programatically should always be done via xPDO to enforce schema relationships. Even better is leveraging the system processors that are available for CRUD operations.

Why it is booting you when you try to edit a resource is puzzling. It could be cache issues, which you could try to resolve by clearing your MODx cache.

I would recommend starting over - import one document record and verify that it can be viewed and edited. Import in small batches, so you can potentially isolate problem records if suddenly something doesn’t work.

HTH

lo9on.com

MODx Evolution/Revolution | Remote Desktop Training | Development

3 Posts

tylera Reply #5, 15 years, 6 months ago

Thanks guys. I’ll try doing it through xPDO, but wanted to avoid it. I am importing through a separate independent script, created in Ruby. The files go straight into the database tables.

I did do incremental testing, and it worked, but the larger set caused the problems.

Thanks again.

Tyler

249 Posts

cyclissmo Reply #6, 15 years, 6 months ago

Be sure to chime back in if you need any assistance with xPDO or the API.

lo9on.com

MODx Evolution/Revolution | Remote Desktop Training | Development