How to export User/UserProfiles with rendered extendedFields?

10 Posts

dr0mfr34k Reply #1, 13 years, 3 months ago

Hey,

currently I'm working in a ModX Site for a Magazine. For the Users we have a member Page where they can download the newest issues and so on. At the moment we have over 25.000 User and need for Marketing Campaigns the User Data from time to time.

The problem is now on the one side we have 4 sites of this kind with such big user databases and i want to write a simple tool to export the user data with rendered extended fields(the JSON formatted stuff) to csv File. I try to achieve this with a script i found here but it seems that such a huge database is to much for the 128mb memory in my php configuration.

I also found a snippet which is for user export and wrote a bit with the developer and he also says for his snippet its to big database.

Is there anyone out there with a affordable solution to achieve this?

I'm stuck at this point :-(

Best regards
[ed. note: dr0mfr34k last edited this post 13 years, 3 months ago.]

81 Posts

Dunnock Reply #2, 13 years, 3 months ago

Do you use getResources or getIterator to fetch the results?

getIterator will be more thank likely bit slower, but does not consume memory even close to what getResources will. The syntax is same just the method name differs.

10 Posts

dr0mfr34k Reply #3, 13 years, 3 months ago

I just used the following code and was trying to modify it with limit to test where is the problem.

<?php
error_reporting(E_ALL);
require_once 'config.core.php';
require_once MODX_CORE_PATH.'model/modx/modx.class.php';
$modx = new modX();
$modx->initialize('web');

$c = $modx->newQuery('modUser');
$c->leftJoin('modUserProfile','Profile');

// add column names that u want to show
$c->select(array(
    'modUser.*',
   'Profile.fullname',
   'Profile.email',
   'Profile.blocked',
));

$c->limit(1, 10);

$users = $modx->getCollection("modUser",$c);

print_r($users);

/*for($i=0; $i<10; $i++){
    echo $users[$i]['username']; #->toArray();
    #echo "
#".$userArray['username']; #."==".$userArray['fullname'];
}*/

?>

81 Posts

Dunnock Reply #4, 13 years, 3 months ago

Don't print_r the users, those can be hefty objects with references to modx per object.

You can try next:

$users = $modx->getIterator('modUser', $c); <-- can handle pretty much unlimited amount of rows.
foreach($users as $user) {
   $out[] = $user->toArray();
}
echo "<pre>" . print_r($out, true);

10 Posts

dr0mfr34k Reply #5, 13 years, 3 months ago

Thank you very much dunnock, no it wokrs like it should!

But one question remains. I just take the data from the ModUserProfile Table and i just need some fields, but the sript doesn't take my Select query. Do you know why?

<?php
error_reporting(E_ALL);
require_once '../config.core.php';
require_once MODX_CORE_PATH . 'model/modx/modx.class.php';
$csvFile = 'user.csv'; // full path to csv file here
$modx = new modX();
$modx->initialize('web');

$c = $modx->newQuery('modUserProfile');

// add column names that u want to show
$c->select(array(
    'id',
    'fullname',
    'email',
    'city'
));

$fp = fopen($csvFile, 'w');

$users = $modx->getIterator('modUserProfile', $c);

foreach ($users as $user) {
    $user_array = $user->toArray();
    $complete_user_array = array_merge($user_array, $user_array['extended']);
    $out[] =  $user_array;

    fputcsv($fp, $out);

    echo "<pre>" . print_r($complete_user_array, true);
}

fclose($fp);

echo"<h2>Success!</h2>";


?>

5,888 Posts

Bruno17 Reply #6, 13 years, 3 months ago

toArray( string $keyPrefix = '', boolean $rawValues = false, boolean $excludeLazy = false, boolean | integer | string | array $includeRelated = false )

$keyPrefix string

An optional prefix to prepend to the field values.
$rawValues boolean

An optional flag indicating if you want the raw values instead of those returned by the {@link xPDOObject::get()} function.
$excludeLazy boolean

An option flag indicating if you want to exclude lazy fields from the resulting array; the default behavior is to include them which means the object will query the database for the lazy fields before providing the value.
$includeRelated boolean | integer | string | array

Describes if and how to include loaded related object fields. As an integer all loaded related objects in the graph up to that level of depth will be included. As a string, only loaded related objects matching the JSON graph representation will be included. As an array, only loaded related objects matching the graph array will be included. As boolean true, all currently loaded related objects will be included.

try:

$user->toArray('',false,true);

-------------------------------

you can buy me a beer, if you like MIGX

http://webcmsolutions.de/migx.html

Thanks!

16 Posts

Chris W Reply #7, 12 years, 11 months ago

Quote from: dr0mfr34k at Jul 01, 2013, 07:14 AM

I just used the following code and was trying to modify it with limit to test where is the problem.

<!--?php
error_reporting(E_ALL);
require_once 'config.core.php';
require_once MODX_CORE_PATH.'model/modx/modx.class.php';
$modx = new modX();
$modx--->initialize('web');

$c = $modx->newQuery('modUser');
$c->leftJoin('modUserProfile','Profile');

// add column names that u want to show
$c->select(array(
    'modUser.*',
   'Profile.fullname',
   'Profile.email',
   'Profile.blocked',
));

$c->limit(1, 10);

$users = $modx->getCollection("modUser",$c);

print_r($users);

/*for($i=0; $i<10; $i++){
    echo $users[$i]['username']; #->toArray();
    #echo "
#".$userArray['username']; #."==".$userArray['fullname'];
}*/

?>

Is this file/snippet being called anywhere on the website?