We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 10378
    • 375 Posts
    Hello and Merry Christmas!

    I hope someone is watching the forums these days ... smiley

    I've written a CMP to import users into the MODX user table. The ExtJS part of the importer implements a modx-console.
    The PHP part consists of a class based processor + a separate class which handles all the CSV import voodoo.

    The import process works like a charm.
    The only problem is, that the modx-console displays the log messages of the processor + importer class in random order.

    I have searched the web for any hints but without success.

    This is the ExtJS part which initializes the modx-console:

    var topic = '/goodnewsimport/';
    var register = 'mgr';
            ...
            ...
            ...
            if (this.console == null || this.console == undefined) {
                this.console = MODx.load({
                    xtype: 'modx-console'
                    ,register: register
                    ,topic: topic
                    ,show_filename: 0
                    ,listeners: {
                        'shutdown': {fn:function() {
                            //reload CMP to reset fields
                            //location.href = MODx.config.manager_url + '?a=' + MODx.request.a + '&action=import';
                        }
                        ,scope:this}
                    }
                });
            } else {
                this.console.setRegister(register,topic);
            }
            this.console.show(Ext.getBody());
            ...
            ...
            ...
            this.getForm().submit({
                url: GoodNews.config.connectorUrl
                ,params: {
                    action: 'mgr/subscribers/import'
                    ,register: register
                    ,topic: topic
                }
                ,success: function(form,action){
                    if(action.result.success){
                        this.console.fireEvent('complete');
                    }
                }
                ,failure: function(result,request) {
                    this.console.fireEvent('complete');
                }
                ,scope: this
            });
    
    


    As far as I understand, the processor handles the log output automatically (writes the messages to the file based modRegistry) if its called with a register and topic parameter? The RTFM isn't very helpful with this...

    This question has been answered by markh. See the first response.

      Freelancer @bitego http://www.bitego.com
      ---
      GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
      More infos here: http://www.bitego.com/extras/goodnews/
      • 22303 MODX Staff
      • 10,725 Posts
      Are you certain the order is "random"? It could be that some messages are being delayed until the end of a process and are not being written in the "expected" order... Or perhaps you are sending messages with keys that define the order, overriding normal timestamps + sequences that would be given to the messages?
        • 10378
        • 375 Posts
        Quote from: opengeek at Dec 26, 2013, 07:45 AM
        Are you certain the order is "random"? It could be that some messages are being delayed until the end of a process and are not being written in the "expected" order... Or perhaps you are sending messages with keys that define the order, overriding normal timestamps + sequences that would be given to the messages?

        Hi Jason,

        no, I'm not certain the order is random.
        I have no clue why the messages could be delayed. I'm simply writing modx log messages within my processor and within the included class.

        Here is a sample part of the process() method of the processor class:
        (please read the comments in the lower part of the code snippet)

        	public function process() {
        
                $error = false;
                
                $this->modx->log(modX::LOG_LEVEL_INFO, $this->modx->lexicon('goodnews.import_subscribers_log_prep_csv_import'));
                sleep(1);
        
                if (!($this->goodnewsimportsubscribers instanceof GoodNewsImportSubscribers)) {
                    $this->modx->log(modX::LOG_LEVEL_FATAL, $this->modx->lexicon('goodnews.import_subscribers_log_no_class'));
                    $error = true;
                }
        
                // Make sure a csv file was specified ($csvfile is an array!)
                $csvfile = $this->getProperty('csvfile');
                if (empty($csvfile['name'])) {
                    $this->addFieldError('csvfile', $this->modx->lexicon('goodnews.import_subscribers_log_ns_csvfile'));
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_log_ns_csvfile'));
                    sleep(1);
                } elseif (!$this->goodnewsimportsubscribers->csvMimeType($csvfile['type'])) {
                    $this->addFieldError('csvfile', $this->modx->lexicon('goodnews.import_subscribers_log_wrong_filetype'));
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_log_wrong_filetype'));
                    sleep(1);
                }
        
                // Make sure a batchsize was specified
                $batchsize = $this->getProperty('batchsize');
                if (empty($batchsize) && !is_numeric($batchsize)) {
                    $this->addFieldError('batchsize', $this->modx->lexicon('goodnews.import_subscribers_log_ns_batchsize'));
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_log_ns_batchsize'));
                    sleep(1);
                }
                
                ...
                ...
                ...
        
                // Only continue with processing if no errors occured
                if ($error || $this->hasErrors()) {
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_log_failed'));
                    sleep(1);
                    unset($this->goodnewsimportsubscribers);
                    return $this->failure();
                }
        
                $this->modx->log(modX::LOG_LEVEL_INFO, $this->modx->lexicon('goodnews.import_subscribers_log_importing_csv').' '.$csvfile['name']);
                sleep(1);
        
                // Initialize the GoodNewsImportSubscribers object
                if ($this->goodnewsimportsubscribers->init($csvfile['tmp_name'], $delimiter, $enclosure) == false) {
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_err_open_csvfile'));
                    sleep(1);
                    $error = true;
                }
                
                // Only continue with processing if no errors occured
                if ($error) {
                    unset($this->goodnewsimportsubscribers);
                    $this->modx->log(modX::LOG_LEVEL_ERROR, $this->modx->lexicon('goodnews.import_subscribers_log_failed'));
                    sleep(1);
                    return $this->failure();
                }
                
                // The importUsers method of the GoodNewsImportSubscribers class holds the 
                // foreach loop, which writes valid users to the database
                //    + it also sends a proper log message for each imported (or not imported) user
                $count = $this->goodnewsimportsubscribers->importUsers($batchsize, $groups, $categories);
                unset($this->goodnewsimportsubscribers);
        
                // Especially this last log message almost never is shown in the MODExt console window!
                // Instead it is written to the console as the first message when I start a complete new import.
                $this->modx->log(modX::LOG_LEVEL_INFO, $this->modx->lexicon('goodnews.import_subscribers_log_finished').$count);
                sleep(1);
                return $this->success();
        
        	}
        
        


        Do modx->log messages generate messages with keys?

        Sorry Jason, I'm a complete noob on using modRegistry...
          Freelancer @bitego http://www.bitego.com
          ---
          GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
          More infos here: http://www.bitego.com/extras/goodnews/
          • 10378
          • 375 Posts
          By the way - this is the content of a single .msg.php file which always remains in the /core/cache/registry/mgr/goodnewsimport directory. It only gets a new timestamp. It's from the last log message from my processor code above.
          As far as I understand it doesn't get removed because it mostly never gets displayed in the console window.

          <?php
          return array (
            'timestamp' => '2013-12-26 14:55:11',
            'level' => 'INFO',
            'msg' => 'Import finished. Successfully imported subscribers: 0',
            'def' => '',
            'file' => '/modxdev/goodnews/assets/components/goodnews/connector.php',
            'line' => '',
          );
          
          [ed. note: gadgetto last edited this post 12 years, 9 months ago.]
            Freelancer @bitego http://www.bitego.com
            ---
            GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
            More infos here: http://www.bitego.com/extras/goodnews/
            • 22303 MODX Staff
            • 10,725 Posts
            What is the filename of the message there in the registry?
              • 10378
              • 375 Posts
              Quote from: opengeek at Dec 26, 2013, 09:14 AM
              What is the filename of the message there in the registry?

              The filename changes with each try.
              Currently it is: 1388071047-000023.msg.php

              Content:

              <?php
              return array (
                'timestamp' => '2013-12-26 16:17:27',
                'level' => 'INFO',
                'msg' => 'Import finished. Successfully imported subscribers: 0',
                'def' => '',
                'file' => '/modxdev/goodnews/assets/components/goodnews/connector.php',
                'line' => '',
              );
              


              I have increased the sleep time after the last log message in the process method (line 72 of the above code) to 2 seconds and now it seems to work!

              If I leave it at 1 second it sometimes works. If I remove the sleep completely, the console output goes crazy and many .msg.php messages remain in the cache/registry/... folder.

              Seems to be a timing issue!?
                Freelancer @bitego http://www.bitego.com
                ---
                GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
                More infos here: http://www.bitego.com/extras/goodnews/
                • 22303 MODX Staff
                • 10,725 Posts
                Are you only retrieving a single message? I'm not sure what is happening without being able to run the code...

                I assume you have a different action checking status messages from the one that is running the import?
                  • 10378
                  • 375 Posts
                  Quote from: opengeek at Dec 26, 2013, 09:32 AM
                  Are you only retrieving a single message? I'm not sure what is happening without being able to run the code...

                  I assume you have a different action checking status messages from the one that is running the import?

                  In the ExtJS part of the CMP - if the user clicks the "Start Import" button, a MODExt console is initialized which polls the registry. Immediately after that the form is submitted (CSV file upload included) and the processor starts his work (you can see the ExtJS code part in my first post). The processor writes modx->log messages to the registry which is polled by the console. Thats all.

                  The processor itself includes a special class which handles the CSV import part. This class creates users in the MODX database and the same method which creates the users also writes the modx->log messages to the registry.

                  No different action.
                    Freelancer @bitego http://www.bitego.com
                    ---
                    GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
                    More infos here: http://www.bitego.com/extras/goodnews/
                  • discuss.answer
                    • 18373 ☆ A M B ☆
                    • 3,141 Posts
                    Quote from: gadgetto at Dec 26, 2013, 09:25 AM

                    ...
                    I have increased the sleep time after the last log message in the process method (line 72 of the above code) to 2 seconds and now it seems to work!

                    If I leave it at 1 second it sometimes works. If I remove the sleep completely, the console output goes crazy and many .msg.php messages remain in the cache/registry/... folder.

                    Seems to be a timing issue!?

                    The ExtJS console polls every second, however the polling is stopped when the processor returns a value.. ie return $this->success(); in your processor. If you have a one second sleep call, that means it can be stopped before the final log message is loaded.. which is then loaded first the next time the console is opened if the register/topic is the same. With a 2 second delay, it has more chance of loading all the items from the registry.
                      Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

                      Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
                      • 10378
                      • 375 Posts
                      Already thought it must be something like this! Thanks for clearing up! Also thank you Jason for your help!
                      So it will be always good to put a proper delay before sending success or failure.
                        Freelancer @bitego http://www.bitego.com
                        ---
                        GoodNews - one of the most advanced and integrated Group Mailer premium add-ons for MODX Revolution!
                        More infos here: http://www.bitego.com/extras/goodnews/