We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 28042 ☆ A M B ☆
    • 24,524 Posts
    I'm working on an SFTP media source driver. A couple of questions have come up.

    What would be more efficient for generating a file tree, opendir() and readdir(), or scandir()?

    What about having the ability to use public key authentication? Would each user have to generate and install his own keys, or does the Manager's server provide the public/private key pair for accessing the remote source - i.e. does the MODX installation whose Manager the user is logged into act as the client? Like the rsync between Clouds https://modxcloud.com/userguide/using-modx-cloud/snapshots/working-with-large-snapshots.html

    It took some doing, but I finally got ssh2 enabled on my MAMP environment, and have gotten to the point of connecting to a MODX Cloud account, returning the server fingerprint for validation, logging in by either password or public key, and listing files in a given directory.
    Next step, make a media source out of it.

    http://www.php.net/manual/en/book.ssh2.php
      Studying MODX in the desert - http://sottwell.com
      Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
      Join the Slack Community - http://modx.org
      • 28042 ☆ A M B ☆
      • 24,524 Posts
      Ok, now I have some even bigger questions. I've looked at several media source type drivers, and don't see any consistent way to initialize a driver or deal with the source's properties. Some have __construct(xPDO &$xpdo), where the lexicon is loaded, some don't have that function at all and load the lexicon and have getService(...) in an initialize() function. Some use the getDefaultProperties() array, some don't.

      I'd like to at least get the type visible, and be able to add a new Media Source using this type with the fields for setting its properties, even if it doesn't actually do anything yet.

      1. What is the "approved" way to construct and initialize a driver?
      2. What is the "approved" way to handle the properties?
      3. How does it get registered into the system so that it at least shows up in the Types tab?


      Is there any documentation on this, or is the only way to figure this out is by reading other people's code?

      I have what appears to be the proper directory and file structure for a media source in core/components/sftpsource, including the lexicon, and am working with the core/components/sftpsource/model/sftpsource/sftpsource.class.php file, which class extends modMediaSource and implements modMediaSourceInterface. This is the point where I find so many variations.
        Studying MODX in the desert - http://sottwell.com
        Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
        Join the Slack Community - http://modx.org
        • 28042 ☆ A M B ☆
        • 24,524 Posts
        After further thought, I've managed to answer a couple of my own questions.

        The Gallery media source type is atypical, since it's not intended for users to create the media sources. It's all handled automatically via the Gallery CMP. So the type doesn't use the default properties array, and it has to use getService() to get access to the Gallery CMP settings for each Gallery-type source.

        For the ordinary user-created sources, the getDefaultProperties() array is needed to establish the defaults, and it won't have a service to get.

        As far as installing the type, I realized that I'll need to make a package out of it to install it properly. MyComponent should do the trick.
          Studying MODX in the desert - http://sottwell.com
          Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
          Join the Slack Community - http://modx.org
          • 18373 ☆ A M B ☆
          • 3,141 Posts
          In terms of installing the media source, I don't think you need an actual package but you need to add it to the extension packages. In 2.2 this is a system setting, in 2.3 this is a table. Either way, simply call $modx->addExtensionPackage('name_of_class', $pathToClass);

          MODX will look at modMediaSource derivatives and make that available.
            Mark Hamstra • Developer spending his days working on Premium Extras and a MODX Site Dashboard with the ability to remotely upgrade MODX and extras to make the MODX world a little better.

            Tweet me @mark_hamstra, check my infrequent blog at markhamstra.com, my slightly more frequent ramblings at MODX.today or see code at Github.
            • 28042 ☆ A M B ☆
            • 24,524 Posts
            Hm. Added it to the system setting extension_packages, but still no sign of it in the Source Types tab. So there has to be a way of installing or registering a new source type.
              Studying MODX in the desert - http://sottwell.com
              Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
              Join the Slack Community - http://modx.org
              • 40045
              • 534 Posts
              What would be more efficient for generating a file tree, opendir() and readdir(), or scandir()?

              maybe this =): https://github.com/modxcms/revolution/pull/11295
                • 28042 ☆ A M B ☆
                • 24,524 Posts
                Well, it might if it works with remote filesystems accessed by a media source. And if it's using the most efficient method for accessing and listing the files from said remote filesystem. That's one problem with using an API function, you're totally dependent on the API developer's choice of PHP functions.

                It looks like I have a rather large selection of file-listing functions

                opendir() - readdir()
                scandir()
                glob()

                and a bunch of iterator classes. RecursiveTreeIterator sounds fascinating, but how efficient is it?
                  Studying MODX in the desert - http://sottwell.com
                  Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
                  Join the Slack Community - http://modx.org
                  • 28042 ☆ A M B ☆
                  • 24,524 Posts
                  Ah. glob() only works on local filesystems.
                    Studying MODX in the desert - http://sottwell.com
                    Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
                    Join the Slack Community - http://modx.org
                    • 28042 ☆ A M B ☆
                    • 24,524 Posts
                    And I still can't figure out how to get my media source type registered or installed or whatever needs to be done to get it listed in the types tab or the type dropdown when creating a media source.
                      Studying MODX in the desert - http://sottwell.com
                      Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
                      Join the Slack Community - http://modx.org
                      • 28042 ☆ A M B ☆
                      • 24,524 Posts
                      Ok, with help from Donald Atkinson, I got my sftp driver set up. Now I just need it to do stuff with the files it finds. I've based it heavily on evd's MODX-Swift OpenStack Object Storage driver.

                      Three things needed to get it to register in the Types tab and as a driver to select when creating a new Media Source.

                      1. Use the $modx->addExtensionPackage() API function or manually add it to the extension_packages setting in System Settings. In my case, I used
                      $modx->addExtensionPackage('sftpsource','[[++core_path]]components/sftpsource/model/');
                      

                      in a snippet to get
                      [{"sftpsource":{"path":"[[++core_path]]components/sftpsource/model/"}}]


                      2. Have these two functions in your class definition:
                      	/**
                      	* Get the name of this source type
                      	* @return string
                      	*/
                      	public function getTypeName() {
                      		$this->xpdo->lexicon->load('sftpsource:default');
                      		return $this->xpdo->lexicon('sftpsource');
                      	}
                      		
                      	/**
                      	* Get the description of this source type
                      	* @return string
                      	*/
                          public function getTypeDescription() {
                              $this->xpdo->lexicon->load('sftpsource:default');
                              return $this->xpdo->lexicon('sftpsource.description');
                          }
                      [ed. note: sottwell last edited this post 12 years, 4 months ago.]
                        Studying MODX in the desert - http://sottwell.com
                        Tips and Tricks from the MODX Forums and Slack Channels - http://modxcookbook.com
                        Join the Slack Community - http://modx.org