We launched new forums in March 2019—join us there. In a hurry for help with your website? Get Help Now!
    • 22303 MODX Staff
    • 10,725 Posts
    History and Concepts

    MODx, as a fork of the Etomite CMS project, has been a wonderful tool that I’ve depended on in my daily business activities for over a year now. And as a core developer, I’ve become intimately familiar with the code. That’s a good thing, right? Well...being the ultra-geeky perfectionist I am, I wanted to see a properly implemented version of this CMS, complete with an extensible, object-oriented API, some more rapid-application development facilities, and the ability to use it with other database-platforms. But in the current codebase, these kinds of changes can be daunting. So, I decided to challenge myself and find a way to rewrite the MODx functionality from scratch, in a purely object-oriented fashion.

    Tattoo Prototypes

    No stranger to the concepts of object-relational management, being a J2EE-developer in my former career path, I went off looking for a way to abstract the database layer of MODx. That brought me into contact with EZPDO and Propel. I ultimately chose Propel for the job, and proceeded to reverse-engineer the db and generate an object model for MODx with Propel. To my surprise, two weeks later, working part-time between client tasks, I was able to run MODx using the new object model in PHP 5. That got me thinking, maybe we should branch off a PHP 5 only version of MODx using Propel as the engine, and the seed for Tattoo was planted.

    A month later, I completed a second prototype with a completely new data structure to handle multi-cultural content, content revisioning, and multi-sites/sub-sites (i.e. multiple contexts). But I was anything but pleased with the performance results. It worked well, but was at least twice as slow as the current MODx system; not at all what I wanted, or expected from taking advantage of the latest features in PHP 5.

    Enter PDO

    After taking a step back from the project and my initial discouragement, I started thinking about all the things that were making Propel so bulky and slow. At the same time, I started studying more about how PDO was being adopted as the standard database layer for PHP 5.1+. Then I found a PHP 4 implementation of PDO someone had hacked together and it all clicked in my head.

    Propel generated class variables and methods for every persistent column and foreign-key relationship defined in a schema, and depended on multiple classes to represent each persistent entity. Though this is nice, I don’t think it’s necessary, and I’ve come up with a very portable and consistent way to represent persistent entities in as little PHP code as necessary, using generic accessors and functions exclusively. This will power the core of MODx 1.0, and I wanted to give you a preview of how the new API looks and works.


    What 1.0 will mean to MODx developers

    Following are just some basic code samples of the 1.0 API and how it’s designed to approach certain problems. This is meant as a discussion point and since the implementation is not completed, I encourage as much feedback as possible.

    Defining persistent objects
    The following is the complete class definition for a modCategory (all classes will adopt a mod prefix to avoid namespace clashes with common class names), which represents a way of defining a hierarchical taxonomy, to “categorize” other data structures, such as web pages.

    <?php
    $this->ds->map['modCategory']['table']= 'categories';
    $this->ds->map['modCategory']['fields']= array (
       'name' => '',
       'description' => '',
       'parent' => 0,
    );
    $this->ds->map['modCategory']['fieldMeta']= array (
       'name' => array('dbtype' => 'TINYTEXT', 'phptype' => 'string', 'null' => false, 'index' => 'unique', ),
       'description' => array('dbtype' => 'MEDIUMTEXT', 'phptype' => 'string', ),
       'parent' => array('dbtype' => 'INTEGER', 'phptype' => 'integer', 'index' => 'fk', 'fkclass' => 'modCategory', 'fkcard' => '?', 'fktype' => 'self', ),
    );
    $this->ds->map['modcategory']= & $this->ds->map['modCategory'];
    
    /**
     * A generic category object for representing hierarchical taxonomies of topics,
     * tags, etc., or organizing other objects into complex taxonomies.
     */
    class modCategory extends modSimpleObject {
       function modCategory(& $modx) {
          $this->__construct($modx); 
       }
       function __construct(& $modx) {
          parent :: __construct($modx);
       }
    }
    ?>
    

    That’s it. You can now create, read, update, or delete (CRUD) this object from the database via the API. No SQL required, and you now have access to some amazingly powerful API functions to manage your object.

    Some explanations as we step through this first code example, starting with the schema definition represented in the map attribute of our data source:
    $this->ds->map

    This defines the table and persistent fields of your class, along with important meta data information that is used by the core to make your life easier, and automate everything from table creation, to building queries to load data by primary key(s), to building queries to get objects related by foreign keys.

    Note the reference made to the map in another map entry with a lowercase name:
    $this->ds->map['modcategory']= & $this->ds->map['modCategory'];

    This is a way to support some quirky differences between the way PHP 4 and 5 return values from the function:
    get_parent_class();

    PHP 4 returns all lower-case class names, PHP 5, in the original case the class was declared in.

    Next, let’s focus on the class definition. The first thing you’ll notice is:
    class modCategory extends modSimpleObject


    This means that modCategory inherits all the class attributes and functions from modSimpleObject, a basic, abstract class (meaning it doesn’t actually store data in a table, but is just a template for creating other classes with similar attributes by extending the class) defined in the MODx core. Here is the declaration for modSimpleObject from the core code:

    <?php
    $this->ds->map['modSimpleObject']['table']= null;
    $this->ds->map['modSimpleObject']['fields']= array (
       'id' => null,
       'deleted' => false,
    );
    $this->ds->map['modSimpleObject']['fieldsMeta']= array (
       'id' => array('dbtype' => 'INTEGER', 'phptype' => 'integer', 'null' => false, 'index' => 'pk', ),
       'deleted' => array('dbtype' => 'BINARY', 'phptype' => 'boolean', 'null' => false, ),
    );
    $this->ds->map['modsimpleobject']= & $this->ds->map['modSimpleObject'];
    
    /**
     * Extend this abstract class to define a class having an integer primary key
     * field and a simple deleted flag for managing the object in a type of
     * recycle bin.
     */
    class modSimpleObject extends modObject {
       function modSimpleObject(& $modx) {
          $this->__construct($modx); 
       }
       function __construct(& $modx) {
          parent :: __construct($modx);
       }
    }
    ?>

    So the modSimpleObject, the parent class of modCategory, adds some basic fields to our object definition; a primary key `id` field, and a `deleted` indicator. This in turn extends the base class of the 1.0 object model, modObject. The following functions are defined by modObject, and inherited by every persistent class defined as a direct or derived subclass of it:


    • addRelated(mixed, object) – adds an object instance in a foreign key relationship to this object
    • fromArray(array) – populates fields of this object from an associative array
    • get(string) – gets the value of a field by name, e.g. $obj->get(’content’)
    • getFieldName(string [, string]) – gets the full qualified column name of a field, using an optional table alias instead of the actual table name, if specified
    • getPK() - gets the name of an array of names of the primary key field(s) for this object
    • getPKType() - gets the type of the primary key field, or ’compound’ if an array of keys
    • getPrimaryKey() - gets the value of the primary key field(s) for this object
    • getRelatedObject(string) - gets an object related by a foreign key relationship with this object
    • getRelatedObjects(string [, object]) – gets a collection of related objects by foreign class name and using an optional criteria object
    • remove() - physically deletes the row representing this object from the database
    • save() - physically saves the object to a row in the database
    • set(string, mixed) - sets the value of a field to a specified value
    • toArray() - returns an associative array representation of the object fields
    So, our modCategory class would also have all of these functions available when getting an instance.

    Getting Persistent Objects from the DB with modCriteria
    Now, in order to retrieve an instance of a modCategory from the database, what do I do? Don’t I need to write a SQL query?

    No, you don’t need to write SQL, because the 1.0 MODx core has a whole framework of generic methods for working with any persistent object in a consistent way, with or without writing any SQL code, depending on what you need to do. This is where the modCriteria class comes into action.

    The class modCriteria encapsulates all the information contained in a typical SQL statement, and takes full advantage of PDO’s PreparedStatement support, allowing parameter bindings to be defined and passed to a statement that can be cached on the database server and used multiple times with various parameter values. Here is a typical scenario of using a modCriteria object, in this instance, looking up a modElement (aka snippet, chunk, TV, etc) by name:

    <?php
    $tbl= $modx->ds->getTableName('modElement');
    $crit= new modCriteria("SELECT * FROM {$tbl} WHERE name = :tagName AND (context_key IS NULL OR context_key = :contextKey)");
    $crit->stmt->bindParam(':tagName', $tagName);
    $crit->stmt->bindParam(':contextKey', $modx->context->get('context_key'));
    if ($element= $modx->getObject('modElement', $crit)) {
    ...
    ?>


    Getting objects by primary key is even easier; instead of a modCriteria instance passed to the modX :: getObject function, you simply pass the primary key value of the instance you want to retrieve (or an array of primary values for tables with compound primary keys). Consider retrieving a modResource (same as a site_content, or MODx document currently) by it’s id field:

    <?php
    $resourceId= 12;
    $resource= $modx->getObject('modResource', $resourceId);
    ?>

    That’s it. You now have the entire document object for the resource with id 12.

    Here’s a little more advanced example to retrieve a collection of resources for a specific context (I’ll introduce modContext in an upcoming lesson):

    <?php
    $tblResource= $modx->ds->getTableName('modResource');
    $criteria= new modCriteria("SELECT * FROM {$tblResource} WHERE (context_key = {$obj->get('key')} OR context_key IS NULL) AND deleted = 0 AND published = 1");
    $collResources= $modx->getCollection('modResource', $criteria);
    ?>

    You’ll notice that there is still SQL involved at this point, but eventually, the API will be evolved (i.e. abstracted one level further) to allow the SQL segments to be automatically constructed from meta data declared for each specific database platform that we target, or created via API methods on top of PDO (e.g. modCriteria :: addWhereClause() or modCriteria :: addGroupBy(), etc.). Thus, we will achieve database independence incrementally, by implementing versions of certain core object classes for each platform we want to target. This will help us avoid the performance issues of abstracting the database layer completely and also allows us to optimize each implementation for the targeted platform and it’s exclusive features.

    Next Lesson: Resources, Elements, and New Ways to Use the API
    In my next lesson, I’ll be covering the basic concepts of Resources, Elements, and the many ways the modX class can be used, as a CMS controller, a stand-alone API object for accessing MODx data and functionality from external applications, as an AJAX request and object broker, or whatever else you might dream up.


    Please feel free to ask questions or suggest future lesson topics you’d like to see covered as we prepare to evolve away from our legacy code.
      • 4018
      • 1,131 Posts
      My god! You’re username pretty much says it all! You’re even a bigger geek than I am! LOL! Anyways...

      Aside from the fact that some of the concepts are beyond my knowledge and need a little explaining (yeah...I’d like to see a layman try to make sense of this!) alot of what you’re talking about is very exciting. Abstraction of the database model is by far the one thing I think MODx really needs. There will come a day when many users will jump ship from MODx due to the fact that it only supports MySQL. But with a good abstraction model in place, there would be nothing to stop a user from using Postgre or any other database solution out there.
        Jeff Whitfield

        "I like my coffee hot and strong, like I like my women, hot and strong... with a spoon in them."
        • 6726
        • 7,075 Posts
        It’s very nice to have a perspective on your work Jason, thanks !

        Actually, I am a layman but I find object oriented programming more attractive than procedural, probably why I am struggling with PHP... I mean it’s funny how you connect or you don’t with a language : it’s not programming but I never connected with german (the first foreign language I learned) while it just "clicked" with english !

        Of course I don’t get the meaning of the code, but the OO approach appeals to me, I started digging into Code Igniter and this I think I can learn...

        Anyway, it’s great to have some input about Tattoo’s logic !
        In due time, these posts could become valuable promotionnal resources aimed at coders, wouldn’t it ?
          .: COO - Commerce Guys - Community Driven Innovation :.


          MODx est l&#39;outil id
          • 22303 MODX Staff
          • 10,725 Posts
          Quote from: davidm at Apr 15, 2006, 04:22 AM

          Of course I don’t get the meaning of the code, but the OO approach appeals to me, I started digging into Code Igniter and this I think I can learn...

          Anyway, it’s great to have some input about Tattoo’s logic !
          In due time, these posts could become valuable promotionnal resources aimed at coders, wouldn’t it ?

          Yes, in fact, my motivation for posting this here was to see the team reaction to it before I prepare it for posting as a public blog series as we provide early access to the 1.0 MODx code and hopefully attract those more advanced developers looking for object-oriented framework solutions similar to CodeIgnitor, the Zend Framework, Symfony, etc, or those fed up with the Ruby/RoR platform. But the best part will be the fact that we can utilize PDO when available on PHP 5 platforms but still provide full functionality on PHP 4, something no other OO framework out there has managed to achieve from the research I’ve done.
            • 1764
            • 680 Posts
            Wow Jason, great stuff! It seems like you’re right on track, keep up the good work.

            I can think of a lot of nice methods that could be added to modObject. I’d love to see some serializers added, XML, SOAP, JSON, etc.

            I also noticed that you are mapping db datatypes to php datatypes. It would be really cool if this could handle the automatic conversion of vairous DB dates and timestamps to Unix timestamps for PHP.
              • 22303 MODX Staff
              • 10,725 Posts
              Thanks Adam! grin
              Quote from: aNoble at Apr 17, 2006, 09:06 AM

              I can think of a lot of nice methods that could be added to modObject. I’d love to see some serializers added, XML, SOAP, JSON, etc.
              I actually have a plan for these types of serializers, though I think I want them outside of the core object (i.e. to avoid their overhead if not used on a particular request, and also to allow collections of objects to be serialized at once); I’ll be discussing this in detail in my next 1.0 lesson covering [Web] Resources and Elements [of Content]. In fact, I hope to make these work similar to the way a widget does now, transforming a content element or collection of elements into these formats for delivery as a web resource, be it a page resource, a web-service resource, or an AJAX resource.

              Quote from: aNoble at Apr 17, 2006, 09:06 AM

              I also noticed that you are mapping db datatypes to php datatypes. It would be really cool if this could handle the automatic conversion of vairous DB dates and timestamps to Unix timestamps for PHP.
              Partially done and I definitely plan on making that portion more robust; currently, you can pass a format string to the modObject :: get() function to apply strftime formatting to the timestamps, although I may want to genericize this into a user-definable callback so more capabilities could be applied via custom formatting functions.

              BTW, I’m also considering releasing a version of just the ultra-light PDO / ORM solution as a stand-alone, LGPL project; just trying to come up with a good name for it; starting to lean towards modDSO ("mod"ular "D"ata "S"ervice "O"bjects).
                • 32963
                • 1,732 Posts
                Very nice concepts here Jason.

                The move to a PDO-like structure is a very good one.

                Keep up the good work.
                  xWisdom
                  www.xwisdomhtml.com
                  The fear of the Lord is the beginning of wisdom:
                  MODx Co-Founder - Create and do more with less.
                  • 6726
                  • 7,075 Posts
                  While you’re here Raymond : I have a little problem connecting the dots regarding the various "developpement branches" (forgive me if the term is improper, I am no dev).

                  My understanding was :
                  - MODx 0.9.x => Eto legacy branch still undergoing drastic improvements
                  - MODx 1.0 => Rewritten from the ground up to be even more modular and extensible, compatible with PHP4.
                  - Tattoo => a version of MODx 1.0 with enhanced (OO) features compatible only with PHP 5.1+

                  How much do MODx 1.0 and Tattoo share ? I mean, how does GreenCore relate to those ?
                  It’s a little fuzzy to me and I better be straight and clear about this... Thanks !





                    .: COO - Commerce Guys - Community Driven Innovation :.


                    MODx est l&#39;outil id
                    • 25663 MODX Staff
                    • 12,272 Posts
                    Hi David, you bring up some very very important points.

                    MODx 1.0 and Tattoo are are mostly based on Jason’s rewriting the MODx core iteratively more than 3 times. Each time it got better as he continued to explore the optimal way to solve several critical things relating to doing a CMF "right". The latest version which is about 80% there at the core level addresses:

                    • core functionality that can be extended without hacking the core
                    • recursive parser
                    • simplified code syntax
                    • much smaller code base that’s completely modular (object oriented)
                    • custom meta content including tags, etc.
                    • more robust (performance-wise) event model
                    • more database options
                    • versions and rollbacks
                    • proper internationalization and localization
                    • multiple sites through "contexts" with one install (subsites/subdomains/etc.)
                    • works with PHP 4.3.x+ and even better/faster with PHP 5.1.x
                    • robust notifications via event plugins

                    Raymond’s GreenCore contains a lot of ideas that could be easily implemented on top of the above.

                    Regardless, I do not think we should remotely call them "forks" as that has a very negative connotation. This can easily be construed as internal discord, which would be tremendously bad for the project. They’re "feature branches" in my book.
                      Ryan Thrash, MODX Co-Founder
                      Follow me on Twitter at @rthrash or catch my occasional unofficial thoughts at thrash.me
                      • 32963
                      • 1,732 Posts
                      Hi David,

                      Let me see if I can connect the dots. Originally Tattoo was said to be a compleley new framework and that it would be php 5 based. It would include things like OO structure with the above things Ryan had mentioned. MODx on the other hand would continue on a php4 pathway that users could eventually move from to Tatto. Not too sure on what the final name will be now but from what it seems MODx 1.0 Tattoo in php4.

                      I had some concerns with Jason’s idea of going with Propel as a DB abstraction layer for Tattoo. Overtime Propel was explored and has since then been customized to work with MODx. I’m not directly involved with the Road Map for version 1.0 and have given the lead to Jason to futher his development of the ORM integrated platform.

                      I on the other hand have some other ideas and ways of doing things. So in an effort to not interfare with work done on 1.0 I’ve decided to move ahead with GreenCore.

                      Some of the features of GreenCore can be ported to work on Tattoo but I think the underlining structure and API will be very different. With 80% of the work done it’s near to completion and developers will get a chance to be able explore some of the powerful features planned for it.

                      GreenCore is not even near 5% completed. It’s just starting it’s journey of a thousand miles (GreenToken, GreenCore, GreenFuse, etc). It might not be as big and powerful MODx 1.0. It’s just the way I like it, flexible, small and simple. It’s been designed for with casual developer in mind.

                      David, I wouldn’t be too concerned about GreenCore as it’s not in any state for display as yet. It’s all ideas at the moment with little bits of code here and there. Let’s just wait and see what it will be able to do.

                      I would like to encourage all the team to remain focused on the path to 1.0 and to offer as much support as necessary in helping to make it stable and reliable.

                      God Bless
                        xWisdom
                        www.xwisdomhtml.com
                        The fear of the Lord is the beginning of wisdom:
                        MODx Co-Founder - Create and do more with less.