Using Subversion and MODx for Team Development

Hey all, I am planning to integrate using revision control with my site development with MODx and one thing I haven’t figured out in the workflow is how to deal with the MySQL tables. Most web application dev scenarios use common table data but MODx as it stores templates and developed content in the DB. It begs the question is this even possible and stay sane?

What I have been thinking of is a basic svn repository on a webserver with all development on local setups and then migrating through staging and then live production.

What am I missing? How can I get this to work without having to tag every addition to the db somehow? Do I dump the SQL with every commit? How do I revert? How can I automate this or is their an easier way to do what I wan’t to do and still enable a team to work on projects with MODx?

All the best,

Jay

IMHO, MODx is not really well suited for this type of development as is, though I am involved in several projects where this is the case. I’m trying various scenarios to deal with it, but I’m not satisfied with any of them. Of course, this is the reason I have been developing 0.9.7, specifically the transport packages for migrating data objects from deployment to deployment, in order to better deal with this. But I have a ways to go to get a smooth workflow going using this method. The idea would be that you could select objects by various criteria from one environment, package it up, and install it on the target enviroment with a click of the button, both ways. It’s the interfaces for creating these package we’re lacking at the moment, and I’m sure the infrastructure will need improvements once we start trying it out.

In the meantime, I try to keep content and code meant to be managed by the team on the file system and include it. Of course, I do this anyway, to keep my cache files lighter (no snippets or chunks filling it up except those the client will manage). I still have to resolve changes to client managed content, but I generally encourage them to "test" their changes in the staging environment and then reproduce them in the production environment, or use diffs of sql scripts that dump key tables from both environments for comparison.

No easy answers on this yet, but it is in the vision...

I haven’t found a good solution for this either. You can certainly do an SQL dump before every commit and you could revert just by reverting the dump file and importing it (assuming that the dump file is under source control). It would be a pain, though, and I think it would make me wait too long to do the commits. I’m not sure what an update would be like when other users have changed things in the DB and I don’t see any workable way to have different branches (not to mention the horror of merging and resolving conflicts.)

I haven’t tried it, but I think you could create a page at the site that would do the sql dump and a command-line commit with SVN and another page that would revert and import for you, but I can’t imagine it working with multiple users.

Bob

Maybe I’m completely insane, but could you do a svn of mysql’s actual db files themselves? I might be completely wrong on this, as well.

It seems a bit of a challenge, most articles suggest dumping the schema but that won’t do. I have read in numerous places that trying to control the dbfiles is not sane or feasible as they are not text files and subversion is really designed from precompiled non binary files.

I will have to test to see if I can automate it but I am guessing that I will have to dump the SQL on commits and log them. The DB won’t be large so it won’t be time consuming for most small sites.

I envision the following for web development with MODx:

Checkout to local environment.
Develop on local environment (each team member)
Test on local environment.
Push to staging environment.
Migrate or symlink to production site.

How many dbs do I use? How would I do DB merges? Do I have dbs at every instance of the site--3 plus every developer has a copy?

Really, MODx 096 only really requires that the db and assets be under VC and only and the manager be a vendor version (for bleeding edge MODx 096 sites).

Any theoretical suggestions are welcome...

Quote from: smashingred at Jan 27, 2008, 07:46 AM

I will have to test to see if I can automate it but I am guessing that I will have to dump the SQL on commits and log them. The DB won’t be large so it won’t be time consuming for most small sites.

I might be misunderstanding what you mean by "log," but if the SQL dump file is in the site directory and you "add" it to version control, you’ll automatically have a version of it for every build as long as you do the SQL dump before the commit.

Bob

@BobRay

You are bang-on with your assumption. I will do the dumps to a db-stor folder in my folders.

As I write this I am playing with setting up the files to ignore all the files and folders that won’t be versioned. I am not sure how best to approach this as I am still very new to vc and learning the basics so my theorizing is mainly stream-of-consciousness .

When I get a simple workflow I will post it back.

Any observations and suggestions are still welcome.

Cheers,

Jay

@smashingred:

svn:ignore properties are your new best friend. These work great for ignoring files you don’t ever want to be versioned, or to ignore versioned files on recursive commits.

This is one of the mysteries for me svn:ignore. I can’t figure out how to make the /manager and /install ignored. If the are in the repo they will end up becoming part of the working copy won’t they?

You can’t turn folders in a working copy into ignored files can you?

I have been able to copy the files into the working dir and make them ignored but then when I do a commit some files seem to not be placed back in the repo.

I am at a loss.

I know that MODx is not well suited for this and for say a CakePHP application that I am developing it is much easier as I am using script libraries and not db controlled actions. I don’t want to put this off til 097 is done because I see the benefit, I just need to work out a flow that won’t make me insane and will be simple enough to instruct team devs to use without major issues.

What I was thinking is that as a repo admin for a MODx shop I want to have the latest stable distro of MODx’s Manager and Install Folder along with index.php and index-ajax.php in a repository on their own and /assets and /ht.access under the project so that the only files to be worked on by the team members would be in that project repo.

So I envision the following now:

<br />repository/client_project<br />    /branches<br />    /tags<br />    /trunk<br />        /assets<br />            /...[assets folders]<br />        ht.access<br />

and

<br />repository/modx_common<br />    /manager<br />    /install<br />    index.php<br />    index-ajax.php<br />

I know that there is one big problem with this is and that is the config.inc.php file which obviously exists in the manager. I guess I could just place the whole thing into the project repository intact but since we are not doing development on the modx_common project anyway.

Hmmm...

More thoughts and problems or am I just spinning wheels?

See http://svnbook.red-bean.com/en/1.4/svn.advanced.props.special.ignore.html for details on ignoring things and how it works.

You may also want to look at using svn:externals for "including" the common elements into your client-specific projects, perhaps as vendor branches. See http://svnbook.red-bean.com/en/1.4/svn.advanced.externals.html and http://svnbook.red-bean.com/en/1.4/svn.advanced.vendorbr.html for details.

I’ve deployed a few sites using SVN working copies as the delivery/deployment mechanism, and have found both of these techniques invaluable in the process.