git workflow on MODX Cloud

Hi bobwal,

No, you don't have to make use of BitBucket at all It's just a process that allows you to adopt version control as your deployment method (so combining too ideas into one) without having to ever directly access the cloud server itself.

What you're describing is very much what we do. We have local files which we FTP-sync to the cloud on save. As you say, it's actually quicker to work on the cloud than it is on xampp.

However, that's just too dangerous when it comes to working with large-scale live sites, especially in a team of developers (which is the problem we face). If one developer makes a change their file is FTP'd up to the cloud. If another developer then changes their local file it will overwrite it. Because it's just FTP there's no revision checking, so this can happen. Eek!

By sticking Git in the middle you're stopping old files from overwriting new ones. However, you obviously don't want to commit and push with every file change in order to see it on the cloud - which means you HAVE to be able to work locally.

However, because we cannot remote access the cloud databases (for security?) you're unable to run MODX locally on a remote DB. Big problem! This makes the cloud unsuitable for a proper dev->production->live deployment model.

I'm a bit stuck.

Some scribbles I did last night...

Some explanation...

This is how I think it should work (or at least, how I think I'd like it to work)

DEV

Multiple developers all pulling/pushing from a shared Git repo (or from each other, if necessary, as the Git model permits). These developers all share the same DB, per project. When the developers are happy with their work, they push to the central Git repo. The DEV site auto-updates with each push to the repo (snippets/chunks/templates/packages etc). This is what we were doing further up the thread with BitBucket. Nice.

Nobody actually takes any notice of the DEV site until all developers have completed their work. All development happens on each developers local machine. An appropriate .gitignore file is obviously setup to ensure config files are not overwritten, etc.

Once everyone has pushed their changes, and everyone is happy with the DEV site, it's ready to go to the next stage...

STAGING

Developers do not have direct access to the STAGING Git repo/branch. This is pushed to from the DEV repo itself by an admin. Developers can only ever access the DEV repo, therefore protecting the STAGING and LIVE sites and enforcing tiered deployment.

At the same time as pushing to the STAGING repo, a SQL Compare tool is used to sync any new DB records to the STAGING DB. This is a "selective" sync, so only DB records for snippets/chunks/templates/packages etc are pushed. Users/resources or any other "content" is not (because that staging site may well already have different content compared to the DEV site. The same applied further up the ladder).

There are a few SQL Compare/Sync tools available, such as http://www.red-gate.com/products/mysql/mysql-comparison-bundle/ or http://www.mysql.com/products/workbench/. Whilst this requires a human eye, it should be a relatively trivial task and just requires some understanding of what should/shouldn't be pushed DB-wise.

The STAGING site is now available for your client to review.

LIVE

Once the client is happy with the site, the processes is effectively repeated. An admin pushes changes from the STAGING repo/branch to the LIVE repo/branch and the DB sync is carried out again.

The site updates are now available to the public and considered LIVE.

This whole process just repeats and repeats with each major update. Developers push their changes to the DEV site for "internal" review, this is then pushed to the STAGING site, which is then pushed to the LIVE site.

Thanks Chris, very helpful, I'll be keeping an eye on this thread. Whilst I don't currently require the above level of workflow it's definitely something I want to get to grips with.

To my surprise, I've just found that TOAD for MySQL has both a schema & data compare feature. Seeing as this is my MySQL DBMS of choice already, that's a great thing to find(!!!) and would be very useful for this deployment model.

I would be very interested to know if you got this worked out, Chris. I'm not going to the cloud, but my production environment is extremely similar to yours (including Git and a staging server, while I work locally). I'm having quite the time figuring out things like which tables in my database will need to be migrated and what to exclude from Git altogether.

I appreciate you talking to yourself here. I'm amazed there aren't more of us developing in this manner. Thanks! [ed. note: stefsull last edited this post 11 years, 1 month ago.]

Hi.

I've had a few messages over the weekend asking more about this. I'm going to put together a blog post this week (link to follow) which'll detail the problems, limitations and solutions to help people find the right solution for them, and some deployment recommendations - now and in the future - for medium/large sites.

Watch this space

That would be GREATLY appreciated. I promise, if it existed already, I would have found it. (The only thing I could really find was related to MOVING a site, not synching one locally, on a staging server and in production.) Cheers!

Sounds great Chris, look forward to it.

Hi Chris,
did you have a chance to do that writeup on git/modx workflow? We're in the process of setting up ours and it would be very helpful to see an example.
Thanks in advance.