Core Schema Versioning

Got my first taste of Revo development today; you have baked a lot of very interesting things into the core. I got bitten for a while with a bug related to plugins not triggering on new events; it turned out that the query to load the eventMap failed because I was missing the "propertyset" field of the site_plugin_events table. I got everything working again by creating the field per the spec in schema.xml.

We’re brave, and are developing against the Subversion trunk; I also had to do a "svn up" to fix setting the plugin’s event trigger from the manager.

All of that to say — how can I prevent this from happening when I update the code from SVN again? We want to pull in upstream fixes and improvements as they happen. Is there any concept of schema versioning? Can I run a migration to make the latest schema changes? Do I need to re-run setup each time I update the code, or manually diff the changes in schema.xml?

Quote from: _phred at Mar 27, 2009, 10:45 PM

Got my first taste of Revo development today; you have baked a lot of very interesting things into the core. I got bitten for a while with a bug related to plugins not triggering on new events; it turned out that the query to load the eventMap failed because I was missing the "propertyset" field of the site_plugin_events table. I got everything working again by creating the field per the spec in schema.xml.

We’re brave, and are developing against the Subversion trunk; I also had to do a "svn up" to fix setting the plugin’s event trigger from the manager.

All of that to say — how can I prevent this from happening when I update the code from SVN again? We want to pull in upstream fixes and improvements as they happen. Is there any concept of schema versioning? Can I run a migration to make the latest schema changes? Do I need to re-run setup each time I update the code, or manually diff the changes in schema.xml?

_phred: Thanks for being interested in Revolution development.

Revolution uses transport packaging to allow for quick deployment across servers; this is what powers the core.transport.zip and the setup process, as well as migrating any DB changes. You can find more information about using Revolution from SVN here:

http://svn.modxcms.com/docs/display/revolution/Development+and+Testing+Notes

Note that you’ll need to run a build (_build/transport.core.php) and run setup anytime that the text "[REBUILD/UPGRADE REQUIRED]" shows up in a commit. This is to add the proper DB changes in that commit. If it doesn’t say that in the commit message, you won’t need to.

Also, anytime you see the text "[LEXICON RELOAD REQUIRED]", you’ll need to go to the Lexicon Management section and click on "Reload Core Entries from Base". This will fill the DB with the newly adjusted lexicon entries for that commit. You could also do this via build/upgrade, but when only lexicon entries are the issue, this method is faster.

See the wiki article for more information; and please don’t hesitate to ask questions here in the forums, or fill bugs out in JIRA.

Thanks,
Shaun

+1 on schema versioning and migration
Though it’s more of an xPDO discussion for over here http://modxcms.com/forums/index.php/board,219.0.html

Awesome, thanks for the quick response and links, guys. I’ll keep a closer eye on the commit logs from now on

As far as tracking migrations and schema changes, the best way I’ve seen is in Ruby’s ActiveRecord (used to good effect in Rails). The nickel tour:

• Schema version stored in a schema_info table
• Each schema change defined as code in db/migrations/<version_number>_description_of_change.rb. This defines two functions, ’up,’ and ’down.’
• There’s a build target called ’db:migrate’ that checks the schema version in the database and, if it is less than the version of the most recent migration, runs the ’up’ function of each migration script starting with the one after current schema version.
• The db:migrate target can also take a version number lower than the current, which starts at the current schema version and runs the ’down’ function in each migration script until the target version is reached.
• If anything goes wrong while upgrading the schema, the build script attempts to restore the database to the previous version by calling the ’down’ functions.

It took me a little while to get used to the idea of making schema changes from code (ActiveRecord has great support for prepared ALTER TABLE statements), but after that it’s become a power tool when working on production sites. I can roll out a production schema change by checking out the latest code on the server and running "db:migrate," knowing that if anything goes wrong, I can quickly get my site back to a running state.