Adopting DBVCS
- by Wes McClure
Identify early adopters Pick a small project with a small(ish) team. This can be a legacy application or a green-field application. Strive to find a team of early adopters that will be eager to try something new. Get the team on board! Research Research the tool(s) that you want to use. Some tools provide all of the features you would need while some only provide a slice of the pie. DBVCS requires the ability to manage a set of change scripts that update a database from one version to the next. Ideally a tool can track database versions and automatically apply updates. The change script generation process can be manual, but having diff tools available to automatically generate it can really reduce the overhead to adoption. Finally, an automated tool to generate a script file per database object is an added bonus as your version control system can quickly identify what was changed in a commit (add/del/modify), just like with code changes. Don’t settle on just one tool, identify several. Then work with the team to evaluate the tools. Have the team do some tests of the following scenarios with each tool: Baseline an existing database: can the migration tool work with legacy databases? Caution: most migration platforms do not support baselines or have poor support, especially the fad of fluent APIs. Add/drop tables Add/drop procedures/functions/views Alter tables (rename columns, add columns, remove columns) Massage data – migrations sometimes involve changing data types that cannot be implicitly casted and require you to decide how the data is explicitly cast to the new type. This is a requirement for a migrations platform. Think about a case where you might want to combine fields, or move a field from one table to another, you wouldn’t want to lose the data. Run the tool via the command line. If you cannot automate the tool in Continuous Integration what is the point? Create a copy of a database on demand. Backup/restore databases locally. Let the team give feedback and decide together, what tool they would like to try out. My recommendation at this point would be to include TSqlMigrations and RoundHouse as SQL based migration platforms. In general I would recommend staying away from the fluent platforms as they often lack baseline capabilities and add overhead to learn a new API when SQL is already a very well known DSL. Code migrations often get messy with procedures/views/functions as these have to be created with SQL and aren’t cross platform anyways. IMO stick to SQL based migrations. Reconciling Production If your project is a legacy application, you will need to reconcile the current state of production with your development databases. Find changes in production and bring them down to development, even if they are old and need to be removed. Once complete, produce a baseline of either dev or prod as they are now in sync. Commit this to your VCS of choice. Add whatever schema changes tracking mechanism your tool requires to your development database. This often requires adding a table to track the schema version of that database. Your tool should support doing this for you. You can add this table to production when you do your next release. Script out any changes currently in dev. Remove production artifacts that you brought down during reconciliation. Add change scripts for any outstanding changes in dev since the last production release. Commit these to your repository. Say No to Shared Dev DBs Simply put, you wouldn’t dream of sharing a code checkout, why would you share a development database? If you have a shared dev database, back it up, distribute the backups and take the shared version offline (including the dev db server once all projects are using DB VCS). Doing DB VCS with a shared database is bound to cause problems as people won’t be able to easily script out their own changes from those that others are working on. First prod release Copy prod to your beta/testing environment. Add the schema changes table (or mechanism) and do a test run of your changes. If successful you can schedule this to be run on production. Evaluation After your first release, evaluate the pain points of the process. Try to find tools or modifications to existing tools to help fix them. Don’t leave stones unturned, iteratively evolve your tools and practices to make the process as seamless as possible. This is why I suggest open source alternatives. Nothing is set in stone, a good example was adding transactional support to TSqlMigrations. We ran into situations where an update would break a database, so I added a feature to do transactional updates and rollback on errors! Another good example is generating change scripts. We have been manually making these for months now. I found an open source project called Open DB Diff and integrated this with TSqlMigrations. These were things we just accepted at the time when we began adopting our tool set. Once we became comfortable with the base functionality, it was time to start automating more of the process. Just like anything else with development, never be afraid to try to find tools to make your job easier! Enjoy -Wes
