Curing the Database-Application mismatch
- by Phil Factor
If an application requires access to a database, then you have to be able to deploy it so as to be version-compatible
with the database, in phase. If you can deploy both together, then the application and database must normally be
deployed at the same version in which they, together, passed integration and functional testing. When a
single database supports more than one application, then the problem gets more interesting.
I’ll need to be more precise here. It is actually the application-interface definition of the database that needs to
be in a compatible ‘version’. Most databases that get into production have no separate
application-interface; in other words they are ‘close-coupled’. For this vast majority, the whole database
is the application-interface, and applications are free to wander through the bowels of the database scot-free. If
you’ve spurned the perceived wisdom of application architects to have a defined application-interface within the
database that is based on views and stored procedures, any version-mismatch will be as sensitive as a kitten.
A team that creates an application that makes direct access to base tables in a database will have to put a lot
of energy into keeping Database and Application in sync, to say nothing of having to tackle issues such as security and
audit. It is not the obvious route to development nirvana.
I’ve been in countless tense meetings with application developers who initially bridle instinctively at the apparent
restrictions of being ‘banned’ from the base tables or routines of a database. There is no good
technical reason for needing that sort of access that I’ve ever come across. Everything that the
application wants can be delivered via a set of views and procedures, and with far less pain for all concerned: This is
the application-interface. If more than zero developers are creating a database-driven application,
then the project will benefit from the loose-coupling that an application interface brings. What is important here is
that the database development role is separated from the application development role, even if it is the same developer
performing both roles.
The idea of an application-interface with a database is as old as I can remember. The big corporate or government
databases generally supported several applications, and there was little option. When a new application wanted access to
an existing corporate database, the developers, and myself as technical architect, would have to meet with hatchet-faced
DBAs and production staff to work out an interface. Sure, they would talk up the effort involved for budgetary reasons,
but it was routine work, because it decoupled the database from its supporting applications. We’d be given our own
stored procedures. One of them, I still remember, had ninety-two parameters. All database access was encapsulated in one
application-module.
If you have a stable defined application-interface with the database (Yes, one for each application usually) you need
to keep the external definitions of the components of this interface in version control, linked with the application
source, and carefully track and negotiate any changes between database developers and application
developers. Essentially, the application development team owns the interface definition, and the onus is
on the Database developers to implement it and maintain it, in conformance. Internally, the database can
then make all sorts of changes and refactoring, as long as source control is maintained. If the
application interface passes all the comprehensive integration and functional tests for the particular version they were
designed for, nothing is broken. Your performance-testing can ‘hang’ on the same interface, since databases are judged
on the performance of the application, not an ‘internal’ database process. The database developers have responsibility
for maintaining the application-interface, but not its definition, as they refactor the database. This is
easily tested on a daily basis since the tests are normally automated. In this setting, the deployment can proceed if
the more stable application-interface, rather than the continuously-changing database, passes all tests for the version
of the application.
Normally, if all goes well, a database with a well-designed application interface can evolve gracefully without
changing the external appearance of the interface, and this is confirmed by integration tests that check the interface,
and which hopefully don’t need to be altered at all often. If the application is rapidly changing its
‘domain model’ in the light of an increased understanding of the application domain, then it can
change the interface definitions and the database developers need only implement the interface rather than refactor the
underlying database. The test team will also have to redo the functional and integration tests which
are, of course ‘written to’ the definition. The Database developers will find it easier if these
tests are done before their re-wiring job to implement the new interface.
If, at the other extreme, an application receives no further development work but survives unchanged, the database
can continue to change and develop to keep pace with the requirements of the other applications it supports, and needs
only to take care that the application interface is never broken. Testing is easy since your automated scripts to test
the interface do not need to change.
The database developers will, of course, maintain their own source control for the database, and will be likely to
maintain versions for all major releases. However, this will not need to be shared with the applications that the
database servers. On the other hand, the definition of the application interfaces should be within the application
source. Changes in it have to be subject to change-control procedures, as they will require a chain of tests.
Once you allow, instead of an application-interface, an intimate relationship between application and database, we
are in the realms of impedance mismatch, over and above the obvious security problems. Part of this
impedance problem is a difference in development practices. Whereas the application has to be regularly built and
integrated, this isn’t necessarily the case with the database. An RDBMS is inherently multi-user and
self-integrating. If the developers work together on the database, then a subsequent integration of the database on a
staging server doesn’t often bring nasty surprises. A separate database-integration process is only needed if the
database is deliberately built in a way that mimics the application development process, but which hampers the normal
database-development techniques. This process is like demanding a official walking with a red flag in
front of a motor car. In order to closely coordinate databases with applications, entire databases have to
be ‘versioned’, so that an application version can be matched with a database version to produce a working build without
errors. There is no natural process to ‘version’ databases. Each development project
will have to define a system for maintaining the version level.
A curious paradox occurs in development when there is no formal application-interface. When the strains and cracks
happen, the extra meetings, bureaucracy, and activity required to maintain accurate deployments looks to IT management
like work. They see activity, and it looks good. Work means progress. Management then smile on the
design choices made. In IT, good design work doesn’t necessarily look good, and vice versa.
