Testing software against multiple database management systems can be tricky. It’s usually a headache to configure and maintain these tests, and then they usually take a long time to run.
Of course there are multiple ways of doing this, and each approach probably has some advantage for your system, the build environment, and whether your developers have access to the test databases.
JBoss DNA is an open source project, and open source projects run into some walls that commercial software developers often don’t have. With open source, each developer likely has a very different environment and likely does not have access to the same databases or same set of DBMSes. It doesn’t work to hard-coding the connection information or even to put it in some property files, because each developer would have to go and change all those settings. Even if we agreed upon a convention, not everyone has access to the same DBMS systems. What we want is to be able to run all of our builds normally without relying upon any external resources, and then at our choosing easily run our tests against the database each developer has access to.
Luckily, we use Maven, and so we can use Maven profiles to define a different environment for each database we want to use. In each database profile, we can add dependencies for the appropriate JDBC driver and specify connection properties. And it becomes very easy to turn profiles on and off, which means developers can choose which databases they want to test against. And, we can use HSQLDB or H2 by default, since these are fast, all developers have them (merely because of Maven dependencies), and there transient.
The only challenge is that we already are using profiles for different kinds of builds we do. One of our profiles (the one used by default) is fast because it simply compiles the source and run only the unit tests. This is the default because we run this build all of the time – it’s actually the easiest way to run all of the unit tests, so I run it constantly throughout the day as I make changes locally.
We have another profile that also compiles and runs the integration tests – these take several minutes to run, so it’s not very nice to have to wait for all these tests while you’re just verifying some changes didn’t cause some unintended behavior. Although I run these tests locally, I suspect most of the developers let our automated continuous integration server do the work.
Other profiles also generate JavaDoc, build our documentation (compiling DocBook into multiple HTML formats and one PDF form), and create the ZIP archives that we publish in our downloads area.
The database profiles are actually orthogonal to these other profiles. Despite this, there is (at least) one way to make them stay independent and play nicely together, and that’s activating them with properties. Our Maven command line becomes:
mvn clean install -Ddatabase=postgresql
and if we want to use any of our other profiles, we can just use the “-P” switch as we’ve always done:
mvn -P integration clean install -Ddatabase=postgresql
That would work great. All we have to do is define each database profile to activate based upon the “database” property.
Oh, you may be wondering why we don’t just explicitly name each profile on the command line. Well, actually we can, and it works as long as the user always remembers to include one database profile along with the other profiles they want to use. With Maven, if the user names a profile, then no other profile is activated by default, which means that our builds may run without a database profile, and this can break the tests. Activating the database profile with a property solves this problem.
Defining the database profiles
As I mentioned earlier, we really want a profile for each database that we want to test against. If we define the profiles in the parent POM, then all subprojects inherit them. So, the first step is to put in our parent POM a separate profile for each database configuration. Here is the HSQLDB configuration:
<profile>
<id>hsqldb</id>
<activation>
<property>
<name>database</name>
<value>hsqldb</value>
</property>
</activation>
<dependencies>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>1.8.0.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<properties>
<database>hsqldb</database>
<jpaSource.dialect>org.hibernate.dialect.HSQLDialect</jpaSource.dialect>
<jpaSource.driverClassName>org.hsqldb.jdbcDriver</jpaSource.driverClassName>
<jpaSource.url>jdbc:hsqldb:target/test/db/hsqldb/dna</jpaSource.url>
<jpaSource.username>sa</jpaSource.username>
<jpaSource.password />
</properties>
</profile>
Note how the profile is activated when the “database” property matches a value (“hsqldb” in this case.) We also define the dependencies on the HSQLDB JDBC driver JAR, and we define the properties that we’ll use in our tests. And we have one of these for each database we want to test against.
But before we look at how those properties get injected into our test cases, let’s define the database profile that should be used if the “database” property is not set. We do this with a different activation strategy:
<profile>
<id>default_dbms</id>
<activation>
<property>
<name>!database</name>
</property>
</activation>
<dependencies>
<dependency>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>1.8.0.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<properties>
<database>hsqldb</database>
<jpaSource.dialect>org.hibernate.dialect.HSQLDialect</jpaSource.dialect>
<jpaSource.driverClassName>org.hsqldb.jdbcDriver</jpaSource.driverClassName>
<jpaSource.url>jdbc:hsqldb:target/test/db/hsqldb/dna</jpaSource.url>
<jpaSource.username>sa</jpaSource.username>
<jpaSource.password />
</properties>
</profile>
I’ve chosen that the default database profile is identical to one of the other profiles, and this means I have some duplication in the POM file. Personally, the cleanliness for the user seems worth it.
Before we leave our parent POM, we also should probably define default values for all of the properties we use (or can use) in the database profiles. This will make it easier to inject those properties into our tests. So in the “<properties>” section of the parent POM, define a few default values:
<properties> <jpaSource.dialect/> <jpaSource.driverClassName/> <jpaSource.url/> <jpaSource.username/> <jpaSource.password/> <jpaSource.maximumConnectionsInPool>1</jpaSource.maximumConnectionsInPool> <jpaSource.minimumConnectionsInPool>0</jpaSource.minimumConnectionsInPool> <jpaSource.numberOfConnectionsToAcquireAsNeeded>1</jpaSource.numberOfConnectionsToAcquireAsNeeded> <jpaSource.maximumSizeOfStatementCache>100</jpaSource.maximumSizeOfStatementCache> <jpaSource.maximumConnectionIdleTimeInSeconds>0</jpaSource.maximumConnectionIdleTimeInSeconds> <jpaSource.referentialIntegrityEnforced>true</jpaSource.referentialIntegrityEnforced> <jpaSource.largeValueSizeInBytes>150</jpaSource.largeValueSizeInBytes> <jpaSource.autoGenerateSchema>create</jpaSource.autoGenerateSchema> <jpaSource.compressData/> <jpaSource.cacheTimeToLiveInMilliseconds/> <jpaSource.creatingWorkspacesAllowed/> <jpaSource.defaultWorkspaceName/> <jpaSource.predefinedWorkspaceNames/> <jpaSource.model/> <jpaSource.numberOfConnectionsToAcquireAsNeeded/> <jpaSource.referentialIntegrityEnforced>true</jpaSource.referentialIntegrityEnforced> <jpaSource.retryLimit>3</jpaSource.retryLimit> <jpaSource.rootNodeUuid/> <jpaSource.showSql>false</jpaSource.showSql> </properties>
Injecting the database properties
Each of the database profiles define a bunch of properties, and we want to use those property values in each of our tests. The easiest way to do this is to use Maven filters to substitute the property values in some of our resource files when it copies them into the ‘target’ directory. We could do that in the parent POM, but it’s probably best to do that in each subproject, where we can be specific about the files we want filtered. JBoss DNA has a “dna-integration-test” subproject, and so in its POM file we just need to turn on filtering:
<build>
...
<testResources>
<testResource>
<filtering>false</filtering>
<directory>src/test/resources</directory>
<includes>
<include>*</include>
<include>**/*</include>
</includes>
</testResource>
<!-- Apply the properties set in the POM to the resource files -->
<testResource>
<filtering>true</filtering>
<directory>src/test/resources</directory>
<includes>
<include>tck/jpa/configRepository.xml</include>
</includes>
</testResource>
</testResources>
...
</build>
Here the first “<testResource>” fragment copies all of the files in the “src/test/resources” directory without doing any filtering, while the latter fragment copies the “src/test/resources/tck/jpa/configRepository.xml” file and does the substitution of the property values. For example, any “${japSource.url}” strings in the file are replaced with the appropriate value for this property as defined in the database profile (or the default).
The “configRepository.xml” file is a configuration file for the DNA JCR engine. What if we want to inject the database properties into our test cases? Well, one easy way is to have Maven filter a property file and then just have our test cases load that property file and use the data in it. We actually do this in one of our other projects, and it works great.
Running the tests
Now we can see the fruits of our labor. So we can run
mvn clean install
or
mvn -P integration clean install
to run all unit and integration tests, just as we could before. Since we don’t specify “-Ddatabase=dbprofile” on the command line, these builds will use the default database profile, which for us is HSQLDB. That means any database-related tests we run are fast and require no external resources. Brilliant!
Of course, once all of our tests pass with the default configuration, we can then run all the tests against different databases with a few simple commands:
mvn -P integration install -Ddatabase=mysql5 mvn -P integration install -Ddatabase=postgresql8 mvn -P integration install -Ddatabase=oracle10g etc.
Pretty sweet! Well, as long as we have some patience.
Hat tip to the Hibernate and jBPM team, since our approach was largely influenced by a combination of their setup.
Filed under: techniques, testing
ModeShape 