One problem is that we are only doing this at the start of the whole test suite. I changed a test class to select all the objects in the table (that corresponds to that object) and remove them all. Well this quickly ran into RI problems where the objects spanned many tables and it was just a pain.
What I think that we should do for the tests now is have a clean method that clears all the tables with data from the tests on each setup method. Now, the problem that I can see with this is that at after you have run the tests, all the tables have been cleared out so you can't see what is actually in the tables. Maybe if we just have a switch on if it will clear out the tables or not. But the problem that I see with that solution is that the tests would probably break if you didn't clear them out between each test.
It would be good if I could isolate the db from the rest of the app like Andrew talks about, but the data persistence is so important to our app that I feel cutting it out too much would be too much of a risk. Right now I am willing to have slightly slower tests because they actually contact a db in the hope that we'll find integration problems earlier. No surprises. (that's actually a team motto. ;-) )
Listening to: The Tragically Hip - Grace Too
I was thinking about this problem too, and thought it might be nice to have a set of tables where the data doesn't change, which I would use to test methods that only read the database. When you're reading, it's nice to know what to expect -- and this could be uniform over all reading tests.
ReplyDeleteThen I'd have another set of tables which are TRUNCATEd after each test method used for times when the database must be changed/written to.
Come to think of it, you don't need two sets of tables ... just two databases -- but not everyone has that luxury.
ReplyDeleteWe don't have a clear definition of what is a "write" test and what is a "read" test. They are mixed by what classes it is.
ReplyDeleteSince we're using CMP I'm not too worried that the reads / writes are done correctly since that's all generated code. Since we *do* go through a translation of the objects that we have near the UI and the ones that are near the db, one of the "standard" tests were are doing is if you can write a object (goes all the way down), and using they key it gave, can you get back an equal object?
And here is the tricky part. This is why I made those dynamic equals, hashcode, and toString methods. If you are putting your trust in the equals method, you better be damn sure that it's solid even after people are changing the system.
http://www.beernut.ca/jim/archives/001763.html
To assist in this, I also ended up making a "dynamic fill object" method where you pass in an object and it fills it up with "random" info. This should catch the cases where a member is added and the defaults are compared and deemed to be "equal". Hopefully armed with these methods our tests will be able to point out if you are not retaining something long before it goes live.
Just wondering for AudioMan, are you running your tests with your db "in memory"?
Not even 2 db's. As long as all your stuff is in one schema, you can use 2 schema's. That's how our stuff was at first. Both developers were using the same instance.
ReplyDeleteHowever, using EJB's, in the mapping it ties the object to the schema too (so you can have "shared" tables in different schema's). So we ended up having an instance per developer. So far it's working out fine. ;-) *knock on wood*
AudioMan, including the "production database", uses in-memory databases everywhere.
ReplyDeleteI'm going to look into using other types of databases, it's just low on the priority list.
It always uses "in memory"? It doesn't write to the disk? You loose everything when you close the app?
ReplyDeleteMaybe I misunderstood... I thought that "in memory" databases meant that the tables, etc (all the db objects) where *just* in memory.
I figured that would be faster since you wouldn't have any I/O overhead for testing. That's one thing that I would really want to speed up tests... in memory db.
Actually, it's interesting how it's done. The database is stored on hard disk as SQL statements. These statements are used to regenerate the in-memory database each time I use it. When the database is closed, it only exists as SQL statements.
ReplyDeleteThe downside is that it takes a bit of time to create the database, and that increases with the database size. So I'm looking into the documentation for having a hard-drive-based hsqldb database that doesn't take so long to prepare.
Really? I'm confused after reading the section titled "All-In-Memory Operation"
ReplyDeletehttp://hsqldb.sourceforge.net/web/hsqlModes.html
Specifically the line:
"The mode is used by the demo applet. When the browser is closed, all data is lost."
It sounds like you are using "Combined Disk and In-Memory Operation" (next section down, same link).
In this case I if you switched to just in memory I bet your tests would run like snot... so fast. ;-)
What's the difference between:
ReplyDeleteTRUNCATE TABLE "table_name"
and
DELETE FROM "table_name"
Leaving out the WHERE clause just causes everything to be deleted, doesn't it?
Check out:
ReplyDeletehttp://dev.mysql.com/doc/mysql/en/TRUNCATE.html
http://www.cs.nott.ac.uk/TSG/manuals/databases/oracle/standard/server.815/a67779/ch4m.htm#13607
;-)