You were playing with the idea to contribute to db4o? So far you did not find a nice, easy and short task to take on? Here is your chance: Write a TypeHandler for db4o and get your code included in the db4o core distribution tomorrow.
db4o comes with a big promise: Just call ObjectContainer#store() and any object and all attached objects will be automagically stored. The challenge to do this is high: Every possible class out there needs to be "understood" by db4o and an efficient byte array representation needs to be created on storage media. Indexing of "parts" of objects and querying is part of the task. With evolving languages and an increasing number of class constructs that need to be supported we decided to make it easier to write the code for marshalling and unmarshalling: Enter TypeHandlers.
TypeHandlers define how objects of a specific class are stored, loaded and queried by db4o.
By default db4o stores and restores all non-transient fields of an object. By writing TypeHandlers it is possible to alter the behaviour. A TypeHandler could decide to store only some members of a class or it could decide to store a completely different and more efficient representation.
The possible benefits of using a Typehandler:
- Storing and restoring objects can be faster.
- The database file can be smaller.
- Objects that consist of sub-objects (Example: collections) can be guarded against inconsistencies in concurrent multi-user environments by preventing "the object" from being split, but using one single contiguous slot as the marshalled representation. ( We will soon also demonstrate a stateful collection implementation over multiple pages, also powered by a Typehandler.)
- Fields that are intended to be transient do not need to be stored.
- Typehandlers can alter behaviour even if the sources of the persistent classes are not available or can not be altered.
- Querying support.
TypeHandlers are expected to do the complete marshalling of an object by writing/reading directly to/from the marshalling buffer.
About a year ago the TypeHandler4 interface consisted of more than 40 methods. Today only 5 methods remain. To make sure that the TypeHandler interface is ready for heavy duty use, we have already used it ourselves to implement some collection typehandlers. See:
com.db4o.typehandlers.CollectionTypeHandler (Java)
com.db4o.typehandlers.MapTypeHandler (Java)
Db4objects.Db4o.Typehandlers.ListTypeHandler (.NET)
Db4objects.Db4o.Typehandlers.MapTypeHandler (.NET)
In the latest development build (7.5.x.y) these TypeHandlers are already turned on to do real work storing, loading and querying some selected collections. To see how internal TypeHandlers are configured, take a look at:
com.db4o.internal.TypeHandlerConfigurationJDK_1_2 (Java)
Db4objects.Db4o.Internal.TypeHandlerConfigurationDotNet (.NET)
You will notice that many TypeHandlers are still missing if the full featureset of JDK and .NET collections is to be supported. Here is where we would like to ask you for your help:
Could you write a TypeHandler for us for a collection class that you think we need to support?
Adapting and registering one of the already existing typehandlers for your favourite collection will probably take you less than an hour of work. You can simply follow the pattern that is already there.
However writing the TypeHandler code is not the complete story. Since we want to be absolutely sure that a new TypeHandler works correctly, the bigger part of the implementation work is required for testing. We want to make sure that objects stored with an older version of db4o can be read and stored even after your new TypeHandler is installed.
For this purpose we developed the "Db4oMigrationTestSuite" test.
On Java you can run this class directly. You will find it in our SVN in the db4ojdk1.2 project.
( If you have not worked with the db4o sources before, here is a guide to get started. )
If you develop for .NET you would check out the db4o.net project from our SVN. You would typically work with the Db4o-2008.sln solution in the top level folder. To run the tests, the Db4objects.Db4o.Tests-2008 project would be your startup project. To save time by not running all the tests every time you can edit Db4objects.Db4o.Tests.AllTests in the Db4objects.Db4o.Tests-2008 project: If you want to run migration tests only, you can comment out all the entries in the #TestCases() method and add the following instead:
typeof(Db4objects.Db4o.Tests.Common.Migration.Db4oNETMigrationTestSuite)
Both for Java and .NET:
If you want the Db4oMigrationTestSuite to truely run against all relevant old db4o engines, you should check out the db4o.archives project from our SVN also.
Once you have testing set up and running, you are ready to go to write your own test for updating your favourite collection.
For an example how an update test could look like, please see:
com.db4o.db4ounit.common.handlers.ArrayListUpdateTestCase (Java)
Db4objects.Db4o.Tests.CLI2.Handlers.GenericListVersionUpdateTestCase (.NET)
Only when you have the test in place it makes start to implement your own TypeHandler as a more efficient method to store the collection. Test first, remember?
Should you have any problems to get started, please ask for help in the db4o product developer forum. It is a good idea to announce any work that you are about to do in the forum in any case. That will ensure that the work for a specific collection is only done once.
Now that you are all set to start:
Good luck for writing your TypeHandler!
All the new Typehandlers we have built so far are now available in the TRUNK of our SVN and with our continous build.
They help to make storage of collections more efficient:
- Less state is pushed around than with our previous translator approach.
- Maps no longer store an internal Entry object for each entry.
- All .NET collections, including all .NET generic collections will soon be handled efficiently with Typehandlers. Current issues with .NET generic collections in concurrent multi-user setups will soon be history.
- Old database files can be read with new Typehandlers installed. Future marshalling format changes are easily possible by writing and installing yet further new Typehandlers.
- We have a very nice hook into collection treatment that we will be able to use for much faster collections.
Enjoy!