Java is a static, strongly typed language. Every variable has an explicitly defined type which cannot be changed during the execution. Conversions from one-type to another have to be done explicitly using casting. This feature helps in safer application development: errors can be caught earlier, during the development and compilation, rather than run-time. But this is also a constraint which makes harder to develop some applications, especially tools and integration components. For example, I have been working for a while now in the development of tools to compute the difference between business entities. This will not be a hard task for specific Java classes such as com.intspc.order.PurchaseOrder or com.intspc.crm.Customer. But it is harder to write a generic diff tool, for an (almost) arbitrary class. This requires a dynamic data model … within a strongly typed language.

Major tools companies: IBM, BEA (now Oracle), Oracle, SAP and others have been working on a specification for handling dynamic data in Java (there are similar APIs for C, C++ and COBOL). It is called “Service Data Objects for Java Specification.” The latest version is 2.1.0 and available at: http://osoa.org/download/attachments/36/Java-SDO-Spec-v2.1.0-FINAL.pdf. This has been released in November 2006 and there are several vendors who are providing tools based on this framework: IBM, Rogue Wave, BEA. I looked at the Apache Tuscany which is an open-source implementation of the SDO specification. The latest version of Tuscany is 2.0.0-M4.

For a while I have been working on an application that manages reference data which has a few packages that deal with handling dynamic data. This happened without knowing about the SDO specification, and dealing with more specific requirements. In this context, reading the specification and seeing the Tuscany implementation was a very interesting experience because it emphasized design and implementation decisions I made sometimes conscious and sometimes without paying a lot of attention. My implementation of the Dynamic Data capabilities is a superset of a subset of SDO specification. This is a common situation when multiple teams create applications for a specific domain. I will refer to my implementation of Dynamic Data in Java as DData.

On the common side, both SDO and DData deal with change explicitly. I found this important because most data frameworks do not handle change explicitly, leaving the developer on his own. Besides slight changes in names, the types of changes are the same: Add, Modify, Delete. A big difference is around the operations of Diff and Merge. In DData these are also part of the interface but they are not provided in SDO. The operations can be implemented in external classes. My experience is that these operations are CPU and memory intensive and having access to the internal implementation of the data objects can alleviate performance issues.

Handling meta-data was a challenge in DData. I felt that I needed to implement a new type system which is a complex exercise and was not the main focus of the application. I decided to put a few constraints on the type of objects that I will model. Every entity can be a list, a record or a field (basic type). This model does not support for XML mixed elements. SDO does that using Sequences. SDO also supports inheritance between types which DData does not. I looked at the Tuscany implementation and the meta-data is based on the EMOF package from Eclipse which is much more powerful than what is in DData. But the simple DData meta-model is very practical for regular business application.

There are more things in DData that are not in SDO: there are several extra Data Access Services: Excel and Flat-File; it is possible to have a mixture of data-object implementations to improve performance. SDO has a larger scope than DData. It supports multiple data-objects in a data-graph. There is also the concept of containment which allows a mixture of managed data and external objects. A strong capability is to generate the strongly-typed classes for a dynamic object. That allows a mixture of strongly-typed and dynamic data processing.

It is always hard to create a clean and practical API. Looking at SDO gave me a chance to understand how other people approached the API for dynamic data after I have done it myself. This was a great learning.