Having worked on Reference Data Management for a long time, I found the need for a categorization of data from the perspective of change. For example, Telco companies are using Business Support Systems (BSS) and Operational Support Systems (OSS) applications to manage terabytes of data stored in hundreds of tables, XML files and properties files. This data is changing all the time: some often, some rarely; some changes have large impact other have limited impact; the impact could be in the number of customers affected, products sold, financials. There are many aspects to consider about data and its change and each aspect is handled in a specific way.

Depending of how often data changes, we have:

• Very rarely changing: Customer Type, Order Statuses – these are at the core of the application and while they are present in the data storage layer, it cannot change without a large impact to the application functionality
• Rarely changing data: Service Type – entities are changing when the organization adds a new line of business, does an acquisition
• Medium Changing: Product Type information – new products added, discounts, campaigns
• Often Changing: Customers and Customer-Product information – new customers added to the system; customer changes the products/services provided
• Very often changing: Customer call records (CDRs) – every time the user makes a phone call a record is added. New data is created every day, every second.

Depending on the impact of the change financially:

• Large Impact: Changes to the price plans, discounts, campaigns – changes affecting many customers and the overall organization financial situation
• Small Impact: Customer orders or cancels a service – changes affect only one customer
• Changes with no financial impact: customer changes the number of rings for a voice mail to be activated – no financial impact on the customer or organization.

Depending on the impact to the service quality:

• High risk: changing the service delivery platform for a service; changing the voice mail provider – impact to all customers
• Low risk: modifying a service for a customer

In time, the trend has been to make the data more ready for change and thus providing more flexibility to the organizations and customers. For example, before, customers had to call the call center to make customer information changes, now the customers can change their profile on-line; before, a vendor had to make a change in the application to support a new Service Type, now the organization can make these changes using a configuration tool.

To reduce the risks associated with the data changes, organizations are using specific methods to deal with change.

(Continue reading …)

Java is a static, strongly typed language. Every variable has an explicitly defined type which cannot be changed during the execution. Conversions from one-type to another have to be done explicitly using casting. This feature helps in safer application development: errors can be caught earlier, during the development and compilation, rather than run-time. But this is also a constraint which makes harder to develop some applications, especially tools and integration components. For example, I have been working for a while now in the development of tools to compute the difference between business entities. This will not be a hard task for specific Java classes such as com.intspc.order.PurchaseOrder or com.intspc.crm.Customer. But it is harder to write a generic diff tool, for an (almost) arbitrary class. This requires a dynamic data model … within a strongly typed language.

Major tools companies: IBM, BEA (now Oracle), Oracle, SAP and others have been working on a specification for handling dynamic data in Java (there are similar APIs for C, C++ and COBOL). It is called “Service Data Objects for Java Specification.” The latest version is 2.1.0 and available at: http://osoa.org/download/attachments/36/Java-SDO-Spec-v2.1.0-FINAL.pdf. This has been released in November 2006 and there are several vendors who are providing tools based on this framework: IBM, Rogue Wave, BEA. I looked at the Apache Tuscany which is an open-source implementation of the SDO specification. The latest version of Tuscany is 2.0.0-M4.

For a while I have been working on an application that manages reference data which has a few packages that deal with handling dynamic data. This happened without knowing about the SDO specification, and dealing with more specific requirements. In this context, reading the specification and seeing the Tuscany implementation was a very interesting experience because it emphasized design and implementation decisions I made sometimes conscious and sometimes without paying a lot of attention. My implementation of the Dynamic Data capabilities is a superset of a subset of SDO specification. This is a common situation when multiple teams create applications for a specific domain. I will refer to my implementation of Dynamic Data in Java as DData.

(Continue reading …)

Application design and Environment management touch each other when the components designed and developed are deployed on servers. A seemingly simple task gets complicated when tens of interconnected applications are deployed in multiple environments made of hundreds of servers. This article is about how to systematically approach a large deployment. Some ideas are:

• develop a conceptual model for the application deployment based on UML,
• identify application design deliverables and environment management tasks involved in the deployment
• use tools to manage the inventory of the applications, servers and their association.

UML has the Deployment View to describe the link between the logical components and the run-time resources that will execute the code. The basic concepts are Node – the run-time computational resource, and Artifact – a physical entity such as a file. The Artifact is a “manifestation” of a component. In the real world, these concepts are not used by the implementation team, instead they operate with “servers”, “databases”, “Weblogic Servers.” UML2 provides ways to accommodate these, higher-level, entities. One way is to use the concept of “execution environment.” Databases and J2EE application container are execution environments and can be represented as Nodes in the deployment diagrams. To show that a database runs on a server I am using associations between nodes qualified with the stereotype <<hostedOn>>. The picture on the right shows how to represent the Weblogic, Database and the Unix Servers in UML.

(Continue reading …)

The Command Pattern is well-know pattern in the OO community. Originally described in the Gang of Four book there have been multiple variants over time (e.g. http://www.javaworld.com/javaworld/javatips/jw-javatip68.html). The main characteristics of the Command pattern are:

• separates the Client of a function from the Receiver through Commands. This separation provides the flexibility to extend independently the client and the receiver;
• the Command is a first class entity that can be manipulated outside the client or receiver. This allows to create new commands from the existing ones through inheritance, group multiple commands in one and add transaction capabilities. The Command is a reification of the method invocation by the client on the receiver.

The original command pattern is powerful in itself, but working with it in multiple projects I found some limitations:

• the Command object knows (i.e. is coupled with) the Receiver in order to invoke the method. That means that the Client, which builds the Commands is also aware of the Receiver (even if not all the details). This is against the original intent of the pattern which is to decouple the client from the receiver;
• in most cases the Client is interested in the results of executing a Command. Only rarely a client has a “send-and-forget” behaviour. There are ways to pass back the results to the client for example by having a “shared” context between  the client and the receiver. This solution creates a hidden dependency. A better option is for the client to register for command execution updates through the Command;
• often a Command invokes multiple Receivers in the same execution.

The article is about extensions to the Command Pattern to avoid these limitations.

(Continue reading …)