September 28, 2005
Transactional Object Memory
I was visiting a customer in China this week who is using WebSphere XD 6.0 to build a robotrading application for their bank. A robotrading application is one where a customer asks to buy or sell a large block of stock and the application then figures out the best way to execute that order, possibly over a couple of days or longer whilst trying to obtain the best price for the shares as well as conceal the identity of the customer. These applications use various strategies to automatically execute these complex trades.
They are using application partitioning (WPF) to scale out the application, provide asymmetric cluster support and provide very fast partition recovery in the event of a server failure. They are using a DB2 database to harden the partition state and xSeries blades to host the application.
However, they also keep transient state (which is fully recoverable) for a partition in POJOs in memory on the JVM. As a transaction is being executed, sometimes the transaction must be rolled back. This is fine for the data in the database as WebSphere makes sure those changes are not committed. The problem is the object state kept in memory. How to undo those changes? They could keep those changes separately from the ‘committed’ state and then merge it after the application is sure the database transaction has committed but this is a lot of work.
Solution, transactional local memory with ObjectGrid
The solution I suggested to them was to use an embedded ObjectGrid in each server to manage the local state. Now, when the application wants to update the in memory state for a partition, it first obtains the state from the local ObjectGrid. The ObjectGrid makes sure that the application is working on a copy and when the transaction is committed, the ObjectGrid makes the changes ‘permanent’ in memory at least. If the transaction rolls back then all changes to the state are discarded. The ObjectGrid can be easily linked to the transaction manager in WebSphere so that WebSphere automatically controls the transaction begin/rollback and commit calls. It can also link to the transaction managers of any other application server.
The ObjectGrid is basically copying the state as the application references it in memory. The application then manipulates the copies and then the transaction is committed then the changes are committed in memory so that the next transaction will find the now updated state to work on. If the transaction rolls back then those changes are discarded.
Distributing shared Objects
Keeping this transient state in the ObjectGrid has other advantages. Peer JVMs are interested in some of the Objects managed by a partition. Maybe, the performance metrics are interesting to other peers or even different applications, possibly running on a tomcat or Geronimo server. Maybe some information on best prices etc. The ObjectGrid can be configured to asynchronously replicate this subset to other JVMs attached to the same grid. This provides transparent distributed of those Objects to peer JVMs that are in need of that data and avoids the application having to write code using JMS etc to manually do this. This solves another problem the customer had without introducing another problem.
So, this shows how the ObjectGrid can be used, not as a cache, but as a local transaction Object manager. It supports services like a replication service that can significantly simplify the coding of a transactional application that uses in memory state for its operation. I guess it also shows how important a little API like ‘rollback’ is. The ObjectGrid is more than just a cache. It’s a framework for managing objects in a transactional fashion. The ObjectGrid added peer to peer replication and are adding client/server supporting using the existing public plug points such as the Loader or by attaching transaction log sniffers to this core framework. These APIs are also available for customers to do what ever they’d like in case what we provide isn’t quite what is needed. This is a big change from the normal way we’d approach middleware in that the ObjectGrid is really flexible. Almost all aspects are customizable and the internals are visible through plugins so that customers can customize it to their needs.
September 28, 2005 | Permalink
In case you have not heard of it yet, I think the following research stuff might interest you. It's also connected somehow to the API vs API-less debate.
Posted by: Alain Rogister | Sep 29, 2005 9:49:02 AM
Billy, just keep it simple and call it a transactional cache (see link ;-)
Posted by: Cameron Purdy | Sep 29, 2005 12:51:56 PM
High folluting indeed :)
I disagree, the state in particular above was a combination of state thats recoverable from the database and ancillary state around it needed by the application. The ancillary state is not backed by the database, it's just state needed by the application singleton. So, its a different use case than a 'cache'. It's transactional memory, memory whose changes can be rolled back which isn't backed by a database. You're right, as usually the case, for memory that is indeed backed by a database etc.
Posted by: Billy | Sep 29, 2005 1:12:22 PM
This is fine for the data in the database as WebSphere makes sure those changes are not committed. The problem is the object state kept in memory. How to undo those changes?
Using an in memory JTA enabled database? Or maybe JBoss Cache ( AFAIK it supports JTA)
Posted by: Horia Muntean | Sep 29, 2005 3:06:09 PM
Trouble with a JTA enabled in memory database is path length, yes you can make it work, but it's not going to be as fast as copying objects which is basically what we do in the ObjectGrid. JBoss cache would be an option in local only mode as would any other transactional cache that runs local with no backend.
Posted by: Billy | Sep 29, 2005 3:44:01 PM
This is a really nice case for AOP type implementation. Just bind against their dataobjects and then based on whatever happens in that objects persistence methods you update your cache.
Posted by: Les | Sep 29, 2005 4:30:12 PM
Sounds like the case you're describing is algorithmic trading. This type of trading used to be limited to the really big mutual fund companies like fidelity, putnam, wellington, etc. Now, the technology is becoming more affordable, so more businesses are starting to explore this area. I have limited experience in this area, but the transactional part actually isn't that hard. Whether the trades are kept in memory or stored in the database doesn't matter too much as long as it's handled correctly.
The more difficult part is having a system that is flexible enough that traders can easily write the trading algorithm in an "analyst friendly format" and deploy it to the systems relatively quickly. Some of the solutions in this space take a business rules approach, while others use sophisticated simulations and mathematics. Thanks for blogging on the topic.
Posted by: Peter Lin | Sep 29, 2005 11:30:14 PM
Very kewl. Neat application. And definitely a good use of ObjectGrid. I said it before and I will say it again. Billy is a good architect and his stuff works. (I am just passionate about API-less approach whenever possible. Not trying to suggest that ObjectGrid was a "bad" idea so fogive the following rant.)
This is a perfect example of AOP. Jonas Bonér actually wrote this exact functionality as a demo during JavaProLive in front of his audience. And then he demo'ed it running by stepping through a method in Eclipse that excplitly set fields on an object and then threw an exception. At the end of the method, the object was back the way it had started...no code, no nothing. The audience then literally asked him to "do it again."
In fact, I think something like transactionality is best suited to a cross-cutting approach since you may need to roll back objects from classes you hadn't realized were changing or you may want certain objects in the graph of a transactional change to actually not roll back. This is the very essence of AOP vs OOP.
You should check it out:
Your customer will be happy in the long run w/ a zero-code [API-less] solution that they can't misuse or break.
Posted by: Ari Zilka | Sep 30, 2005 10:08:22 AM