October 27, 2005
Pluggable index support in ObjectGrid
The ObjectGrid shipping in Jan (V6.0.1) has support for the indexing of Objects in the ObjectGrid by attributes on the Object. It has two kinds of index built in and you can build your own index implementation if you wish. For example, if you want to index Objects using time based queries or use geospatial queries on those Objects then we provide all the events needed to plugin such an index implementation as a first class index just like the ones we provide.
Defining an index
You need an index implementation. This is an Object that implements MapIndexPlugin. This object can be added to a BackingMap before the ObjectGrid in initialized in the JVM. Any changes to the Map are then passed to the index using the applyUpdate callback. This provides a list of all changes to the Map (inserts, updates, deletes). The index should then keep it's data structures up to date.
Using an index
Applications that wish to use the ObjectGrid, first get a session (ObjectGrid#getSession) and then ask the Session for ObjectMaps for each Map they need to access on that Session. Once the application has an ObjectMap then it can ask the ObjectMap for a reference to an index defined on the BackingMap for that Map.
Each session is a transaction and may have changes pending inside it. An application making queries on an index would expect the index to incorporate these changes. When the application asks the ObjectMap for the index, the framework calls the MapIndexPlugin#getIndexProxy method. The plugin is also given a MapIndexInfo reference. This reference allows the plugin to obtain all changes pending in the current session/transaction. The index plugin should then create a composite index incorporated the committed changes as well as the pending ones. This is actually tricky as the commited values are continually changing in the background because of other threads. The index must be written carefully to work in this environment. We don't block other sessions because of the index. The supplied index implementations use an optimistic scheme that maximises concurrency. They won't block other threads while the index is being used.
The application then casts the returned index proxy to be the right interface. Different indexes can have different methods like findBetween or findCloseToCoords or whatever. So, thats why we return an Object for the index proxy. The application then interacts with the index proxy and can issue simple programmatic queries like find all objects < value or > value as well as between x and y. We don't offer a query language such as Jofti but Jofti could be easily plugged in as an index implementation.
If the application gets a reference to the index then it should use it before making further changes as the index will likely just use the current pending changes when the index was created. I guess we could in the future, create another index automagically when the data in the session is modified to make this more transparent. But, for now, the index will snapshot the session at the time the index was returned to the application.
It was interesting reading the documentation for Jofti but it's a shame that we don't have standard APIs to allow indexes to be attached to cache implementations. For now, I think the ObjectGrid offers very good extensibility in that we pretty much allow most aspects of it to be delegated to an ISV or customer.
October 27, 2005 | Permalink