Overview

Coding For State Replication

In the ticker example, the Feeder and the Enricher both use state replication. We'll focus on the Enricher class to demonstrate how to setup state replication in an X application. The Enricher's functionality is pretty straightforward. It receives tick updates from the Feeder on the Tick channel, stores the received updates in a queue which it uses to determine the symbol's historical trend, and then enriches the received Tick messages with its calculated trend info and sends it off on the EnhancedTicks channel where the receiver is listening. The key point here is that the Enricher is operating using the ApplicationState that we modeled in the previous sections, and the platform is transparently handling replication under the covers. In the event of an application failure (such as the one the Sample Induces), processing can resume where it left off: the application state and in doubt state of any messages are transparently and atomically resolved on the backup.

The following sections walk through the code of the Enricher sample discussing the steps for configuring the engine.

Configuring the Engine

Step 1: Register Object Factories

In this sample the same factory class is used for both messages and state. An instance of the object factory has to be registered with both the MessageViewFactoryRegistry and the StoreObjectFactoryRegistry. The latter registry allows the underlying state replication machinery to deserialize replicated state objects based on the ids encoded in the replication stream while the former is used for deserializing replicated messages. In the sample this is done in a Common superclass used by all the applications since this step is common to all of them:

Step 2: Set up the Store

The object store is what handles state replication. The native store implementation forms clusters with other stores of the same name using a discovery protocol and elects a Primary based on a peer election algorithm. In the snippet below we create a store cluster called "Enricher" (NAME), and save it to the configuration repository.

Step 3: Configure the Engine Descriptor

The snippet below shows the configuration of the Enricher's engine descriptor. We create a new descriptor and set the HAPolicy to use StateReplication. When using state replicaton you must also use a MessageSendPolicy of ReplicateBeforeSend which indicates that state must be replicated before outbound messages are sent. This ensures that messages and state are replicated to backups prior to sending messages downstream. This is crucial in state replication because it ensure that inbound messages receipt and mutations to state are stabilized prior to committing the send of outbound messages that result from those mutations.

Finally, we specify a ReplicationPolicy of Pipelined to ensure that replicated state is acknowledged by our backup prior to sending the enriched message on to the Receiver to ensure that we don't produce duplicates.

Unsolicited Senders

For applications that use state replication with an unsolicited sender (i.e. those not done from a message handler) such as the Feeder app, the application should additionally configure the engine to replicate unsolicited sends so that state and messages are replicated for the sender. The Feeder is such an application, so it additionally sets this value to true:

Step 4: Provide a StateFactory

An AEPEngine is application state agnostic: it deals with state only abstractly as a replicated graph of object graph node updates using the underlying store. As such it needs to be able to create new application state during the processing of messages. To facilitate this, an engine's constructor takes a IApplicationStateFactory parameter as shown from the Enricher snippet below.

Configuration Summary

The table below lists the configuration options we have discussed so far:

Creation of Application State

Application state is always created by the AEP engine and is created in one of three different ways:

1. By a primary engine when it receives the first message of a flow in which case it utilizes the application provided IApplicationStateFactory.
2. By a backup engine when application state is replicated from a primary.
3. Explicitly by an application on an AepMessagingPrestartEvent handler.

There are two constraints that need to be met when the state is created explicitly by the application

1. For performance reasons, the state creation is not thread safe. Thus, the state must be created before the engine's messaging has been started.
2. The state needs to be created after the engine has bound to the applications's ODS store. This needs to be done so that the created state is attached to the opened store.

The first constraint implies that the state must be created before the engine has started messaging. The second constraint implies that the state needs to be created during or after start (since the engine opens the store during start()) i.e. the only time the state can be explicitly created by the application is between start() and the receipt of the messaging started event. The messaging prestart event is such a point in the flow.

Interacting With Application State

From a message handler

The below snippet comes from the Enricher message handler. From with a message handler application state should be retrieved using the Engine's getApplicationState(MessageView) accessor which retrieves the state associated with the message's flow. In the Ticker sample the flow is defined by the hashCode of the symbol and is set by the Feeder. Flows define the order in which messages are processed in the system. Flows essentially partition traffic into ordered parallel processed streams and, therefore, enable parallelized, concurrent message processing.
Under the covers the Engine ensures that both messages and state are replicated to the backup to prevent loss or duplication in the event of a failure.

Unsolicited Sends

The Feeder operates slightly differently. Because it is the origin point of the traffic it can't retrieve its state from the Engine using a MessageView, but as mentioned previously Creation of State can only be done from an engine thread. It must therefore create its state from within an AepMessagingPrestartEvent handler. If the Feeder were to operate with multiple flows (and consequently multiple application state instances, it would need to create them all here.

The sample application is simple and operates with a single flow based on the symbol. For simplicity we use the hashcode of the symbol name to map this to an integer id representing the flow. The sender creates the state and stores it in a private variable _state for use during runtime. The created state is not threadsafe, so in this context in can only be used by the feeder's thread will sending its messages.

Limitations and Upcoming Enhancements

There are few limitations to current state model all of which are under consideration for a future version.

Single Parent Restriction

Currently an entity may only be placed in the state graph as the field of a single other object. Using the same object instance in the graph in multiple locations is not supported:

State Graph Cycles

As a corollary to the Single Parent Restriction, cycles in the object graph are not supported (included self references).

Support of graph cycles of non field entities is planned for a future of the platform. 

Multiple Entity Fields of Same Type

Transactional entities don't currently support more multiple fields of the same entity type. The following is not supported due to current limitations in the transaction machinery. 

Inheritance

Support for Inheritance is under consideration for a future platform version.

Generic Collections Support and Inline Collections

Support for inline collection types will be added in a future version of the platform. These use the generic collection implementation built into the platform. Collections must be parameterized with the types that they hold by following the collection type with a bracketenclosed comma separated list of the types.

Support for Annotation Based Modeling

The following support for annotation based state modeling is planned in an upcoming release.

Annotations

@Persist

This annotation specifies that a type should be instrumented for persistence. It supplies factory name, factoryId, type Id and transactional attributes.

@Transient

This annotation can be applied to fields in a @Persistent annotated class to indicate that the field should not be instrumented for persistence. It is also possible to declare the field with a transient modifier rather than using this annotation they both have the same effect.

The Ticker Sample on Annotations

The annotated version of the Ticker sample codes would look like the following. Recall from the XML sample that we modeled the Application state as a single entity called ApplicationState with a sequence number of type Long, and a collection that was a Queue to hold the past ticks received by the Enricher. In the annotated analog we simply write a class with these two fields as fields in the class, and mark it with a @Persist annotation that specifies the object factory to use for the entities that the class defines:

At compile time the ADM annotation processor then instruments these classes along with some additional supporting source files. Notably, it creates the source code for the TickerObjectFactory that we declared above which will need to be registered at run time. Under the covers the code generator generated a shadow class that serves as the backing replicated object for replication of the ApplicationState and is used rematerialize the ApplicationState during recovery or replication.

Field Types supported for persistence

Only types supported by the ADM Model will be instrumented for replication by the annotation processor. 

Annotation Processing

The Annotation Processor is included in the platform's adm jar. The javac compiler should automatically run the processor at compile time using the default java 6 discovery process. If you need to manually control annotation processing the annotation processor is com.neeve.adm.annotations.Processor.