Overview

...

State Replication

...

In the ticker example, the Feeder and the Enricher both use state replication. We'll focus on the Enricher class to demonstrate how to setup state replication in an X application. The Enricher's functionality is pretty straightforward. It receives tick updates from the Feeder on the Tick channel, stores the received updates in a queue which it uses to determine the symbol's historical trend, and then enriches the received Tick messages with its calculated trend info and sends it off on the EnhancedTicks channel where the receiver is listening. The key point is that the Enricher is operating using the ApplicationState that we modeled in the previous sections, and the platform is transparently handling replication under the covers. In the event of an application failure (such as the one that the Sample Induces), processing can resume where it left off: the application state and in doubt state of any messages are transparently and atomically resolved on the backup.

The following sections walk through the code of the Enricher sample, discussing the steps for configuring the engine.

Configuring the Engine

Step 1: Register Object Factories

In this sample, the same factory class is used for both messages and state. An instance of the object factory has to be registered with both the MessageViewFactoryRegistry and the StoreObjectFactoryRegistry. The latter registry allows the underlying state replication machinery to deserialize replicated state objects based on the ids encoded in the replication stream, while the former registry is used for deserializing replicated messages. In the sample, this is done in a Common superclass used by all the applications since this step is common to all of them.

// for messaging
MessageViewFactoryRegistry.getInstance().registerMessageViewFactory(new ObjectFactory()); 
// for state replication:
StoreObjectFactoryRegistry.getInstance(). 
registerObjectFactory(new ObjectFactory());

Step 2: Set up the Store

The object store handles state replication. The native store implementation forms clusters with other stores of the same name using a discovery protocol and it elects a Primary based on a peer election algorithm. In the snippet below we create a store cluster called "Enricher" (NAME), and save it to the configuration repository.

// store descriptor
StoreDescriptor storeDescriptor = StoreDescriptor.create(NAME);
storeDescriptor.setProviderConfig("native://.");
storeDescriptor.save();

Step 3: Configure the Engine Descriptor

The snippet below shows the configuration of the Enricher's engine descriptor. We create a new descriptor and set the HAPolicy to use StateReplication. When you use state replication, you must also use a MessageSendPolicy of ReplicateBeforeSend which indicates that state must be replicated before outbound messages are sent. This ensures that messages and state are replicated to backups prior to sending messages downstream. This is crucial in state replication because it ensures that inbound messages receipt and mutations to state are stabilized prior to committing the send of outbound messages that result from those mutations.

Finally, we specify a ReplicationPolicy of Pipelined to ensure that replicated state is acknowledged by our backup prior to sending the enriched message on to the Receiver. This ensures that we don't produce duplicates.

AepEngineDescriptor descriptor = AepEngineDescriptor.create(NAME);
descriptor.setHAPolicy(AepEngine.HAPolicy.StateReplication);
descriptor.setMessageSendPolicy(AepEngine.MessageSendPolicy.ReplicateBeforeSend);
descriptor.setReplicationPolicy(ReplicationPolicy.Pipelined);
descriptor.setStore(NAME);

Unsolicited Senders

For applications that use state replication with an unsolicited sender (i.e. those not done from a message handler) such as the Feeder app, the application should additionally configure the engine to replicate unsolicited sends so that state and messages are replicated for the sender. The Feeder is such an application, so it additionally sets this value to true:

descriptor.setReplicateUnsolicitedSends(true);

Step 4: Provide a StateFactory

An AEPEngine is application state agnostic: it deals with state only abstractly as a replicated graph of object graph node updates using the underlying store. As such, it needs to be able to create new application state during the processing of messages. To facilitate this, an engine's constructor takes a IApplicationStateFactory parameter as shown from the Enricher snippet below.

_engine = AepEngine.create(descriptor,
    new IAepApplicationStateFactory() {
       final public ApplicationState createState(final MessageView view) {
         return ApplicationState.create();
       }
    },
    handlers, null, null); 

Configuration Summary

The table below lists the configuration options we have discussed so far.

is the simpler of Talon's 2 HA models. With State Replication Talon replicates changes to state and the outbound messages emitted by your application's message handlers. In the event of failover to a backup your application's state is available at the same point where processing left off, and the engine will retransmit any outbound messages that were left in doubt as a result of the failure. 

Coding For State Replication

A basic state replication application is straightforward. A quick way to get started is to use the nvx-talon-sr-processor-archetype described in Maven Archetype Quick Starts. The general flow for creating a state replication application involves:

• Modeling Messages and State
• Declaring a main class annotated for StateReplication
• Providing Talon with a state factory for creating your application state
• Writing message handlers to perform business logic. 

A Basic App

Model Application State

The steps outlined below assume that you have already modeled some messages and state for your application to use. See the Modeling Message and State sections to get started. 

Annotate App Main for State Replication

@AppHAPolicy(value=AepEngine.HAPolicy.StateReplication)
public class Application {
 ...
}

Provide a StateFactory

An AEPEngine is application state agnostic: it deals with your application an plain old java object graph with a single root. Given the root object for your application's state the underlying store will track changes made to fields on the root (and its descendants) and replicate or persist those changes. As such, an AepEngine needs to be able to create a new application state during when your application is initialized. This is done by finding a method on your main application class anotated with @AppStateFactoryAccessor. The state factory accessor returns a newly initialized set of state for the engine to manage. As messages are processed the engine will pass the relevant state root back into the application to be operated upon. 

@AppStateFactoryAccessor
final public IAepApplicationStateFactory getStateFactory() {
    return new IAepApplicationStateFactory() {
        @Override
        final public Repository createState(MessageView view) {
            return Repository.create();
        }
    };
}

As your application makes changes to this root object (setting fields etc), the engine will monitor the root and replicate deltas to backup members or disk. 

Inject Message Sender

If your application will send messages, it can add an injection point for the underlying AEPEngine to inject a message sender.

private AepMessageSender messageSender;
 
@AppInjectionPoint
final public void setMessageSender(AepMessageSender messageSender) {
    this.messageSender = messageSender;
}

Declare Message Handlers

When working with an application using StateReplication, the underlying AepEngine will pass in the root object for your application state along with the message. Outbound message sends and state changes are managed by Talon as an atomic unit of work. 

@EventHandler
final public void onMessage(Message message, Repository repository) {
    // update state
    repository.setCounter(repository.getCounter() + 1);
    // send event
    Event event = Event.create();
    event.setVal(message.getVal());
    event.setCount(repository.getCounter());
    messageSender.sendMessage("events", event);
}

Configuring the Engine

Register Object Factories

When working with state replication both the ADM and message and state object factories need to be registered with the runtime. Registering the state factory allows the underlying state replication machinery to deserialize replicated state objects based on the ids encoded in the replication stream. The message factories allow are used for deserializing replicated outbound messages as well as messages received from message buses. The state factories can be declared in your config.xml or programmatically:

<app name="processor" mainClass="com.sample.Application">
  <messaging>
    <factories>
      <factory name="com.sample.messages.MessageFactory"/>
    </factories>
  </messaging>
  <storage enabled="true">
    <factories>
      <factory name="com.sample.state.StateFactory" />
    </factories>
 </storage>
</app>

@AppInjectionPoint
public void initialize(AepEngine engine) {
  // for messaging
  engine.registerFactory(new com.sample.messages.MessageFactory()); 
  // for state replication:
  engine.registerFactory(new com.sample.state.StateFactory());
}

Enable Storage

To actually achieve high availability storage must be configured for the application. The primary means of storage is for Talon apps is through clustered replication to a backup instance. Talon also logs state changes to a disk based transaction log as a fallback mechanism. Storage and persistence can be enabled in the application's configuration xml. 

<app name="processor" mainClass="com.sample.Application">
  ...
  <storage enabled="true">
    ....
    <clustering enabled="true"/>
    <persistence enabled="true">
      <!-- 
        When using Xbuf encoded entities,
        detached persist is not supported.
        -->
      <detachedPersist enabled="false"/>
    </persistence>
 </storage>
</app>

Enabling clustering allows 2 applications of the same name to discover one another and form an HA cluster. When one or more instances of an application connect to one another one instance is elected as the primary via a leadership election algorithm. The primary member will establish messaging connections and begin invoking message handlers in your application. 

Enabling persistence causes the replication stream that is sent to backup instances to also be logged locally on disk to a transaction log file. The transaction log file can be used to recover application state from a cold start, and is also used to initialize new cluster members that connect to it so when clustering is enabled, persistence must be enabled as well. 

There are many configuration knobs that can be used to customize the store's behavior and performance. See DDL Config Reference for a listing of configuration knobs for storage.

Unsolicited Senders

For applications that use state replication with an unsolicited sender (i.e. those not done from a message handler) such as the Feeder app, the application should additionally configure the engine to replicate unsolicited sends so that state and messages are replicated for the sender. The Feeder is such an application, so it additionally sets this value to true:

descriptor.setReplicateUnsolicitedSends(true);

Creation of Application State

Application state is always created by the AEP engine and is created in one of three different ways:

...