The Talon Manual

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
div
idtoc
classtoc
div
classtocTitle

In This Section

div
classtocContents

Table of Contents
maxLevel2
indent8px
stylenone

The figure below outlines the high level components that make up a Talon application. 

 

 

Talon Development Artifacts

From an application developer's perspective, authoring an X application involves:

...

Code Block
languagexml
titleData Model Sample
<?xml version="1.0"?>
<model xmlns="http://www.neeveresearch.com/schema/x-adml" 
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
       namespace="com.example.helloapp"
       defaultFactoryId="1">
    <factories>
      <factory name="HelloDataFactory" id="1"/>
    </factories>
    <messages>
        <message name="HelloRequest" id="1">
            <field name="fromName" type="String" id = "1"/>
        </message>
        <message name="HelloReply" id="2">
            <field name="text" type="String" id = "1"/>
            <field name="count" type="Long" id = "2"/>
        </message>
    </messages>
    
    <entities>
      <entity name="MyState" id="100">
        <field name="counter" type="Long" id = "1"/>
      </entity>
    </entities>
</model>

 

See Modeling Message and State for more information on modeling messages and state.

Application Event Handlers

...

Code Block
titleAn Example App
@AppHAPolicy(value = StateReplication)
public class HelloApp {
    private AepMessageSender sender;

    /**
     * Injected by Talon at creation time. 
     */
    @AppInjectionPoint
    public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }
 
    /**
     * Called by Talon on receipt of a HelloRequest message. 
     */
    @EventHandler
    public void onMessage(HelloRequest helloRequest, MyState state) {
        // update state
        state.setCounter(state.getCounter() + 1);
        // send hello reply
        HelloReply helloReply = HelloReply.create();
        helloReply.setText("Hi There");
        messageSender.sendMessage("hello-replies", helloReply);
    }
 
    /**
     * Invoked by Talon the very first time a message is received
     * by the application to create clean state. From that point forward
     * Talon will ensure that changes to state aren't lost in the event
     * of failover of restart. 
     */
    @AppStateFactoryAccessor
    public IAepApplicationStateFactory getStateFactory() {
        return new IAepApplicationStateFactory() {
            @Override
            final public MyState createState(MessageView view) {
                return MyState.create();
            }
        };
    }
}

For See Programming Talon Applications for more information on writing event handlers, see the Talon Programming section. 

Configuration

X Application code doesn't work directly with the underlying infrastructure components. For example, binding of an application to a particular message bus implementation is a configuration concern. The platform's Domain Descriptor Language (DDL) is the platform's configuration schema. It allows developers to configure a group of related applications in a singe xml-based configuration artifact. Deployment tools such as Robin use this configuration to seed the the platform's configuration repository, where it is then used by Talon VMs when they are launched. 

Code Block
xml
xml
<model>
    <buses>
        <bus name="hello-bus">
			<provider>solace</provider>
            <host>192.168.1.100</host>
            <port>55555</port>
            <channels>
                <channel name="hello-requests">
                   <key>hello/requests</key>
                   <qos>BestEffort</key>qos>
                </channel>
			    <channel name="hello-replies">
                   <key>hello/replies</key>
                   <qos>BestEffort</key>qos>
                </channel>
            </channels>
        </bus>
    </buses>

	<apps>
		<app name="hello-app" mainClass="com.example.HelloApp">
			<messaging>
                <factories>
                  <factory name="com.example.messages.HelloMessageFactory"/>
                </factories>
				<buses>
					  <bus name="hello-bus">
            		    <channels>
            	  	    <channel name="hello-requests" join="true"/>
            	  	  <channel name="hello-replies" join="false"/>
               	    </channels>
				  </bus>
				</buses>
			</messaging>
		</app>
	</env>apps>
      <servers>
	    <server name="hello-vm">
            <apps>
                <app name="hello-app" autoStart="true" />
            </apps>
        </server>
    </servers>
</model>

Because the platform handles much of the non-functional operational aspects under the covers, many of the knobs it exposes don't have an impact on application code. Therefore, an application developer often won't need to work with much more than the configuration specified above.

See Understanding Configuration for more information about configuration.

Talon Runtime Components

Atomic Event Processor (AEP)

...

  • Replication: The primary means of achieving persistence for an ODS Store is by pipelined, memory-memory replication of transactions to one or more hot backup peers. If a primary application instance fails the backup takes over with no loss of data.
  • Transaction Logs: An ODS Store can also log to binary transaction logs as a secondary persistence mechanism if the connection to the backup is lost or a cold restart is required. 
  • Change Data Capture (CDC): To allow asynchronous yet transactionally consistent syphoning of application state to back end or legacy systems, ODS supports the ability to perform Change Data Capture with a simple callback-based mechanism to push state changes. 
  • Inter Cluster Replication (ICR): Inter-Cluster Replication allows replication of the store's recovery stream to another data center over messaging, providing an asynchronous and transactional consistent disaster recovery mechanism. 

See Application Storage

XVM

 A A Talon XVM serves as a container for Talon micro apps and provides management capabilities for the XVMs apps that it contains. Key features of the Talon XVM include:

  • Lifecycle management of applications.
  • Alerting and Lifecycle Event emission. 
  • Emission of heartbeats that contain detailed system, platform and application exposed statistics for low-impact remote application monitoring.
  • Emission of application trace for low-impact remote application monitoring.
  • Command and Control; XVMs provide capabilities for exposing and invoking commands that are out of band with application messaging. 

See The XVM

Discovery

The platform's discovery facilities provides plugin mechanisms for ODS store's to discover one another, and for tooling to discovery discover running instances of applications. Out of the box, the platform supports discovery of over multicast and over messaging. 

Talon Application Flow

From an application perspective Talon's application flow is not dissimilar to many other event or message processing architectures: the application exposes message handlers to the platform which in turn passes inbound messages to it for processing. In the act of processing the inbound event, the application will make changes to its state and send some outbound events. The key difference between the Talon architecture and traditional architectures is that application state is stored in memory and resiliency is provided not by synchronous persistence to disk or a data grid, but instead by streaming state changes to a backup's memory in an asynchronous, pipelined fashion. The combination of in memory state and asynchronous 'persistence' allows Talon to operate at extreme performance levels without sacrificing on reliability. 

Key aspects of any streaming application platform that are of critical concern for stream oriented applications are:

  • Exactly once processing of inbound and outbound messages.
  • Atomicity between state updates and the messaging stream: A trading application that thinks it has sent a request to buy 100,000 shares of IBM must be able to rely on that being what is actually sent out. This is particularly important in application or machine failure scenarios. If the application fails after sending out the request to buy 100,000 IBM and the application were to recover and reprocess the event this time buying 200,000 shares it could turn into a costly business problem. 

In traditional application architectures the problems of state/messaging atomicity and exactly once processing are often solved using distributed transactions. For example, in J2EE state changes would be committed to a databases and messaging sent over JMS with XA transactions used to coordinate commit on both. Such schemes involve a lot of overhead and kill performance. The application flow in Talon provides the same level of reliability without the synchronous overhead of distributed transaction coordination. The following diagram depicts the application flow in a Talon application:

Image Added

Elaborating on the diagram above, Talon ensures the same level of reliability as traditional architectures as follows:

  1. Talon receives an inbound message
  2. It dispatches the message to the application
  3. Application updates state (monitored by the Talon) 
  4. Application sends Messages (through Talon)
  5. Talon replicates the state changes and outbound messages to one or more hot backup instances via memory to memory replication. 
  6. Once the state and oubound effects (messages) are stabilized on a backup (via an asynchronous stablitity acknowledgement), outbound messages are released. 
  7. Acknowledgement of outbound messages allows cleanup up indoubt state
  8. Acknowledgement of inbound message which has not been fully processed. 

Key Takeaways

  • Stabilization of state changes and outbound effects to the backup before sending outbound messages ensures that in the event of process or machine failure that the backup has the same conception of state as the former primary communicated externally (e.g. I have an outstanding request to buy 100,000 shares of IBM). 
  • Acknowledgement of the inbound message after replication to the peer ensures that in the event of failure a duplicate can be detected to prevent duplicate dispatch to the application if the message is re-transmitted. 
  • The entire flow above is pipelined. The application can begin processing the next inbound message before receiving stability from the backup. Nowhere in the flow is there a need to block or perform synchronous operations. 
  • State and outbound messages can be journalled to a transaction log on disk, but it is not fsync'd and is not a primary recovery mechanism. Disk based journalling is used more for operational tasks, but can also serve as a backup recovery mechanism. 
  • And best of all ... the above is done transparently to the application which need only concern itself with writing the business logic in message handlers.