The Talon Manual

Skip to end of metadata
Go to start of metadata

In This Section

Serialization and Deserialization

Applications work with messages as normal java objects. These views must be serialized into a provider specific message when sent, and when received they must be deserialized (or wrapped). The Talon Application Data Modeler generates Factory objects that facilitate The application registers the generated factories that it uses with the Talon runtime to allow deserialization of messages as they are received from the messaging provider. 

Message Engine Independent Serializers

Generated Embedded Entities and Messages can be used independently of the Messaging Engine and support serializers/deserializers to byte [], ByteBuffer and json.

Message Metatadata

When a message is transported over a bus binding, metadata is attached to the message to assist in routing, encoding/decoding, and de-duplication. The mechanism of how this metadata is attached to a message is specific to each bus binding implementation. For cases where you are working with the messaging provider directly (not through a bus binding implementation it useful to understand the structure of message metadata and how it is serialized. 

FieldTypeDescription
Channel Identification
MessageChannelNameString

The name of the channel.

This is used by the receiver to lookup the channel on which the message should be dispatched to the application.

MessageChannelIdshort

The id of the channel.

This is used by the receiver to lookup the channel on which the message should be dispatched to the application. Using a channel id allows for more efficient lookup of the channel, but care must be taken that the specified channel id matches that configured for receivers of the message will be flagged as an unhandled message.

Message Encoding
Message encoding fields allows a receiving application to lookup a message factory and use it to deserialize or wrap a message.
MessageViewFactoryshort

The id of the factory that is used to deserialize the message.

This is used by the receiver to lookup the factory that is used to deserialize (or wrap) the received message payload.

The message view factory id can be determined by calling MessageView.getVfid()on a message. For ADM modeled messages, this will be the negative of the factory id defined in the model (user factory types are negative at runtime to distinguish between user and platform factory types).

MessageViewTypeshort

The id of the type (unique to its factory) used to deserialize the message.

The message view type can be determined by calling MessageView.getType() on a message. For ADM modeled messages, this will be the id of the message defined in the ADM model.

MessageEncodingTypebyte 

The encoding type of the message.

The encoding type is used by the factory to differentiate between possible encodings supported by the message, but may also be used by wire sniffers as a hint to decoding the message. Valid values are:

  • 3: Protobuf The encoded payload is a series of google protobuf bytes
  • 4: Xbuf The encoded payload is a series of google protobuf bytes which may use extensions employed by talon's protobuf implementat.
  • 5: Json The encoded payload is a utf-8 encoded string with json content.
  • 1: Custom The encoded payload is opaque bytes.
Message Sequencing
Message sequencing fields allow for the receiving application to filter duplicate and out of order messages. For a given sender id and flow, an AepEngine will filter out received messages that don't have a monotonically increasing sequence number
MessageSenderintA globally unique id that identifies the sender of a message.

By default an AepEngine uses the hashcode of the engine name as the sender id.  
MessageSnolong

A monotonically increasing sequence number.  

  • A sequence number of <=0 indicates that the message is not sequenced. Receivers should not perform duplicate checking on it.
  • A sequence number of 1 indicates that the start (or restart of a stream). A receiver that receives sequence numbers 1,2,3,1,2,3 should not consider the 4th message a duplicate.
  • Otherwise, receivers should consider a sequence number not greater than the previous sequence number as a duplicate.  
MessageFlowintIndicates the flow to which the message belongs. Flows allow for partitioning of message traffic in conjunction with application state and allow intra application state partitioning.

(warning) Usage of flows is not currently supported for end users. External senders should always set the flow to 0.

SMA MessageMetadata Class

SMA provides a helper class, com.neeve.sma.MessageMetadata, that can be used by applications to assist in working with message metadata. This helper class provides the ability to serialize data into a binary format and also serves a view around this metadata. Binding such as the Solace binding encode metadata in binary serialized format for optimized transmission, while other lower performance bindings encode metadata as property values in the message header. 

The serialized form is as follows where (encoding is little endian). The current metadata version is V2, and its binary encoding is as follows.  

V2 Serialized Metadata Format

V2 metadata adds the message view type to the metadata. Prior to the introduction of the V2 receiving application were forced to determine the message type by inspecting the serialized message content for the xRogType field. 

Byte OffsetDescription
02 (version)
1
Message encoding type
2-3
Message view factory
4-5
Message view type
6-9
Message sender
10-13
Message flow
14-21
Message sequence number
22-23
Message channel id
24-25
number of characters in channel name (0 if channel name not included)
26...
ascii encoded characters in channel name

V1 Serialized Metadata Format

Byte Offset

Description

01 (version)
1
Message encoding type
2-3
Message view factory
4-7
Message sender
8-11
Message flow
12-19
Message sequence number
20-21
Message channel id
22-23
number of characters in channel name (0 if channel name not included)
24...
ascii encoded characters in channel name

Sending Messages from External Applications

When integrating with legacy applications that do not use the platform's message abstraction layer it is useful to understand how to populate transport level message in a manner that a Talon application can receive and deserialize them. Each bus binding implementation will transport the message in its byte encoded form along with the metadata described above using a mechanism specific to the message bus providers messaging API.

Refer to the message bus binding provider documentation for each bus for details. 

  • Executor BindingThe Executor Bus Binding is a special bus binding that allows applications to provide send work (via a message) to be processed on a separate thread. The executor binding can be used to perform processor intensive work in a thread other than an application's main dispatch thread, or can be used as a means to implement outbound gateways in which the executing thread 'pushes' the sent message to an external system. Work done by the processor of an executor bus is acknowledged and therefore Guarant
  • JMS BindingThe JMS binding works with JMS 1.1 level JMS clients and is configured using JNDI lookup. In addition to JNDI lookup the platform provides a subclassed version of the JMS binding optimized for working with ActiveMQ. 
  • Kafka BindingThe Kafka provides messaging connectivity with Kafka Brokers. It is not currently documented as it is still in an incubation phase. If you are interested in using the Kafka binding, let us know mailto:contact@neeveresearch.com.
  • Loopback BindingThe platform's loopback binding can be used to send to other applications running in the same process, not across different processes. This binding is often used for unit testing in which several applications may be launched in the same process for easy debugging.
  • Solace BindingThe platform's solace binding includes both a java based and JNI-based implementation and allows connectivity to a solace message router https://solace.com/products/message-routers.

 

  • No labels