Serialization and Deserialization

Applications work with messages as normal java objects. These views must be serialized into a provider specific message when sent, and when received they must be deserialized (or wrapped). The Talon Application Data Modeler generates Factory objects that facilitate The application registers the generated factories that it uses with the Talon runtime to allow deserialization of messages as they are received from the messaging provider. 

Message Engine Independent Serializers

...

@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IMyMessage extends extends IRogNode, IRogMessage {

	public final byte [] serializeToByteArray();
	public final ByteBuffer serializeToByteBuffer();
	public final String serializeToJson();
 
	public final void deserializeFromByteArray(byte[] bytes);
	public final void deserializeFromByteBuffer(ByteBuffer bytes);
	public final void deserializeFromJson(String json)
}

Message Metatadata

Anchor
MESSAGE_METADATA MESSAGE_METADATA

When a message is transported over a bus binding, metadata is attached to the message to assist in routing, encoding/decoding, and de-duplication. The mechanism of how this metadata is attached to a message is specific to each bus binding implementation. For cases where you are working with the messaging provider directly (not through a bus binding implementation it useful to understand the structure of message metadata and how it is serialized. 

Field	Type	Description
Channel Identification
MessageChannelName	String	The name of the channel. This is used by the receiver to lookup the channel on which the message should be dispatched to the application.
MessageChannelId	short	The id of the channel. This is used by the receiver to lookup the channel on which the message should be dispatched to the application. Using a channel id allows for more efficient lookup of the channel, but care must be taken that the specified channel id matches that configured for receivers of the message will be flagged as an unhandled message.
Message Encoding Message encoding fields allows a receiving application to lookup a message factory and use it to deserialize or wrap a message.
MessageViewFactory	short	The id of the factory that is used to deserialize the message. This is used by the receiver to lookup the factory that is used to deserialize (or wrap) the received message payload. The message view factory id can be determined by calling MessageView.getVfid()on a message. For ADM modeled messages, this will be the negative of the factory id defined in the model (user factory types are negative at runtime to distinguish between user and platform factory types).
MessageViewType	short	The id of the type (unique to its factory) used to deserialize the message. The message view type can be determined by calling MessageView.getType() on a message. For ADM modeled messages, this will be the id of the message defined in the ADM model.
MessageEncodingType	byte	The encoding type of the message. The encoding type is used by the factory to differentiate between possible encodings supported by the message, but may also be used by wire sniffers as a hint to decoding the message. Valid values are: 3: Protobuf The encoded payload is a series of google protobuf bytes 4: Xbuf The encoded payload is a series of google protobuf bytes which may use extensions employed by talon's protobuf implementat. 5: Json The encoded payload is a utf-8 encoded string with json content. 1: Custom The encoded payload is opaque bytes.
Message Sequencing Message sequencing fields allow for the receiving application to filter duplicate and out of order messages. For a given sender id and flow, an AepEngine will filter out received messages that don't have a monotonically increasing sequence number
MessageSender	int	A globally unique id that identifies the sender of a message. By default an AepEngine uses the hashcode of the engine name as the sender id.
MessageSno	long	A monotonically increasing sequence number.   A sequence number of <=0 indicates that the message is not sequenced. Receivers should not perform duplicate checking on it. A sequence number of 1 indicates that the start (or restart of a stream). A receiver that receives sequence numbers 1,2,3,1,2,3 should not consider the 4th message a duplicate. Otherwise, receivers should consider a sequence number not greater than the previous sequence number as a duplicate.
MessageFlow	int	Indicates the flow to which the message belongs. Flows allow for partitioning of message traffic in conjunction with application state and allow intra application state partitioning. Usage of flows is not currently supported for end users. External senders should always set the flow to 0.

SMA MessageMetadata Class

SMA provides a helper class, com.neeve.sma.MessageMetadata, that can be used by applications to assist in working with message metadata. This helper class provides the ability to serialize data into a binary format and also serves a view around this metadata. Binding such as the Solace binding encode metadata in binary serialized format for optimized transmission, while other lower performance bindings encode metadata as property values in the message header. 

The serialized form is as follows where (encoding is little endian). The current metadata version is V2, and its binary encoding is as follows.  

V2 Serialized Metadata Format

V2 metadata adds the message view type to the metadata. Prior to the introduction of the V2 receiving application were forced to determine the message type by inspecting the serialized message content for the xRogType field. 

Message encoding type

2-3

Message view factory

4-5

Message view type

6-9

Message sender

10-13

Message flow

14-21

Message sequence number

22-23

Message channel id

24-25

26...

V1 Serialized Metadata Format

Message encoding type

2-3

Message view factory

4-7

Message sender

8-11

Message flow

12-19

Message sequence number

20-21

Message channel id

22-23

24...

Sending Messages from External Applications

When integrating with legacy application applications that do not use the platform's message abstraction layer it is useful to understand how to populate transport level message in a manner that a Talon application can receive and deserialize them.  

Solace

The platform's Solace binding transports encoded messages in a BytesMessage. The encoded messages is set as the message data, and the metadata is stored as serialized MessageMetadata in the "x-sma-metadata" property. The example below shows how an external application can send a message natively using the solace JCSMP client:

...

// populate a message
OrderEventMessage orderEvent = OrderEventMessage.create();
orderEvent.setOrderId(1);

// serialize the payload to a byte buffer
byte [] serializedPayload = view.serializeToByteArray();

// prepare message metadata
MessageMetadata metadata = MessageMetadataFactory.getInstance().createMessageMetadata();
metadata.serializeV2(view.getMessageEncodingType(),
                     view.getVfid(),
                     view.getType(),
                     0,  // message sender id
                     0,  // message flow
                     0,  // message sequence number (unsequenced)
                     -1, // channel id (unspecified)
                     XString.create("order-events"));

// prepare solace message
com.solacesystems.jcsmp.BytesMessage message = producer.createBytesMessage();
message.setData(serializedPayload);
message.setDeliveryMode(DeliveryMode.DIRECT);
SDTMap props = producer.createMap();
if (serializedMetadata != null) {
  props.putBytes("x-sma-metadata", serializedMetadata);
}
message.setProperties(props);

// send 
com.solacesystems.jcsmp.Topic topic = producer.createTopic("order-events");
producer.send(message, topic);

// dispose
orderEvent.dispose();
metadata.dispose(); 

JMS

...

// populate a message
OrderEventMessage orderEvent = OrderEventMessage.create();
orderEvent.setOrderId(1);

// serialize the payload to a byte buffer
byte [] serializedPayload = view.serializeToByteArray();

// prepare message metadata
MessageMetadata metadata = MessageMetadataFactory.getInstance().createMessageMetadata();
metadata.serializeV2(view.getMessageEncodingType(),
                     view.getVfid(),
                     view.getType(),
                     0,  // message sender id
                     0,  // message flow
                     0,  // message sequence number (unsequenced)
                     -1, // channel id (unspecified)
                     XString.create("order-events"));

// The jms binding decomposes metadata into message properties:
message.setBooleanProperty(JmsMessageBusBinding.SMA_METADATA_PRESENT_JMSPROP, true);
message.setByteProperty(JmsMessageBusBinding.SMA_METADATA_VERSION_JMSPROP, (byte)metadata.getVersion());
message.setByteProperty(JmsMessageBusBinding.SMA_METADATA_ENCODING_JMSPROP, metadata.getMessageEncodingType());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_VFID_JMSPROP, metadata.getMessageViewFactory());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_VTYPE_JMSPROP, metadata.getMessageViewType());
message.setIntProperty(JmsMessageBusBinding.SMA_METADATA_SENDER_JMSPROP, metadata.getMessageSender());
message.setIntProperty(JmsMessageBusBinding.SMA_METADATA_FLOW_JMSPROP, metadata.getMessageFlow());
message.setLongProperty(JmsMessageBusBinding.SMA_METADATA_SNO_JMSPROP, metadata.getMessageSno());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_CHID_JMSPROP, metadata.getMessageChannelId());
message.setStringProperty(JmsMessageBusBinding.SMA_METADATA_CHNAME_JMSPROP, metadata.getMessageChannelName());

// prepare JMS message
javax.jms.Message message = null;
if(view.getEncodingType == MessageView.ENCODING_TYPE_JSON) {
  message = session.createTextMessage();
  ((TextMessage)message).setText(new String(serializedPayload));
}
else {
   message = session.createBytesMessage();
   ((Bytes)message).setBytes(serializedPayload);
}

// send 
Topic topic = session.createTopic("order-events");
messageProducer.publish(topic, message, DeliveryMode.PERSISTENT, 0, 0);

// dispose
orderEvent.dispose();
metadata.dispose(); 

Receiving Message in External Applications

Solace

The following code sample shows how an external application using the Solace JCSMP api directly can unpack a message sent over the platform's built in JMS binding and materialize it into a message view. 

...

public void onMessage(BytesXMLMessage message) {
  if (!message instanceof if (message instanceof BytesMessage) {
     // not an x message. 
     handleNonXMessage(message);
     return;
  }
  BytesMessage message = (BytesMessage) message); 
 
  // extract sma metadata
  final SDTMap props = message.getProperties();
  final ByteBuffer serializedMetadata = null;
  if (props != null) {
    // metadata
    serializedMetadataBytes = props.getBytes("x-sma-metadata");
    if (serializedMetadataBytes != null) {
      serializedMetadata = ByteBuffer.wrap(serializedMetadataBytes);
    }
  }
 
  if(serializedMetadata == null) {
     // not an x message. 
     handleNonXMessage(message);
     return;
  }
  final byte encodingType = MessageMetadata.getMessageEncodingType(serializedMetadata);
  final short vfid = MessageMetadata.getMessageViewFactory(serializedMetadata)
  final short vtype = MessageMetadata.getMessageViewType(serializedMetadata)
 
  // extract payload
  byte [] payloadBytes = ((BytesMessage)message).getData();
  Object payload = payloadBytes ;
  if(encodingType == MessageView.ENCODING_TYPE_JSON) {
     payload = new String(payloadBytes);
  }
 
  // lookup factory
  // (assumes that view factory have already been registered)
  MessageViewFactory factory = viewFactoryRegistry.getMessageViewFactory(vfid);
  if (factory != null) {
    MessageView view = factory.wrap(vtype, encodingType, message);
    if(view instanceof OrderEventMessage) {
      handleOrderEvent((OrderEventMessage) message);
    }
  }
  else {
    handleNonXMessage(message); 
    return;
  }
}

JMS

The following code sample shows how an external  application using the JMS api directly can unpack a message sent over the platform's built in JMS binding and materialize it into a message view. 

...

public void onMessage(javax.jms.Message message) {
  // extract sma metadata
  final boolean isMetadataPresent = message.getBooleanProperty(SMA_METADATA_PRESENT_JMSPROP);
  if (!isMetadataPresent) {
     // not an x message. 
     handleNonXMessage(message);
  }
  final byte encodingType = message.getByteProperty(SMA_METADATA_ENCODING_JMSPROP);
  final short vfid = message.getShortProperty(SMA_METADATA_VFID_JMSPROP);
  final short vtype = version > MessageMetadata.V1 ? message.getShortProperty(SMA_METADATA_VTYPE_JMSPROP) : 0;
 
  // extract payload
  Object payload = null;
  if(message instanceof TextMessage) {
     payload = ((TextMessage) message).getText(); 
  }
  else if(message instanceof BytesMessage) {
     payload = ((BytesMessage) message).getBytes(); 
  }
  else {
    handleNonXMessage(message);
    return;
  } 
 
  // lookup factory
  // (assumes that view factory have already been registered)
  MessageViewFactory factory = viewFactoryRegistry.getMessageViewFactory(vfid);
  if (factory != null) {
    MessageView view = factory.wrap(vtype, encodingType, payload);
    if(view instanceof OrderEventMessage) {
      handleOrderEvent((OrderEventMessage) message);
    }
  }
  else {
    handleNonXMessage(message);
    return;
  }
}

Each bus binding implementation will transport the message in its byte encoded form along with the metadata described above using a mechanism specific to the message bus providers messaging API.

Refer to the message bus binding provider documentation for each bus for details.