The Talon Manual

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

In This Section

Overview

Xbuf is the X Platform's zero garbage, optimized implementation of Google Protocol Buffers. It allows generating ADM objects (messages and entities) that act as a view into the backing protobuf encoded binary content. This section discusses some the additional advanced configuration knobs that are available XBuf generated types.In addition to providing a means of working with messages in a zero garbage fashion, Xbuf backed ADM views also support the ability to change the manner in which message or deserialized when read from the wire or disk. The manner in which deserialization is done is important because while protobuf is a compact format on the wire it can be costly to decode or "desync" the fields from a message to make them available to application accessors. The sections below discuss the following available desync policies:

  • Copy: the default desync policy, which walks the messages backing buffer, decodes the field values and copies them into the ADM generated POJO as fields that can be read by an application.
  • FrameFields: with this policy the code desyncing the message walks the buffer and stores pointers to each field in the backing buffer. As the application calls getters on the ADM POJO the fields are deserialized from their pointer to the backing buffer. 
  • FrameContainer: with this policy the backing buffer is wrapped in the ADM view which isn't walked at all. With this policy the caller can't see any of the fields. This policy can be useful for applications working with messages as binary blobs. 

 

Xbuf Desync Policies

Copy

This policy deserializes by copying the contents of fields from the container's backing buffer into field private data structures. Using this policy set the desynced fields to be read-write, but in practice, the wrapped message view around the message is marked as read only and will not allow writes. The read-write capability is important, however, if a received message is copy() via a MessageView.copy() operation. In this case the copied Message's fields will be writeable. 

FrameFields

This policy deserializes by framing fields at the offsets into the container's backing buffer where the serialized form of the fields start. Using this policy sets the framed fields to read-only and causes field getters to deserialize and return field contents directly from the backing buffer. 

Because the protobuf binary format is variable length, fields on messages deserialized with a FrameFields policy cannot be modified, even if the message is copied.

FrameContainer

This policy deserializes by framing the entire message view around the backing buffer. Using this policy sets the late sync boundary to the end of the supplied buffer. The act of setting fields post deserialization using this policy will result in the set value being syncd (serialized) after the end of the supplied buffer. 

The ability to append additional fields to message desynced with FrameContainer is still an experimental feature.

Setting desync policies

As Xbuf types are pooled, configuration of desync policies is a global setting per type. The desync policy can be configured either programatically or through the environment. 

Programatic Configuration

Your application can at startup set desync policies on a type by type basis programattically by calling a static setter on either a Message or Entity type: 


DDL Based Configuration

Desync policies can be set using the envrionment property nv.xbuf.<fullyqualifiedclassname>.desyncpolicy=[Copy|FrameFields|FrameContaine]. In DDL this would look like:

XBuf Pooling Configuration

Each message and entity generated with Xbuf encoding is poolable. By default when the platform is optimized for throughput or latency (e,g. via the nv.optimizefor=latency) environment parameter), message and entity pooling is enabled. Pooling parameters for Xbuf generated types can be configured as environment properties using nv.xbuf.<fullyqualifiedobjectname>.pool.<propertyName> where:

propertyName is one of the bean properties on UtlPool.Params:

  • initialCapacity
  • maxCapacity
  • threaded
  • preallocate
  • detachedWash

For example:

or in the DDL environment section

Message and Entities themselves are actually pooled in tandem with the underlying low level native platform packet. When looking at pool stats emitted by the platform you will notice that their pool names are of the form packet.MessageTypeXbufPacket.<factoryid>.<viewid>.<poolcounter>. Consequently, xbuf pools for non embedded entities can also be configured using the platform general pool configuration parameters for their backing packets. The primary motivation for allow Xbuf pools to be configured in the above fashion is to allow configuring them in a single place. If configuration of both packet and the xbuf pool described here are defined, the packet pool configuration will take higher precedence.

  • No labels