...
The data modeler supports generating objects with 3 diffferent ecodings, 4 different encodings - Json, Protobuf, Xbuf and XbufXbuf2. From an API perspective the generated interfaces are functionally equivalend equivalent (with a few exceptions), but each encoding has different performance characteristics that are described in the sections below.
...
Note also that all encoding types allow a message to be serialized to or from json as a secondary functionality.
Protobuf
With protobuf endoding encoding objects are create with backing google protobuf generated objects. Protobuf is suitable for applications with higher performance requirements than is afforded by Json encoding. It should be used by applications with moderate to high performance requirements.
...
- Google protobuf generated messages are not zero garbage, and the serialized Protobuf messages are technically reusable so for very low latency applications that aren't using the Platform's zing distribution, this can result in garbage related pauses.
...
Xbuf
Xbuf generated objects, are backed by the X Platform's high performance implementation of Google protobufs which supports zero garbage operation and cut-through serialization (the ability to read/write fields directly to from a backing buffer). It should be used for applications with the most stringent performance requirements.
...
- Accessors for String and Uuid arrays fields in Xbuf are not currently zero garbage. Consequently, Xbuf may be more expensive than Protobuf for types that use array fields.
...
Xbuf2
| Status | ||||
|---|---|---|---|---|
|
Xbuf is the next version of the Xbuf encoding type. It is 100% wire compatible with Protobuf and Xbuf but offers considerable improvements in performance and memory efficiency over both these encoding types.
...
| Note | ||
|---|---|---|
| ||
Xbuf2 is the intended replacement of Xbuf. It is presented in the current version as a separate encoding type only because the API of the generated Xbuf2 classes have some minor compatibilities with the Xbuf generated classes (see below). Therefore, Xbuf2 is being offered as a separate encoding type in the current version. The next major Talon version will only contain the Xbuf encoding type which will be the current Xbuf2 encoding renamed to Xbuf. Talon applications that currently use the Xbuf encoding type, particularly those that use the Xbuf API methods not supported by Xbuf2 (see below), are strongly encouraged to move to Xbuf2 to ease the migration to the next major version of Talon |
...
- All the benefits of Xbuf listed above plus the following
- Fastest serialization of all the encoding types
- Significantly lower memory footprint than the Xbuf
- Lower memory footprint than the Protobuf encoding type
- Between being faster, having a lower memory footprint and being zero garbage, Xbuf2 is a better choice as an encoding type than Protobuf across all dimensions. As a result, the Protobuf encoding is likely to be deprecated or removed in the next major Talon release.
- Offers several knobs to manage the tradeoff between performance and memory conservation
- Offers multiple data access patterns
- Random access (as is offered by the other encoding types)
- Serial access
- Direct Deserialization: The ability to serially traverse a Google Protobuf encoded buffer and dispatch the fields to the application via a callback
- Direct Serialization: The ability for an application to directly serialize application fields into a buffer in the Google Protobuf wire format
- Optimized for both messages and state
...
- Xbuf2 stores field data off-heap. This can result in more complexities in the following areas
- Troubleshooting issues
- Monitoring memory utilization
- Performing capacity planning particularly related to memory utilization.
- Xbuf2 generated classes are larger.
Known Limitations:
- Not currently supported for State Replication applications
- Does not support the following field types
- UUID
- UUID[]
- Currency
- Currency[]
...
- An application may not hold onto an XBuf/Xbuf2 encoded message beyond the scope of a message handler.
- An application may not hold onto an XString , or Embedded Entity type or embedded entity type from a message beyond the scope of a message handler because these objects are pooled along with the message and will be reset once the message is returned to its pool. See Zero Garbage Nested Entities for detailed usage, but the general rule of thumb is to copy any entity that needs to be retained by applications state, or to use the more advanced 'take' apis. Note that 'take' is not supported for String fields as string fields are not pooled for Xbuf messages.
- Setting an XString or Embedded Entity field or embedded entity field on a message transfers ownership to the message. If the application wants to retain the entity is application state, then it should copy it into a new entity or use the more advanced 'lend' apis. See Zero Garbage Nested Entities for more details.
- An application may not mutate a returned array type from a message and should not hold onto to the array beyond the duration of a message handler. See Zero Garbage Array Accessors for more details.