The Talon Manual

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Excerpt

A goal of the platform is to make working with messages and state as simple as working with plain old java objects. The X Platform's Application Data Modeler (ADM) allows modeling of messages and state entities in XML and provides utilities to generate source code that is instrumented to transparently handle encoding, serialization and transactional functionality that underpin the platform. This section and its subsections will familiarize you with theses modeling constructs and code generation tools.

The ADM model allows the definition of three types of complex or composite types: Entities, Embedded Entities, and Messages. These will be discussed in more detail below. Here is a high-level definition of the three.

...

  • Array types are limited to primitive, built-in, and embedded entity types. They are intended to allow embedded values that are transported along with a message or entity, and consequently:
  • Queue, Set and Queue and Map types can only be used with non-embedded entities and may only use other non-embedded entities as their values.

...

Messages can be thought of as a special type of entity that additionally implement implements and exposes a MessageView interface allowing them to be transported via the platform's Messaging layer. Because messages must be transportable as a single serialized unit, messages can only declare fields that are primitive, built-in, embedded entities or arrays of the previous; non embedded entities and collections aren't supported.

Messages can be used independently from the Messaging Engine and Message Store.

...

The X Platform generates source code for entities and messages for use with the runtime from an xml XML model specified by the x-adml.xsd schema included at the root of the nvx-adm-<version>.jar and also bundled into the nvx-all.jar. Be sure to update your editor's schema validator to reference it.

Tip

If you are working in an IDE such as eclipseEclipse, try importing the ADM xsd XSD schema into your eclipse XML catalog so that you you can get usage tips on the ADM model by pressing control-space.

The x-adml.xsd is published online with each release and also included at the root of talon jars containing ADM. Download the file for the platform version you are using to your computer and reference it:

...

The root element of a message model is the model element, which is used to define a namespace qualified set of modeling elements. To conform with xto x-adml.xsd schema and pass validation, a model must define target XML namespace xmlns="http://www.neeveresearch.com/schema/x-adml". This is not to be confused with the namespace attribute described below.

...

AttributeDescriptionRequired
nameThe Model name can now be specified in the model element itself instead of being supplied externally. If the name contains spaces, then in cases where it used as type name it will be converted to camel case with no spaces (for example "Trading model" would become "TradingModel"No
namespaceThe model's name spacenamespace. Model elements use the model name space namespace as the package name when generating other models, and model importing another model refer to imported model elements using the imported model's namespace.Yes
defaultFactoryIdThe default factory id to be used on model elements that require a factoryId but don't specify one. If the model doesn't define a factory element with a matching id, one is created implicitly by camel casing the model name and appending 'Factory', e.g. TradingFactory.No
docA brief one-line description of the model.No

...

ADM model has a name property which is used in certain ways during code generation.

Two main uses of a model name are:

  • Name of .proto file when IDL is generated.
  • Name of outer java class that wraps model types when code is generated with Protobuf encoding.

...

For the example above, name property will have a value of ModelNameGoesHere (it will be converted to Pascal Notation). If we do not define name attribute, the model name would be derived from filename: sampleModel.xml -> SampleModel.

Note
titleNaming the model

Conversion The conversion of a user-defined name to an internally used ADM name can only handle white space as a word separator. Since name will be used to declare a Java class, the developer must take care with when specifying a model name to only use white space and characters that are allowed in a class name. 
If we named the model sample-model, the resulting ADM name would be Sample-model, which cannot be used as a Java class name because it contains a dash character. Generating code with Protobuf encoding would then cause Java compilation errors. This applies both to name defined through XML attribute and filename-derived ones.

...

Most model elements can be documented by supplying a child <documentation> element or doc attributes. The documentation added to the model is used to generate javadoc Javadoc in the generated source code.

...

Types in an ADM Model can be marked as deprecated via a deprecation attribute. Such elements are generated with deprecation annotions annotations and javadocJavadoc. Adding a child deprecation element optionally allows providing a 'since' attribute and brief message describing the reason for deprecation. 

...

The import statement allows you to import messages entities and fields from a model in a different namespace. 

...

Note
titleMixing encodings via import

Types from a model imported must be generated with the same encoding type as the model importing them. It is not possible to mix and match different encoding types within a message or entity. So if MessageA is generated with Xbuf and embedded EntityB is generated with Protobuf, Message cannot use EntityB as a field.

Status
colourGreen
titleSINCE 3.4

When generating code and copying model to output, encoding information is written to target model XML as a directive. If the model for which code is generated and any of its imports have an encoding mismatch, the code generator will raise a model validation error. For imported models that do not have encoding info (resolved directly to OS path or packaged to jar with earlier versions of ADM), this validation is not enforced.

...

The factories section defines object factories that are used to instantiate the generated object. Each factory in an application must have a unique factory id. The ID is serialized along with the object or transported in MessageMetadata and is used to reconstitute its objects during deserialization. A single model can define multiple factories, allowing Messages and Entities to be grouped together as the application sees fit. A factory can contain a maximum of 32767 types, so in practice, it is rarely a requirement to use multiple factories within a single model. Each type in the model defines the factory to which it belongs via its factoryid attribute. User The user application may define factories with ids greater than or equal to 1, ids <= 0 are reserved for platform internal use. 

...

AttributeDescriptionRequired
nameThe factory name to be used for code generation.No
id

The factory's id which is used to register the factory with the runtime and identify the factory to use when decoding the factory's serialized objects.

The factory id must be between 1 and 65536 inclusive. Values of 0 or less are reserved for use by the platform.

Yes
docA brief one-line description for the factory. If more detailed documentation is needed, a child <documentation> element can be used.No
deprecatedWhen a factory is marked as deprecated it will be marked as deprecated by the code generator.No

...

Enumeration types can be modeled as follows and can be declared as fields of entity or message types. Unlike other model elements, enumerations aren't tied to a particular factory despite being scoped to the model's name spacenamespace. Generated java enumeration donenumerations don't have dependencies on the rest of the platform and can be used standalone. 

...

Generated Enumerations can specify a type which is accessible in the generated code. The type can be int, char or String. A const should not be removed from an enumeration, if the newly generated code is expected to deserialize enums that were persisted with an earlier version, in addition, the order of constants is important and should not be changed.

...

The ADM model allows fields either to be defined in place (directly on the message or entity that is using the field) or by reference to a field declared in the model's <fields> section. It is a matter of preference which approach an application developer uses: if many messages contain the same field, then it may be more convenient to model the fields in a reusable fashion in the <fields> element, but in most cases, it is more convenient to define the fields in place. 

...

Code Block
<!-- Defines fields which may be referenced by other model elements-->
<fields>
	<field name="myStringField" type="String" id="10000" length="16" doc="My String field" />
</fields>
 
<messages>
	<message name="Message" factoryid="1" id="1">
    	<fieldRef  ref="myStringField" name="myField"required="true" />
    	<field name="myOtherField" type="String" length="16" id=10001" doc="My Other String field" />
	</message>
</messages> 
Field Attributes

...

The type of the element. If the type is defined in this namespace or is a primitive or collection type, only the simple name of the type need be used. If the type belongs to another namespace from an imported file, then the fully qualified name should be used. The best practice when importing a type from another model is to use the fully qualified name of the imported type (e.g. com.example.importedmodel.MyEntity), as this insulates your model from future conflicts in the event that imported models are changed.

If the field is an arraytype, it should be suffixed with array indices such as MyEntity[] to denote it as an array.

...

Contains the name of the JSON property that will be used for the field when the message is serialized to JSON. Defaults to use the value defined in name.

 

...

The id of the field. The id must be unique with the scope of the containing type. For Xbuf/Protobuf encoding this tag is used as the tag value for the field on the wire. If not set, a unique id will be generated by the source code generator. For better control over compatibility it is reccomended that the application set this value manually.

Field ids must be between 0 and 32767 inclusive

...

If this field refers to a variable length type (such as string), this indicates the maximum length of the field.

...

Note
titleQualifying conflicting type names

It is not possible to define two types with the same name in a given model. One exception to this rule is that it is possible (though strongly discouraged) to define a type with the same name as a built-in type. In this case, using an unqualified type reference will favor the built-in type. In this case, it is possible to reference the local type by using the this keyword, or the fully qualified name:

Code Block
xml
xml
<model xmlns="http://www.neeveresearch.com/schema/x-adml" 
       xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
       name="Trading" 
       namespace="com.mycompany.trading" 
       defaultFactoryId="100">
       doc="Contains Messages used by the Trading Application">
 
  <import model="../orderprocessing.xml"/>
 
  <!--An unfortunately named entity:-->
  <entity name="Currency" id="1">
    <field name="currencyCode" type="String"/>
  </entity>
 
  <entity name="MyEntity" id="2">
    <!-- 
      Without qualifying the Currency type, the built 
      in Currency type is used, and the generated entity
      will return a java.util.Currency.
    -->
    <field name="javaCurrency" type="Currency" />
    <!-- 
      The 'this' keyword indicates that this is the Currency
      entity defined in this model:
    -->
    <field name="currencyEntity" type="this.Currency"/>
    <!-- 
      Namespace qualifying the typeindicates that this is the Currency
      entity defined in this model:
    -->
    <field name="currencyEntity2" type="com.mycompany.trading.Currency"/>
    <!-- 
      Namespace qualifying the type indicates that this is the Currency
      entity defined in some imported model:
    -->
    <field name="currencyEntity3" type="com.mycompany.orderprocessing.Currency"/>
  </entity>
</model>

...

AttributeDescriptionRequired
refThe name of the referenced field. Name may be with or without the namespace of owner model (i.e. full-qualified name). In both cases, the field will be looked up in both the current model and its imports. If the non-fully-qualified name is declared in the current model and in one of the imports, the field from the current model will take precedence. If multiple imports define the same name, an error will be reported. The best practice when importing a field from another model is to use the fully qualified name of the imported field (e.g. com.example.importedmodel.someField), as this insulates your model from future conflicts in the event that imported models are changed.(tick)
nameOptionally, a name for the field that overrides the referenced field's name. Otherwise, the field name defaults to that of the referenced field. 
jsonNameOptionally overrides the referenced field's jsonName. 
id

Optionally overrides the id specified by the field being referenced.

Field ids must be between 0 and 32767 inclusive

 
requiredWhether or not the field is a required field for the type. Fields declared as required will cause a check to be added for the value being set in the generated types isValid() method. 
pinned

Indicates whether or not the field is pinned. A pinned field always occupies the same space on the wire.

 
isKey

True if this field should serve as the key when inserting the entity into a LongMap or StringMap collection. In this case, insertion of the message in the collection with a given key will update this field with the key value.

 
docThe doc to use for the field which overrides that specified by the referenced field. 
deprecatedWhen true all methods generated for the field will be marked as deprecated. 

Generated Field Accessors

...

List Accessors for array types

When xbuf Xbuf encoding is used for an array type and pooling is enabled, the underlying implementation doesn't store the field value in an array because pooling varying array sizes is impractical. To achieve zero garbage for fields modeled as arrays, a List interface is more appropriate, and an XIndexedList is returned. An A XIndexedList implements java.util.List, but also provides reusable iterators. These accessors are discussed in detail in the section Zero Garbage Array Accessors.

...

For zero garbage access on date fields, it is possible to use the 'AsTimestamp' accessors to access the field as the number of milliseconds since the epoch. 

...

To avoid garbage for embedded messages entities when using Xbuf encoding, while still providing a mechanism for applications to hold such fields in state beyond the lifespan of the message from whence they came, additional take/lend accessors are generated for embedded entity fields. Considerations for using embedded entity fields in the context of message pooling are discussed in detail in the knowledge base article at: http://docs.neeveresearch.com/display/KB/Zero+Garbage+Nested+Entitiessection Embedded Entities.

Code Block
java
java
@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IMyMessage extends extends IRogNode, IRogMessage {


    final public void setChildField(final Child2 val);
    final public Child2 getChildField();
 
    final public void lendChildField(final Child2 val);
    final public Child2 takeChildField();}

...

Type
Unset Field Return Value
hasXXX
Notes
Java PrimitiveDefault The default value for java primitive type(tick)For numeric values 0, for boolean false, for char ''.
String, Date,null(tick) 
Enumnull(tick) 
Entitynull(tick) 
ArrayEmpty Array

(error)

Depends on code gen configuration property generateArrayGetterEmptyIfNull

For java primitive types, the hasXXX() method can be called to distinguish whether the value was explicitly set or is just returning the default value.

For arrays, it is not possible to distinguish between an empty array or the array not being set. This allows ADM code to operate with Protobuf encoding under the covers, which is not able to distinguish between an empty or unset array. For this reason, ADM generated code does not generate a hasXXX method for arrays. Setting the code generation property generateArrayGetterEmptyIfNull controls whether an empty or null array is returned when there are no values encoded in the underlying message or entity. 

...

  • The field or type element that defines it has a poolable="true" attribute. The String type will be created based on the name of the field. 
  • A semantic type is defined with a base type of string and the poolable attribute set to true. The String type will be based on the name of the type. 
  • For applications that would rather separate the notion of pooling from the model itself, the Code Generator can be run with a directive indicating that all String fields should generate a poolable type (see Directives in the sections below).
  • In the future, different narrower directives may be introduced for finer-grained control over what String fields will create poolable types.

...

  1. Preallocate the pooled string type with preallocated application state, and copy the string from the inbound message into domain state. In this approach, the message keeps its reference to the pooled string field and reuses the same reference when the message is recycled through its pool. 
  2. Use take/lend semantics in the same fashion as with a Nested Entity as described in Xbuf and Embedded Entities. The domain state needn't preallocate the pooled string type but rather takes if from the message which then draws a new instance from a globally configured preallocated when the message itself is recycled to its pool.  

...

  Default
<qualifiedClassName>.initialLengthThe initial length of the backing buffer to allocate to back the string in serialized form.taken from length attribute if specified in the model, otherwise 1
<qualifiedClassName>.isNativeWhether or not a native backing buffer should be used. This is recommend recommended for performance sensitive apps.true
<qualifiedClassName>.pool.enabledWhether the factory is backed by a pool or if new instances are created on demand.true
<qualifiedClassName>.pool.threadedWhether or not the backing pool is thread-safe. The pool must be thread safe if more than one thread will create strings from this factory, or, crucially, if a separate thread may return the item to the pool. For the default pool, this value should almost always be true, because the thread creating the string and the one that ultimately disposed of it are usually different.true
<qualifiedClassName>.pool.preallocateCountThe number of objects to preallocate, or 0 if no preallocation should be done. Note that preallocation is not done until the class is loaded.0

...

If the application will handle allocating Strings used in application state up front , and will copy values from messages into application state, then the application should create a separate Factory that preallocates the expected number of types. This can be done with the Factory's newFactory(...) method:

...

AttributeDescriptionRequired
name

The name of the field, must be unique within the model.

(tick)
id

The id of the entity, which must be unique within the scope of all types in the entity's factory.

Entity / Message ids must be between 0 and 32767 inclusive

(tick)
factoryIdThe id of the factory, which must be unique within the scope of all factories used within an application. When a message is received, the factoryId and entityId uniquely identify the type to be deserialized. factory ids IDs <= 0 are reserved for platform use.(tick)
asEmbeddedDefaults to false, Indicates whether or not this entity is generated to be used as an embedded or child field of another entity. Embedded entities are always serialized transported with its parent entity. Entities used as fields in messages must be declared as embedded. 
transactional

Whether or not this entity supports transaction commit and rollback via the applications ODS store.

 

...

AttributeDescriptionRequired
type

The type of the element. If the type is defined in this namespace, is defined in only one of the imported models, or is a primitive or collection type, only the simple name of the type need be used. Non A non-fully-qualified name will be looked up in both the current model and its imports. If name is declared in the current model and in one of the imports, the type from the current model will take precedence. If multiple imports define the same name, an error will be reported. Best The best correction in such case is to use the fully-qualified name.

 If If the field is an arraytypearray type, it should be suffixed with array indices such as MyEntity[] to denote it as an array.

(tick)
nameThe name of the field, must be unique within the model.(tick)
jsonName

Contains the name of the JSON property that will be used for the field when the message is serialized to JSON. Defaults to use using the value defined in name.

 
id

The id of the field. The id must be unique with the scope of this entity. For Xbuf/Protobuf encoding, this tag is used as the tag value for the field on the wire. If not set, a unique id will be generated by the source code generator. For better control over compatibility, it is recommended that application set this value manually.

Field ids must be between 0 and 32767 inclusive

 
length

If this field refers to a variable length type (such as string), this indicates the maximum length of the field.

 
pinnedA pinned field always occupies the same space on the wire. 

isKey

True if this field should serve as the key when inserting the entity into a LongMap or StringMap collection. In this case, insertion of the message in the collection with a given key will update this field with the key value. 

...

Messages are defined in the messages element and must have a unique id with respect to other types within the scope of their factory. Messages can use all of modeling capabilities of entities, but cannot use non-embedded entities or non-array collections as fields.

...

For Xbuf and Protobuf encoding, the id is used to generate the protobuf field tag, so changing field ids will break wire compatibility. If fields are not explicitly assigned ids IDs in the model, then the ADM generate will assign them automatically. in this case, fields should not be removed or changed in order to maintain wire compatibility with earlier version versions of the generated code. 

Messages can only declare primitive types, built-in types, embedded types as fields (or arrays of those types). Collections, Messages, and non-embedded Entities can't be used.

Generated Source

For an entity named "MyMessage" an interface and an implementation will be generated using the model's namespace. Entities will extend IRogNode or IRogMessage, marking that they can be used as nodes with the platform's Replicated Object Graph (ROG) framework and the platform's Simple Message Abstraction (SMA) Layer.

...

Collections are defined in the collections <collections> element and must also have a unique id with respect to other types within the scope of their factory. Collections may not be declared as embedded at this time. 

...

Code Block
languagexml
<collections>
  <collection name="MyQueue" is="Queue" contains="MyEntity" 
    factoryid="1" id="5" />
  <collection name="MySetMyLongMap" iscontains="SetMyEntity" containsis="MyEntityLongMap" 
    factoryid="1" id="6" />
</collections> 
<entity <collection name="MyLongMap" contains="MyEntity" is="LongMap" 
    factoryid="1" id="71" />
 <field <collection name="MyStringMaplongKey" containstype="CollectionObjectLong" isisKey="LongMaptrue" 
    factoryidid="1" id="8" />
</collections>  <entity<field name="MyEntityaQueue" factoryidtype="1MyQueue" id="12"/>
 <field name="aQueueaMap" type="MyQueue"/>
 <field nameMyStringMap id="aMap3" type="MyStringMap/>
</entity> 

Generated Source

Code Block
java
java
@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IChildLongMap extends IRogLongMap<Child1>, Map<Long, Child1> {}
 
@Generated(value="com.neeve.adm.AdmXbufGenerator", date="Fri Jan 23 02:03:22 PST 2015")
final public class ChildLongMap extends RogLongMap<Child1> implements IChildLongMap, IXbufDesyncer, IRogJsonizable {...}

...

Code Block
xml
xml
<!-- Defines fields which may be referenced by other model elements-->
<fields>
	<field name="myStringField" type="String" id="10000">
<fields>
 
<entities>
	<entity name="MyEmbeddedEntity" asEmbedded="true" factoryid="1" id="2000">
    	<fieldRef ref="myStringField" id="2000">
    	<field name="myIntField" type="Integer" id="2001" required="true">
	</entity>
	<entity name="MyOtherEmbeddedEntity" factoryid="1" id="2001">
    	<field name="myDateField" type="Date" id="2002"/>
    	<field name="myBooleanField" type="Boolean" id="2003"/>
    	<field  name="myBooleanField" type="Boolean" id="2004"/>
	</entity>
</entities>
 
<messages>
	<message name="Message" factoryid="1" id="1">
		<inline ref="MyEmbeddedEntity"/>
        <inline ref="MOtherEmbeddedEntityMyOtherEmbeddedEntity"/>
		<field name="myEnumField" type="MyEnumeration"/>
	    <!--  Override required and isKey fields on the myIntField inlined from MyEmbeddedEntity -->
    	<field name="myIntField" type="Integer" id="2001" required="false", isKey="true">
	</message>
</messages>

...

  • An embedded entity cannot be inlined into another embedded entity in a manner that will produce inlining cycles.
  • An embedded entity cannot inline a Message or Entity.
  • Messages and non-embedded entites entities cannot be inlined.
  • A Message can't inline 2 embedded entities that have conflicting field definitions (e.g. fields with the same name but with attributes that differ in any way other than their doc or required attributes). In the case of required, if an embedded entity defines a field as required, then it will be required on the message.
  • The isKey and pinned field attributes on an embedded entity are ignored when the entity is inlined into another message or entity. 
  • A Message can't define fields that conflict with any of its inlined embedded entities (e.g. fields with the same name but with attributes that differ in any way other than their doc or required attributes). 
    Exceptions to this rule are:
    • required: In the case of the required attribute, the value set on a message field wins (a particular message can override the required field attribute on an inlined embedded entity).
    • isKeyField: entity or message inlining the embedded entity can set the isKey attribute for a given inlined field. 
    • pinned: the pinned attribute can be overridden by the inlinining inlining entity. 

Generated Source

Note that javadoc Javadoc is omitted below and only the simple accessors are listed for brevity.

...

  1. Don't change the ids of factories. 
  2. Don't change the ids of types. 
    1. If a type will no longer be used, it is better to deprecate it to avoid accidentally declaring a new type the reuses the old type id later. 
  3. Declare the id attribute on fields. 
    1. If you don't declare field ids, ADM will assign them in the order they are declared, but this means that any reordering or removal of the field would result in different type ids. 
  4. Don't remove fields on messages.
    1. It is better to deprecate a field rather than remove it. This avoids the potential for accidentally reusing the field id. 
  5. Don't change the type of a field. If a field type needs to be changed, deprecate the existing field and add a new field with the same type. 
    1. It is generally okay to change a scalar type such as a byte to a short, int or long, or a float to a double, but not vice-versa.
  6. Don't change the name of fields.
    1. For Json JSON encodings, changing the field name is a breaking change.
    2. For Xbuf and Protobuf encodings, it is technically okay to change the name of the field as long as the id remains the same. 

...

There are a few additional compatibility issues to consider if the generated code will be shared between multiple projects which is . These are covered in Runtime Compatibility.

...