This release provides the ability for applications to rollback states changes and outbound message to application defined "savepoints".

See AEP Transactions for details.

This release provides the ability to include message type latency stats in xvm heartbeats. To include message type latency stats in hearbeats the stats must be enabled for the corresponding app and the server must be configured to emit them:

<apps>
  <app name="my-app" mainClass="com.acme.MyApp">
    <captureTransactionLatencyStats>true</captureTransactionLatencyStats>
    <captureMessageTypeStats>true</captureMessageTypeStats>
  </app>
</apps>
<servers>
  <server name="my-xvm">
    <heartbeats enabled="true" interval="5">
      <collectSeriesStats>true</collectSeriesStats>
      <includeMessageTypeStats>false</includeMessageTypeStats>
      <tracing enabled="true"></tracing>
    </heartbeats>
  </server>
</servers>

See the DDL Schema Reference for more information

When using dynamic substitution for channel keys, we will replace non digit or letter characters with an '_' when resolving a topic using substitution values at runtime if nv.sma.cleanmessagekey == true. Prior to 3.7 X would not perform the same substitution on the subscribe side, which could lead to channels being subscribed to incorrect topics. As of 3.7, this substitution is also performed on the subscribe side by default if nv.sma.cleanmessagekey == true.
Compatibility Notes:

This change will replace non letter and digit characters in channel keys when 'nv.sma.cleanmessagekey=true'. To preserve the old behavior set 'nv.sma.cleanchannelfilter=false'

This release adds the ability to configure an AepEngine such that it calls MessageView.sync() on injected messages. This feature can save the AepEngine cpu cycles in cases where serializes the message for logging or replications.

See DDL Config Reference: syncInjectedMessages (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-\<syncInjectedMessages\>) for additional details.

This release ads the ability to enable solace binding orphan subscription checks introduced in 3.5 for JCSMP in the JNI binding.

See The Solace Binding documentation for more details.
Compatibility Notes:

When using JNI is it critical that you have the most up to date version of the nvx-native runtime (3.6.11).

This release adds copying accessors for embedded entities to ADM generated messages and entities of the form getXXXTo(Entity target), and also creates copy() and copyInto(Entity target) methods on generated entities.

In Xbuf and Protobuf generated code the copy() operations preserve pass-through field value for received entities that have not been modified (a pass-through field is a field not defined in the generated code, but encoded on the wire by a sender). Additionally, in Xbuf generated code the copy is zero garbage.

Prior to this change applications could:

• use the clone() method to copy an embedded entity in a zero garbage fashion, but does not preserve pass-through fields.
  or
• use EmbeddedEntity.deserializeFrom(other.serializeToByteBuffer()); which preserves pass-through fields, but is not zero garbage.

The new copy() methods provide a way to do a pass-through field preserving copy that is also zero garbage.

See Xbuf Embedded Entities

This release provides the ability to globally configure the initial backing buffers size for embedded xbuf entities by setting the property:

For example:

-Dnv.xbuf.com.myapp.messages.Address.initialbackingbuffersize=32

or in DDL

<env>
    <nv>
      <xbuf>
        <com.myapp.messages>
          <Address.initialbackingbuffersize>32</Address.initialbackingbuffersize>
          <Recipient.initialbackingbuffersize>32</Recipient.initialbackingbuffersize>
        </com.myapp.state>
      </xbuf>
  </nv>
</env>

To take advantage of this functionality code regeneration is required. When not configured, the default backing buffer size of 256 bytes will continue to be used.

This release adds a new 'rewrite' command to the TransactionLogTool which writes the contents of a query into new transaction logs. This command can be used to remove 'poison' transaction log entries to allow recovery in situations where the removal of the event doesn't affect recoverability of subsequent entries.

See Transaction Log Tool Command Usage

This release adds the ability to configure the nv.server.autostop.onlastappstop using the new autoStopOnLastAppStop element in DDL.

See DDL Schema Reference (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-\<autoStopOnLastAppStop\>)

This story adds the ability to trigger a thread dump via a talon admin command (available through lumino or robin admin clients).

This story adds the ability to trigger garbage collection via a talon admin command (available through lumino or robin admin clients).

This release adds the ability to configure nv.server.adminclient.outputqueue.capacity via an adminClientOutputQueueCapacity element on the server in DDL.

See DDL Config Reference (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-\<adminClientOutputQueueCapacity\>)

This release makes the MessageBusBindingFailPolicy a DDL configurable option. See the DDL Reference (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-\<messageBusBindingFailPolicy\>)

This release makes the MessagingStartFailPolicy a DDL configurable option. See the DDL Reference (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-\<messagingStartFailPolicy \>)

This release provides experimental support for configuring pools at runtime via the environment or DDL config.

This can be achieved by setting properties of the form "nv.pool.<poolKey>.propertyName" where

poolKey is:
the name of the pool as reported in pool stats without the trailing pool ".<instanceId>" suffix.

propertyName is one of the bean properties on UtlPool.Params:

• initialCapacity
• maxCapacity
• threaded
• preallocate
• detachedWash

For example, pool stats may show stats for the packet backing an XBuf generated message as "packet.MyMessageXbufPacket.71.1.216". The key to use for the pool would thus be "packet.MyMessageXbufPacket.71.1", and the pool could be configured for preallocation in DDL as:

<env>
    <!-- pool parameters -->
    <nv>
      <pool>
        <packet.MyMessageXbufPacket.71.1>
           <initialCapacity>16384</initialCapacity>
           <preallocate>true</preallocate>
        </packet.MyMessageXbufPacket.71.1>
      </pool>
    </nv>
</env>

This release introduced the ability to configure pools for xbuf generated messages and entities at runtime in the same way that initial backing buffer sizes can be configured (introduced in 3.5)

Pooling parameters for Xbuf can be configured as environment properties using nv.xbuf.<fullyqualifiedobjectname>.pool.<propertyName> where:

*propertyName* is one of the bean properties on UtlPool.Params:

• initialCapacity
• maxCapacity
• threaded
• preallocate
• detachedWash

For example:

-Dnv.xbuf.com.myapp.messages.MyEntity.maxCapacity=1024

or in the DDL environment section:

<env>
  <nv>
    <xbuf>
      <com.myapp.messages>
        <MyEntity>
          <initialbackingbuffersize>32</initialbackingbuffersize>
          <pool>
             <maxCapacity>1024</maxCapacity>
             <preallocate>true</preallocate>
          </pool>
        </MyMessage>
      </com.myapp.messages>
    </xbuf>
  </nv>
</env>

Runtime configuration done for xbuf pools takes lower precedence than the general pool configuration mechanism introduced in XPLATFORM-1742. The primary motivation for allow Xbuf pools to be configured in the above fashion

This release provides the ability to record message and transaction stats on per transaction basis in a transaction log file. At the expense of additional processing overhead, this provides a means by which more detailed analysis of transaction latencies can be performed than those emitted in histographical summary in server heartbeats.

See Also:

• Per Transaction Stats
  
• Enabling Per Transaction Stats

This release now records the time at which Messages are enqueued on an application's input queue. EnqueueTs is set for any message queued for an application including those received from a message a bus or injected by the application. Like preProcessingTs and outTs, the timestamp is recorded in microseconds. A message's getEnqueueTs() accessor returns the value in milliseconds since the epoch and its getEnqueueTsMicros() returns the value in microseconds since the epoch.

Like preProcessingTs, enqueueTs is set on the Primary application instance and is persisted in transaction logs and replicated to backup instances. It is not updated when recovering from a transaction log or on the backup.

This release provides better names for threads created by the solace API. The solace api doesn't provide a mechanism for specifying meaningful thread names. In previous releases the Neeve Solace binding would append "X-SMA-<busname>IN" to the solace reactor thread on receipt of the first message. This release now also sets appends "X-SMA<busname>-OUT" on receipt of the first ack to provide a more meaningful name to the producer session thread.

This release allows global configuration of xbuf generated types desync policy. The desync policy can now be globally configured via:

nv.xbuf.default.desyncpolicy=Copy

or on a type by type basis via:

nv.xbuf.typename.desyncpolicy=FrameFields

for example in DDL one might set:

<model>
  <env>
    <nv.xbuf>

      <default>
        <desyncpolicy>Copy</desyncpolicy>
      <default>

      <com.acme.MyMessage>
        <desyncpolicy>FrameFields</desyncpolicy>
      </com.acme.MyMessage>

    </nv.xbuf>
  </env>
</model>

Where the valid values are one of Copy, FrameFields or FrameContainer.

This release also removes the restriction against calling getXXXUnsafe() for String fields when FrameFields is the desync policy which makes it possible to use FrameFields when application code uses Unsafe accessors.

This release allows specifying a blank (empty string or whitespace) 'queue_name' solace bus descriptor property. When a blank value is provided it is interpreted to mean that a queue should not be used.

This is useful in situations in which a default queue name property is specified for a bus via DDL config, and one would like to override the binding not to use a queue.

This release adds the ability to set displayName on the Command, Argument, Option and RemainingArgs annotations. Specifying a display name allows for a better presentation in UI tools such as Lumino

This release changes the XVM server stats thread to populate the heartbeat fields for individual thread stats with cpu usage, user mode cpu usage, affinitization information and the collection time in nanoseconds. These stats can be used by monitoring tools to determine the number of cycles the thread used in a sampling interval and can help to diagnose problems cpu affinitization configuration.

See SrvMonSysIndividualThreadStats Javadoc

This release provides the ability to set the property 'sma_metadata_version' on solace binding bus descriptors. This property can be used to control the version of the metadata to tag on the message. The solace binding currently defaults to metadata version 1, which does not include the transmitted message's type id. Setting it uses the current metadata version which does include the type id, but the metadata version used must be supported by all downstream receivers that deserialize it.

This release improves error handling in the Solace binding to handle the case where there is an issue contructing an UnhandledMessageEvent. With this change:

• If there is an error preparing the UME or enqueuing it for the engine to process,
  the binding now catches the exception and traces the error.

• The binding is closed based on the value of failOnUnhandledMessageEvents

  Align
  align right
  Status subtle false colour Yellow title SMA

  Align
  align right
  Status subtle false colour Yellow title Solace Binding

TBD

This release adds the IOElasticBuffer.getTo(index, IOBuffer, bufferOffset, length) method to copy data from an IOElasticBuffer to an IOBuffer

Prior to this release, the RogLog did not validate that the open-close-open sequence is not permitted on a log. This release adds that validation. From this release onwards, if one opens, closes and then tries to open a log again, it will throw an IllegalStateException

XPLATFORM-1010 changed the store transaction log to include the serialized object being removed in Remove entries. This changes the TransactionLogTool and RogLog to surface the removed store objects in RogLog.Entry making the object content available for query.

The release provides the capability to set the compaction window size when using the transaction log tool's 'compact' command.

This release disables auto optimization of an XVM's binary heartbeat logger so that it doesn't busy spin when nv.optimizefor is latency or throughput.

This release changes server heartbeats to include pool stats for a pool if size highwater mark has changed or if there are evicts in a given interval.

• The size high water mark shows the most number of elements returned to the pool without being taken for use. Knowing the maximum size is useful in inferring whether pool MISSES that are not balanced by subsequent PUTs are the result of leaks.

• Reporting evictions is useful in determining if a pools max capacity may be size too low.

  Align
  align right
  Status subtle false colour Yellow title Stats and Alerts

  Align
  align right
  Status subtle false colour Yellow title Talon

Prior to this release, entities in an entity array in an XBUF message were deserialized into objects during the deserialization phase of message receipt. Bursty traffic containing a large number of messages with a large number of entities in the message's entity arrays could result in a large number of entities being in-flight before they are added to their pools. This can result in large pool sizes effectively resulting in large working heap set size even though the active working set of the heap during steady state is a small fraction of the heap.

This release fixes this issue. It does so by holding the serialized form of the entity arrays contained in a message until actual access of the entities. The entities are deserialized on demand on invocation of the generated getXXX() entity array accessor or the next() method of the generated getXXXIterator() entity accessor method.

A few points to note related to this change

• This change does not affect the behavior of an entity array's generated setter methods

• An entity array's getXXX() generated accessor continues to not be a zero garbage accessor. However, with this change, the amount of garbage generated by this method will increase. The increased garbage is released to the garbage collector immediately after the invocation of the getXXX() method as soon as the caller releases its reference to the returned entities.
  Compatibility Notes:

  div
  class releaseNote

  Code generated by versions prior to 3.7 will continue to work with the 3.7 runtime. However, in such a configuration, the getXXXIterator() will not be zero garbage. Messages need to be regenerated using 3.7 code generator for the getXXXIterator() to be zero garbage

3.7 generated code is not compatible with earlier versions of the X runtime.

Common (non solace specific) bus binding properties such as 'auto_add_catchall_channel' are currently flagged by the solace binding by logging a warning indicating that the properties are unknown solace properties that will be ignored and not passed to the underlying solace layer.

Because these properties are valid properties for configuring the generic portion of the binding logging warnings is not necessary.

When solace is used as a discovery provider discovery properties such as 'maxEntityAge' are currently flagged by the solace binding by logging a warning indicating that the properties are unknown solace properties that will be ignored and not passed to the underlying solace layer.

Because these properties are valid properties when solace is is used as a discovery provider logging warnings is not necessary.

This release changes the ADM generated clone() method for embedded entities and messages to preserve pass through fields by leveraging the new embedded entity copy() functionality introduced in 3.5.
Compatibility Notes:

This change required a bump in the ADM compatibility level from 3.4.1 to 3.5.0. Generated code with the version will not be compatible with earlier versions of the platform (or with code generated by earlier versions of the platform).

This release changes changes the solace binding property for the detached sender cpu affinity mask from 'sender_cpu_affinity_mask' to 'detached_sends_cpu_affinity_mask' so that is is more consistent with the the other 'detached_sends_*' binding properties.

The old property is still honored when specified.

See SolaceBindingProperties javadoc
Compatibility Notes:

Applications are encouraged to update the solace binding property for the detached sender cpu affinity mask from ''sender_cpu_affinity_mask' to 'detached_sends_cpu_affinity_mask'

This release changes the code generation for Xbuf messages and entities to use the SPARSE_CONTENT optimization hint by default.

When this optimization is enabled the sync/desync code paths for the generated type uses a loop to loop through the type's fields syncing or desyncing generically. When the optimization is not enabled custom generated methods for sync/desync are used. Recent testing has shown that in recent JVMs that the generated methods can actually perform worse which has motivated this change.

This release also implements the ability to configure the optimization value globally by setting the XRuntime property:

nv.xbuf.default.optimizationhints=SparseContent|SparseInterest

... or on a type by type basis with:

nv.xbuf.<classname>.optimizationhints=None

where the optimizations hints are specified as a '|' delimited set of the values None, SparseInterest, and SparseContent.

Compatibility Notes:

To leverage the new optimization defaults, code regeneration is required. The newly generated code won't be compatible with older releases as it relies on new platform apis.

This change also removes the constants SPARSE_CONTENT and SPARSE_INTEREST from the generated types. The constants can now be found in com.neeve.xbuf.XbufOptimizations

This release changes the AepEngine to treat a failure to join a channel as a bus binding open failure. Prior to this change if a subscription could not be issued the AepEngine would log an exception and continue, allow the bus to be opened with potentially missing subscriptions.
Compatibility Notes:

This change may result in applications that used to successful start with missing to subscription to no longer startup.

This release changes the resolution of the preProcessingTs and outTs timestamps stored in transaction logs to use a microsecond resolution timestamp obtained from UtlTime#nowSinceEpoch(). The existing timestamps accessors getPreProcessingTs() and getOutTs() continue to return a millisecond resolution timestamp for backwards compatibility; this change adds additional accessors getPreprocessingTsMicros() and getOutTsMicros() to return the number of microseconds since the epoch.

Thus one can query the timestamps in either resolution with the TransactionLogTool as follows:

SELECT simpleClassName, enqueueTs, enqueueTsMicros, preProcessingTs, preProcessingTsMicros, outTs, outTsMicros FROM logs

|Entry.simpleClassName|                  enqueueTs|     enqueueTsMicros|            preProcessingTs|preProcessingTsMicros|                      outTs|         outTsMicros|
+---------------------+---------------------------+--------------------+---------------------------+---------------------+---------------------------+--------------------+
|NewOrder             |2017-08-01 12:10:46.583 PDT|    1501614646583108|2017-08-01 12:10:46.583 PDT|     1501614646583118|                           |                   0|
|OrderEvent           |                           |                   0|                           |                    0|2017-08-01 12:10:46.585 PDT|    1501614646585182|
+---------------------+---------------------------+--------------------+---------------------------+---------------------+---------------------------+--------------------+

Compatibility Notes:

As these timestamps are stored in transactions logs, this change required a bump in their serialization version. Consequently, transaction logs created with this version can not be read by earlier versions of the platform.

The release fixes an issue in which app commands that returned String values had their contents trimmed of leading and trailing whitespace.
Compatibility Notes:

This release change command invocation not to strip whitespace from returned string values. This could have an impact on robin scripts that use the 'wait' or 'validate' commands.

This release fixes an issue in which Xbuf embedded entities were not being disposed when initializing a backup member from a primary member's transaction log.

Failing to dispose the entities during initialization can cause a lot of allocations to occur which can lead to significant garbage collection pauses.

This release allows specifying boolean solace bus descriptor properties in either the JCMSP true/false format or the JNI 1/0 formats. The solace binding will now rationalize them appropriately depending on the value of the use_jni property.

This release fixes an issue in the Solace binding that can occur when detached dispatched is enabled. Prior to this fix, with detached dispatch enabled, it was possible that the the solace reactor thread would dispose the topic it dispatched to the detached dispatch thread before the detached dispatch thread copied it into the channel key field of the message being dispatched to the application.

Prior to this release, An AepEngines would automatically creates a direct bus binding with the same name as the application to avoid the need for boilerplate configuration. However when optimized for latency, this caused a bus manager with a BusySpin thread to be created which ties up an additional core.

Consequently, with this release auto creation of a direct binding has been disabled unless the runtime property nv.aep.autocreatedirectbus=true.

Note that this change has no effect on applications that explicitly create a bus with the same name as the application.

This release fixes the following exception in the solace binding when usejni=true&set_key_on_receipt=true&detached_dispatch=true

java.lang.IllegalArgumentException: source and dest buffers offsets and copy length must be >= 0
	at com.neeve.util.UtlBuffer.copy(UtlBuffer.java:377)
	at com.neeve.io.IOElasticBuffer.putFrom(IOElasticBuffer.java:900)
	at com.neeve.lang.XString.copyInto(XString.java:1603)
	at com.neeve.rog.impl.RogDirectMetadata.setMessageKeyAsRaw(RogDirectMetadata.java:875)
	at com.neeve.rog.impl.RogNode.setMessageKeyAsRaw(RogNode.java:341)
	at com.neeve.sma.impl.MessageBusBindingBase.wrap(MessageBusBindingBase.java:446)
	at com.neeve.sma.impl.MessageChannelBase.onMessage(MessageChannelBase.java:656)
	at com.neeve.solace.SolaceMessageChannel.onMessage(SolaceMessageChannel.java:221)
	at com.neeve.solace.SolaceMessageBusBinding.dispatchMessage(SolaceMessageBusBinding.java:306)

Prior to this release it was possible that with detached dispatch enabled in the binding that the solace reactor thread would dispose the received topic prior to it be in copied into the message by the detached dispatch thread.

This release resolves a bug in the solace JNI binding in which the jni portion of the binding can overwrite another memory location with a NUL (0x00) character.

This would occur when the solace JNI binding appends a NUL character at the end of topic string with a backing buffer size of exactly the topic length – in such a case the null character could overflow into the beginning of an adjacent memory buffer.

This release fixes a bug in which preProcessingTs is not read when reading transaction logs created with version 3.4 or earlier.

In the Talon version 3.5 the transaction log metadata serialization version was changed from 2 to 3. When a v2 version of the transaction log is read by 3.5 the metadata is converted to the v3, but this conversion was not reading the preprocessingTs field.

Fixes a bug with XString when it is initialized from a java String and charAt is called prior to the java String being sync'd into its backing buffer. Prior to this fix charAt would return an incorrect value.

Prior to this release, AEP Engine threw an NPE when a primary environment provider returned null on recording end i.e. if no data was recorded. This release fixes that issue.

This release fixes the following JVM crash in the Solace JNI binding related to freeing message memory after encountering a solace error code in send.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f2288309512, pid=18855, tid=139786289620736
#
# JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build 1.8.0_60-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libsolclient.so.1+0xd6512]<1,18855,perf4.neeveresearch.com> 20170522-19:27:25:022 (dbg)...[MsgChannel<sender.default@nvx-perf-sma(Solace)>]  closing channel [state=Open, flags=0]
  solClient_msg_free+0x12
#
# Core dump written. Default location: /home/root/work/nvx-perf-3.7-SNAPSHOT/core or core.18855
#
<1,18855,perf4.neeveresearch.com> 20170522-19:27:25:022 (dbg)...[MsgChannel<sender.default@nvx-perf-sma(Solace)>]  leaving channel
# An error report file with more information is saved as:
# /home/root/work/nvx-perf-3.7-SNAPSHOT/hs_err_pid18855.log
<1,18855,perf4.neeveresearch.com> 20170522-19:27:25:022 (dbg)...[MsgChannel<sender.default@nvx-perf-sma(Solace)>] Closing channel...
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
./xperftool.sh

This release fixes an TLT issue that occur when dumping Remove entries from a State Replication log that resulted in the an exception like the following:

java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Incompatible ODS class: Unknown message view encoding: -1 (through reference chain: com.neeve.rog.log.Entry["contentEncodingType"])
        at com.neeve.rog.log.RogLogUtil.dumpLogEntryJson(RogLogUtil.java:1403)
        at com.neeve.tools.TransactionLogTool.writeEntry(TransactionLogTool.java:2988)
        at com.neeve.tools.TransactionLogTool.writeEntry(TransactionLogTool.java:2978)
        at com.neeve.tools.TransactionLogTool.access$700(TransactionLogTool.java:93)
        at com.neeve.tools.TransactionLogTool$Next.execute(TransactionLogTool.java:270)

This release fixes a regression in the cstart and tleg1 transaction latency stats that could cause them to report time spent waiting in the engine's input disruptor.

This release fixes a bug in discovery over SMA in which a discovery descriptor containing slashes in the address portion (e.g. solace://tcp://solacehost:55555) results in the error "bus name cannot contain '/'"

This release resolve a regresion in Talon that cause the backup instance to shutdown gracefully if a primary crashes before replicating any new transactions to the backup.

This release fixes a regression that results in a NullPointerException when setting the value of an XbufRepeatedFieldBuffer from another XbufRepeatedFieldBuffer that has not been initialized:

XbufRepeatedLongFieldBuffer destination = IOIEventMessage.createIOIMarketSegmentIDsFieldBuffer(1, true);
XbufRepeatedLongFieldBuffer source = IOIEventMessage.createIOIMarketSegmentIDsFieldBuffer(1, true);

destination.setValuesFrom(source); // <-- Null Pointer Exception

java.lang.NullPointerException
at com.neeve.io.IOElasticBuffer.putFrom(IOElasticBuffer.java:944)
at com.neeve.xbuf.XbufRepeatedLongFieldBuffer.setValuesFrom(XbufRepeatedLongFieldBuffer.java:133)