This release provides the ability to do basic injection of XRuntime configuration values into objects provided objects. See Annotation Driven Config

This release adds two new failure handling policies for failures persisting messages to an application's inbound or outbound message logs:

• AepEngine.OutboundMessageLoggingFailurePolicy: StopLogging|StopEngine
• AepEngine.InboundMessageLoggingFailurePolicy: StopLogging|StopEngine

Prior to this release, failures emanating from the inbound or outbound message loggers were trapped and an error was traced to trace logs along with dispatching either an AepInboundLogExceptionEvent or AepOutboundLogExceptionEvent which the application could trap and optionally stop the engine on its own.

With the introduction of the 2 new policies, the above trace and exception events are still emitted, but now an AepEngine can be configured to either shutdown with StopEngine or to stop logging with StopLogging policies. There is no longer an option to simply ignore the exception because in most cases such failures are permanent and continued attempts to log new messages typically just result in repeated trace and alerts.

See DDL Configuration (https://docs.neeveresearch.com/display/KB/DDL+Schema+Reference#DDLSchemaReference-<inboundMessageLogging) for more details.

Compatiblity Notes:

The default policy on either an inbound message logging failure or outbound message logging failure is to shutdown the engine. Additionally, an outbound message logging failure policy of StopEngine will cause outbound logging failures to unsolicited senders whereas before the engine would trap the exception.

This release introduces the ability to configure message buses with a single 'catch all' channel. When a bus configured with a catch all channel receives a message with no channel specified, or on a channel that doesn't exist or is not joined, the catch all channel is used to dispatch the message.

A channel is denoted as being the buses catch all channel by assigning it a channel id of 32767 (Short.MAX_VALUE). If a bus channel is not explicitly configured by the application with the catch all channel id, the message bus can be configured to create one automatically by setting the bus descriptor property 'auto_add_catchall_channel=true' in which case the bus will automatically add a channel called 'CATCHALL' for use as a catch all channel.

See also:

• MessageBusDescriptor.PROPNAME_AUTO_ADD_CATCHALL_CHANNEL
• MessageBusDescriptor.CATCH_ALL_CHANNEL_NAME_DEFAULT

• MessageBusDescriptor.CATCH_ALL_CHANNEL_ID

  SMA

This release provides additional knobs that can be used to control cpu spinning. Prior to this release when nv.optimizefor was set to 'latency' or 'throughput' disruptor event processor threads would assume 'BusySpin' or 'Yielding' wait strategies. In some situations this behavior is not desirable.

New Knobs
The following knobs affect cpu spinning operations such as tcp eagerread and disruptor wait stategies:

• nv.conservecpu=true Setting this value as an XRuntime environment property, instructs the platform to avoid cpu spinning regardless of the value for nv.optimizefor or explicit tuning configuration from the user. This property is intended for use in testing situations where an application is being running on a lower powered machine to avoid spinning and thus conserve cpu. Applications may consult XRuntime.conserveCPU() to see if cpu conservation is enabled.
• nv.disablewaitstrategyoptimizations=true Setting this value as an XRuntime environment property not to optimize threads for cpu spinning if nv.optimizefor is set. This property is useful in cases where the application has explicitly configured disruptor cpu spinning.

See Also:

• XRuntime.createWaitStrategy()

• XRuntime.conserveCPU()

  HUMMINGBIRD
  TALON

This release adds the ability to configure message bus binding to set the message bus, message channel name, and message key on inbound messages.

The message bus and channel can be set in the inbound message by setting the global environment property 'nv.sma.setbusandchannelonreceipt=true' or on a bus by bus basis by adding the bus descriptor property 'set_bus_and_channel_on_receipt=true' in the bus url.

For buses that support transport of the message key (topic), the message key can be set on inbound messages by setting the global property 'nv.sma.setkeyonreceipt=true' or on a bus by bus basis by setting 'set_key_on_receipt=true'. For the solace binding, this property is equivalent to the now deprecated property 'set_topic_in_inbound_msg'.

See also:

• MessageBusDescriptor.PROPNAME_SET_KEY_ON_RECEIPT

• MessageBusDescriptor.PROPNAME_SET_BUS_AND_CHANNEL_ON_RECEIPT

  SMA
  TALON

This release adds the ability to avoid attaching the generated source directory or model source folder to a Maven project from nvx-core-maven-plugin:

This release implements the ability for user to register the set of fields in a message that they are interested in. This results in only those fields that are registered in being deserialized thus allowing for faster deserialization.

To utilize this feature, the user needs to register field interest via a static setFieldInterest() method generated for each message type. This method is currently only generated for X-Buffer encoded messages
Compatiblity Notes:

Messages generated with this change are NOT compatible with older versions of the X runtime.

This release introduces the ability to set a MessageViewFilter on an AEPEngine to filter live messages prior to starting a transacton or dispatch to user handlers.

A message filter does not apply to messages replicated by a peer or recovered from disk. Filtered messages are acknowledged, and are not logged to a transaction logs.

Effects of a MessageFilter:

Filtered messages

• Are acknowledged.
• Are not logged to the transaction log, inbound message log or replicated to cluster peers.
• Are not dispatched to application EventHandlers.
• Do not start new transactions or contribute to adaptive batching counts.

Monitoring
The number of filtered messages are reported in AepEngine stats as numMsgsFiltered. This stat is also emitted in server heartbeats. Additionally message filter latencies are captured to record the time spent filtering messages for an the application, and these timings, are also reported in engine stats.

Message filtering occurs prior to dispatching to application registered message handlers or pre or post dispatch handlers. Filtered message contribute to numEventsProcessed, numMsgsReceivedBestEffort, numMsgsReceivedGuaranteed and numMsgsSourced.

Prior to this release, network read, de-serialization of the packetize form of a store commit entry and dispatch of the deserialized object to the engine multiplexer is done by the network reader thread. This story adds the capability to perform this activity by a dedicated thread. This can considerably improve overall system throughput. The following parameters on the store descriptor control this functionality:

detachedDispatch

• "true" switched on detached dispatch functionality while "false" disables it. Default
• default=false

dispatcherQueueDepth

• Controls the depth of the detached dispatch queue
• default=1024

dispatcherCpuAffinityMask

• Specifies the CPU mask to which the dispatcher thread should be affinitized to
• default=0

dispatcherQueueWaitStrategy

• Specifies the dispatchers queue wait strategy. Permissible values are
  • Blocking
  • Sleeping
  • Yielding
  • BusySpin
• default=Blocking

The name of the dispatcher thread is "X-ODS-Native-Dispatcher-<storename>.<membername>

This release adds the numEventsProcessed to server heartbeats. This stat complements the numEventsRcvd statistic and is incremented once event processing has completed. This stat can be used to tell if there are events currently being processed.

Prior to this release, the wait strategy for the detached send and dispatch queues are hard coded. This release makes them configurable.

• To configure the wait strategy for the detached send queue, set the detached_sends_queue_wait_strategy property in the Solace message bus binding descriptor to the desired wait strategy.
• To configure the wait strategy for the detached dispatch queue, set the detached_dispatch_queue_wait_strategy property in the Solace message bus binding descriptor to the desired wait strategy.

Permissible wait strategies are as follows:

• Blocking
• Sleeping
• Yielding
• BusySpin

The above values can be set in the bus binding descriptor in a case insensitive manner. Setting a value other than any of the above will cause the binding to throw an exception during the binding open.

This release adds the ability to disable native file io by setting nv.file.usenative=false.

This release enhances the ordering guarantees for scheduled events such that events scheduled with the same delay are scheduled in order of event creation. For applications that use AepEngine.scheduleMessage this means that 2 events scheduled with the same delay (including high priority, negative delay) will be scheduled in the order the calls were made to the engine. This also applies to Hornet's MessageInjector interface which is just a thin wrapper around scheduleMessage.

Prior to this release, not all UnhandledMessageEvents emitted by the Solace message bus binding would include the topic from which the message was received via UnhandledMessageEvent.getMessageKey(). This release closes those gaps and populates the received topic when it is available.

To assist in troubleshooting, we should make exception text for inbound message handling faults more descriptive:

• Include the message of a linked exception in the wrapping exception's text.
• Link linked exception's in UnhandledMessageEvent's getReason.
• Include channel name and id when available and applicable (e.g. in channel not found exceptions).

This release improves the UnhandledMessageEvent that is emitted by a message bus binding on receipt of a message it can't handle to include the message provider specific message id via getMessageSMATransportMessageId() along with a corresponding field in its backing message event SrvMonUnhandledMessageMesssage available via getTriggeringMessageSMATransportMessageId().

The SMA transport specific message ID may be supplied by bindings for which there is a notion of a unique identifier for a message. A null value means that either the binding doesn't support the notion of such an id or that that the id wasn't available.

This release makes available the serialized metadata and payload blobs on the UnhandledMessageEvent emitted by message bus bindings on receipt of a message that can't be handled. When available these fields can be used to persist or quarantine the message via its backing SrvMonUnhandledMessageMessage because they allow the message to be reconstituted providing the values are not themselves corrupted.

Prior to this release, the X the trace logging machinery does not automatically create log directories. This release implements auto creation of trace log directories.

This release allows setting inbound and outbound initial log lengths of less than 1Gb in the platform.xml by relazing x-ddl.xsd to accept a float rather than an integer for initialLogLength.

Prior to this improvement events injected into an AepEngine's event loop that were not part of a transaction batch would result in the user's transaction batch being committed. The primary reason for the old behavior was to prevent the batch from languishing in an open state. The old behavior can have a detrimental impact on throughput if interleaved not transaction events frequently terminate transaction batches prematurely. The new behavior now only closes and commits the transaction batch if there aren't further events ready from processing in the input queue that could potential contribute to the transaction batch.

This release adds the AepEngine stat, numMsgsSourced to monitoring heartbeats - before it was only collected and reported in aep engine stats trace.

Sourced messages apply only to messages not received from directly from a message bus they include:

• messages replicated to the backup
• initialized from the event log at the primary
• initialized from the primary at the backup.
• messages injected by the application.

The total number of messages received by the engine can be computed by adding numMsgsRcvdBestEffort + numMsgsRcvdGuaranteed + numMsgsSourced.

As of this release licensing is enforced by the platform. Without a valid license the JVM will exit when using platform libraries. To obtain a license you may:

• To obtain a free or evaluation license, visit http://license.neeveresearch.com and select the type of license you'd like to create and a license will be e-mail to you with instructions.
• Contact your Neeve sales representative if you have purchased a different license type.

You can install your license in your application's runtime environment and/or as an application resource and/or by placing it in the user home folder of the user running the JVM of the platform

Installing the license in your application's environment
1. Copy your license file to a disk location from which you will run the platform
2. Set one of the following environment variables or Java system properties to point to the license file

• NVLICLOCATION
• nv.lic.location
• nv_lic_location

Installing your license as an application resource
Modify your application build to include xplatform.lic at the resource root i.e. /xplatform.lic (in either a jar or in your classes folder)

Installing your license in your home directory
Copy your xplatform.lic file to the home directory of the user used to run the X application(s)

A few points to note
1. You must not modify the contents of xplatform.lic or it will be rendered invalid and will fail to validate

2. License enforcement is performed when using most libraries distributed with the platform, a failure to find a valid license file will result in System.exit(1) being called which will terminate the JVM.

3. If multiple environment variables/system properties with the above name(s) are set, then the following rules apply:

• Environment variables take precedence over system properties
• The precedence of property names is as listed above.

4. A license installed in the application's runtime environment will take precedence over a license installed as an application resource.

5. A license installed as an application resource will take precedence over a license installed in the user's home directory.

Compatiblity Notes:

As of this release the JVM will exit if using platform libraries without a valid license for the platform.

This release enables validation of x-ddl (platform.xml) by the VMConfigurer and DdlXMLParser. Prior to this release validation was not done which could cause configuration errors to go unnoticed.

Though not recommended, validation can be disabled by specifying the System Property nv.ddl.schema.validate=false. NOTE: this must be set as a System property, not an XRuntime property or ddl property
Compatiblity Notes:

Prior to this change schema validation errors in platform.xml would be ignored without causing errors, this release validates x-ddl by default which could cause validation failures on application startup.

This version upgrades Core X to use Tanuki v3.5.25 from 3.5.19
Compatiblity Notes:

If your application also uses Tanuki check this could introduce a version conflict. Such applications should also update the Tanuki version if so.

This release separates out the JMS binding from the core platform and packages it separately in its own nvx-jms.jar separately from nvx-core-all. Users of the jms binding must explicitly add the nvx-jms dependency.
Compatiblity Notes:

Users of the jms binding must now explicitly add the nvx-jms.jar as a dependency as it is no longer bundled with nvx-core-all or references as a transitive dependency. For maven users this amounts to declaring:

This release changes the name of the 'p2p' message bus binding to use the friendlier 'direct' binding moniker. The 'p2p' provider prefix in the bus descriptor is deprecated, but can still be used, so both of the following are valid descriptors for a specifying a direct binding descriptor:

direct://orderprocessingapp

p2p://orderprocessingapp

though support for the latter may be dropped in a future release.

This release changes the group attribute on the server element in x-ddl.xsd to be an optional attribute. When not specified the server's group will default to "default".

This release changes handling of message trace logging exceptions to be non fatal. Prior to this release such exceptions would result in the shutdown of the engine. With this release message trace logging errors are trapped and result in the emission of a AepMessageTraceLoggingExceptionEvent (a new AlertEvent). For applications running in a Talon server, this event will be backed by a SrvMonMessageTraceLoggingExceptionMessage.

An exception in trace logging usually is indicative of a problem with the message being logged, and are often times serious, but the reasoning behind this change is that if the problem with the message is serious that it will cause a failure at a later point in the transaction pipeline. Applications that would like to treat message trace logging errors as fatail may add a handler for the AepMessageTraceLoggingExceptionEvent and stop the engine.

This release changes the behavior of message receipt during message bus binding close. Prior to this change, it was possible that when a message is received in the middle of close, that the message's channel would have been un-joined or closed leading to either a "channel not joined" error "channel not found" error which depending on the binding implementation would result in either an UnhandledMessageEvent or error being logged to the trace logs. This release formally introduces a Closing state to SMA bindings and channels and specifies that when in a Closed, Closing or Failure state that bindings ignore messages received. Such messages are not acknowledged by the binding leaving them eligible for redelivery.
Compatiblity Notes:

This change introduces a 'Closing' value to com.neeve.sma.MessageBusBinding.State and com.neeve.sma.MessageChannel.State. Applications that explicitly test for binding or channel state should make sure they account for the new states. Note: the added stats do not extend to the the server monitoring models (heartbeats, lifecycle and alerts).

Fixes a NullPointerException that can occur in transaction log queries when selecting on fields that are monotonic and unindexed:

java.lang.NullPointerException
        at com.neeve.query.impl.QueryPlanImpl.addStep(QueryPlanImpl.java:1638)
        at com.neeve.query.impl.QueryOptimizer.and(QueryOptimizer.java:690)
        at com.neeve.query.impl.QueryOptimizer.createPlan(QueryOptimizer.java:456)
        at com.neeve.query.impl.QueryOptimizer.createPlan(QueryOptimizer.java:407)
        at com.neeve.query.impl.QueryEngineImpl.execute(QueryEngineImpl.java:615)
        at com.neeve.rog.impl.log.RogLogQueryEngineImpl.execute(RogLogQueryEngineImpl.java:161)

Prior to this fix disabling native buffers (-Dnv.buffermanagement.usenative=false) would cause persisting to a transaction log to fail when native I/O is enabled because the native implementation requires using native buffers. This fix forces creation of a native buffer for use with transaction logs when native file I/O is enabled even if native buffers have been disabled.

Fixes an issue where if a StringMap, IntMap or LongMap put() operation that is replacing an existing value fails it can leave the map in an inconsistent state that will yield the following exception on a subsequent put() for that the same key:

java.lang.IllegalArgumentException: node does not have a fid
        at com.neeve.rog.impl.RogContainerNode.removeChild(RogContainerNode.java:319)
        at com.neeve.rog.impl.RogStringMap.put(RogStringMap.java:164)

This fix also adds additional diagnostic trace around failures during map updates, particularly around recording the removal of the previous value in the store's transaction log.

Fixes a regression in cleaning channel keys when nv.sma.cleanMessageKey is enabled. This fix ensures that non letter/digit characters are replaced with an '_' when the property is enabled.

Fixes AepEngine.getEngineTime() to only return the the current message processing time for EventSourcing applications if the calling thread is the engine's dispatch thread. Prior to this fix it would return the current message processing time regardless of the thread calling the method which can lead to an inconsistent time when called by a non dispatch thread.

This fix improves SMA MessageMetadata serialization/deserialization to differentiate between a channel name being null and empty. Prior to this change setting SMA with an channel name that is null would result in deserialization of that metadata producing a channel name that is an empty String.
Compatiblity Notes:

Recipients that decode SMA metadata should be aware that if the channel name is set to null on the sending side the encoded length of the channel name will be set to -1 when there is a null value. Prior to this change a null channel name set by an X sender would result in a 0 length string with an encoded length of 0.

Fixes a race condition in which shutting down a JVM and creating and starting discovery can lead to a NullPointerException in the DiscoveryCache shutdown hook.

This release updates the dependency version of commons-compress from 1.1 to 1.3. This resolves an issue that can occur extracting transaction log archives that contain large transaction log files and/or many archive entries. See COMPRESS-129

Fixes a bug in which the flag indicating whether an event had been handled was not being reset when an event is recycled to a pool which could cause an AepEngine not to dispatch an AepUnhandledMessageEvent to an application with no event handler for a message when pooling is enabled.

The AepChannelDownEvent was not getting dispatched to the application on binding failure even if the application has an AepChannelDownEvent handler. This release fixes this issue.

Prior to this release, if the same message type is transmitted through two different channels, only the messages from the channel that dispatched the first message to the handler will continue to dispatch messages to the handler while messages from the other channel will be dropped. This release fixes this issue

This release fixes an issue in the Solace JNI binding where an unrecognized solace payload type would not be reported to the application as an UnhandledMessageEvent

This release fixes an ArrayIndexOutOfBoundsException that can occur during cluster replication when output buffer sizes are grown.