Release Date: Monday, February 27, 2017
Release Highlights
- Adds support for property based DDL Configuration ... to allow external configuration formats to be used more easily.
- Various Configuration Improvements and Knobs:
- Ability to configure xvm stats via DDL
- Ability to disable support related metadata fields from being replicated to reduce overhead for low latency apps
- Support for setting channel filters in DDL.
- Performance improvements - reduced buffer copies.
- Support for running ADM maven plugin goals from the platform maven plugin (useful for users of the maven bom as the nvx-platform-maven-plugin follows the nvx-platform version)
- Reduced memory footprint for State Replication applications by employing more compact metadata representation and late allocation of fields that are not used.
- Solace Binding Improvements
- Option not to attempt to create a queue on behalf of the session.
- Support for querying the list of subscriptions for a solace session (JSCMP binding only ... CCSMP support to be added in a following release).
- This release introduced javadoc for the solace binding making it easier to understands solace specific configuration properties. See Solace Binding Javadoc
- Bug Fixes
Compatibility Notes
ADM Compatibility Level: 3.4.0 (see ADM Compatibility Matrix) The Adm Compatibility level last changed in Core X 3.4.373 release
The following issues have compatibility implications:
- XPLATFORM-1564 - Provide support for property based DDL configuration
- XPLATFORM-1569 - Enable server heartbeat and stats config in DDL
- XPLATFORM-1567 - Add model validation to prevent duplicate names of model elements
- XPLATFORM-1454 - Disallow using same transaction log for inbound and outbound message loggers
- XPLATFORM-1561 - Change DDL app message bus configuration schema to wrap bus lists in a <buses> element
- XPLATFORM-1532 - StatsDumpTool reports the store's commit latencies instead of cqs in stats dump
Change Log
- New Feature
- XPLATFORM-584 - Enhance packet recovery log to use memory mapped IO for flush operations
- XPLATFORM-1343 - DDL should allow setting a channel key filter for an app's bus.
- XPLATFORM-1382 - Allow Solace Binding not to try to create the configured queue if it doesn't exist
- XPLATFORM-1541 - Create nvx-platform-maven-plugin that provides goals of all individual plugins
- XPLATFORM-1564 - Provide support for property based DDL configuration
- XPLATFORM-1569 - Enable server heartbeat and stats config in DDL
- Improvement
- XPLATFORM-923 - Reduce Memory Footprint of Rog Node metadata.
- XPLATFORM-1425 - Remove warnings in generated code
- XPLATFORM-1531 - Store Member should display its advertised connector and discovery settings on startup
- XPLATFORM-1533 - Stamp Post Wire Send Timestamp and report Wire Send latency in bus binding stats
- XPLATFORM-1536 - Channel Key, Filter, QoS, Id and join status should be displayed in engine config trace.
- XPLATFORM-1537 - Emit Inbound and Outbound Message Logger Statistics
- XPLATFORM-1538 - Enhance StatsDumpTool to allow filtering of admin app stats.
- XPLATFORM-1539 - Change StatsDumpTool to accept start and end time using query format.
- XPLATFORM-1540 - Emit 'o2p' (OfferToPoll) stats in heartbeart for persister and ICR sender
- XPLATFORM-1545 - Add disk usage stats to server heartbeats.
- XPLATFORM-1547 - Enable direct serialization of object metadata into ODS Subheader buffer
- XPLATFORM-1552 - Add SMA bus configuration flag to control whether sequence number related metadata is set by an SMA bus or not on inbound messages
- XPLATFORM-1553 - Add SMA bus configuration flag to control whether flow related metadata is set by an SMA bus or not on inbound messages
- XPLATFORM-1554 - Add AEP engine configuration flag to control whether support related ROG metadata is set by an AEP engine or not
- XPLATFORM-1555 - Add AEP engine configuration flag to control whether sequence number related ROG metadata is set by an AEP engine on outbound messages
- XPLATFORM-1562 - Create Goal Aliases for ADM Maven MOJOs
- XPLATFORM-1563 - Support generating code for multiple models from maven plugins
- XPLATFORM-1567 - Add model validation to prevent duplicate names of model elements
- XPLATFORM-1578 - Javadoc solace binding properties.
- XPLATFORM-1579 - Update DDL to allow adding displayName on ddl elements
- Change
- XPLATFORM-1454 - Disallow using same transaction log for inbound and outbound message loggers
- XPLATFORM-1561 - Change DDL app message bus configuration schema to wrap bus lists in a <buses> element
- XPLATFORM-1577 - Upgrade Solace CCSMP to 7.2.4.8 and JCSMP to 7.2.2.100 to allow better support for authentication schemes.
- Bug
- XPLATFORM-1525 - nv.aep.trace, nv.sma.trace and nv.ods.trace does not take effect in the env section of a DDL
- XPLATFORM-1532 - StatsDumpTool reports the store's commit latencies instead of cqs in stats dump
- XPLATFORM-1534 - It is not possible to specify channel key variables in DDL because they get replaced during DDL substitution.
- XPLATFORM-1535 - Increased serialization costs on engine thread for outbound logging.
- XPLATFORM-1559 - preProcessingTs is not persisted or replicated when replicateInParallel is true
- XPLATFORM-1568 - Host field not properly set on ESA packets
- XPLATFORM-1574 - Talon distribution xconfig scripts fail with "X license verification failed [Heap limit exceeded" with free license
- XPLATFORM-1575 - EXbufFieldCorruptException: deserialization error: corrupt data stream [unsupported field wire type code 4
- XPLATFORM-1576 - Deserialization of an Xbuf message with String arrays can require reserialization.
- XPLATFORM-1580 - NPE in Solace binding during close after messaging start failure.
- XPLATFORM-1587 - Solace reconnect attempt doesn't use all properties of original connect attempt
- XPLATFORM-1589 - NullPointerException resolving unqualified type import
- XPLATFORM-1590 - XLinkedHashMap can erroneously return null retrieving an existing entry after another entry is removed
New Feature
XPLATFORM-584 - Enhance packet recovery log to use memory mapped IO for flush operations | ||||||||||||||||||
Prior to this release, the packet recovery log used RandomAccessFile to perform file reads and writs. This story enhances the recovery log to use memory mapped IO instead for flush operations improved write latencies and performance. The recovery log continues to use the RandomAccessFile for read operations | ||||||||||||||||||
XPLATFORM-1343 - DDL should allow setting a channel key filter for an app's bus. | ||||||||||||||||||
This release update DDL xml to allow configuring a channel filter for an apps channels: SMA | ||||||||||||||||||
XPLATFORM-1382 - Allow Solace Binding not to try to create the configured queue if it doesn't exist | ||||||||||||||||||
This release introduces the ability to suppress automatic provisioning of a queue for solace message bus bindings when the 'queue_name' property is set for a solace binding by setting the property 'provision_queue=false'. See: SolaceBindingProperties.PROVISION_QUEUE_PROPNAME SOLACE BINDING | ||||||||||||||||||
XPLATFORM-1541 - Create nvx-platform-maven-plugin that provides goals of all individual plugins | ||||||||||||||||||
This release introduces an nvx-platform-maven-plugin that aggregates the goals from nvx-core-maven-plugin, nvx-robin-maven-plugin and nvx-eagle-maven-plugin. This is particularly useful for applications that use the nvx-platform-bom to provide versions of various platform components because maven boms don't cover plugin versions. With the nvx-platform-maven-plugin, the plugin version will remain the same as the platform-bom which reduces the number of versions that need to be update in the pom. The nvx-platform-maven-plugin uses the convention of prefixing the goals with individual plugins' short name to avoid conflicts in the names of goals. The individidual plugins have been correspondingly updated to support these additional goal names as well:
Usage BUILD/INFRASTRUCTURE | ||||||||||||||||||
XPLATFORM-1564 - Provide support for property based DDL configuration | ||||||||||||||||||
This release allow representing a DDL xml document as a set of properties. This change makes it easier for external configuration mechanism to map into platform configuration. For example, it is now trivial to use HOCON to configure as it maps very easily to properties. Applications that will use external configuration and bypass XML DDL configuration entirely may do so, by calling VMConfigurer.configure(properties) See Also:
|
The DDL XML parser will still parse the old heartbeatLogging element, but will generate a warning.
This change also introduces new configuration properties for enabling server stats and configuration heartbeat emission so that these properties no longer need to be set in the environment.
Improvement
XPLATFORM-1425 - Remove warnings in generated code |
This release improve the ADM code generators import statement generations to reduce unused imports which results in fewer warnings in eclipse. ADM |
XPLATFORM-1531 - Store Member should display its advertised connector and discovery settings on startup |
When config trace is enabled for nv.ods, a store configuration for replication will now trace its discovery provider and connection information when it advertises itself. ODS |
XPLATFORM-1533 - Stamp Post Wire Send Timestamp and report Wire Send latency in bus binding stats |
This release adds a new message bus stat to capture the time spent in a message bus provider's send call when sending messages. Prior to this change, the message latency stats stopped at the point just before a message is written to the wire. This was because message latency stats were designed to cover the portion of time spent in the platform and provider specific send time was considered to be external. However, in the case that a provider blocks in flow control or is otherwise slow, the provider send time may have an impact on the sending thread's throughput making the time spent writing to the wire of interest from a platform statistic standpoint. This story implements capturing the post wire send timestamp and reports the delta between preWireTs and this timestamp as wire send time. This new latency stat is reported in engine stats output with other bus stats as 'ws' for wire send times and in hearbeats as WireSendLatencies. Sample of printed output ...[ ws] [sample=65536, min=7 max=54 mean=10 median=11 75%ile=11 90%ile=13 99%ile=19 99.9%ile=23 99.99%ile=26] ......NewOrder...[ ws] [sample=0, min=-1 max=-1 mean=-1 median=-1 75%ile=-1 90%ile=-1 99%ile=-1 99.9%ile=-1 99.99%ile=-1] ......OrderEvent...[ ws] [sample=65536, min=7 max=24 mean=8 median=8 75%ile=9 90%ile=12 99%ile=15 99.9%ile=19 99.99%ile=22] These latencies are also emitted in heartbeats. See Also:
|
XPLATFORM-1536 - Channel Key, Filter, QoS, Id and join status should be displayed in engine config trace. |
This release improves the trace of an engine's config to include channel details when engine config trace is enabled. <1,24578,perf4> 20161213-01:47:01:495 (cfg)... Engine 'market' Configuration { ...state_factories=[] ...message_factories=[] ...channels ......xsim-morders@market [join=true] .........Id : 1 .........Key : orders/${Region} .........Filter : null .........QoS : Guaranteed ......xsim-mconfirm@market [join=false] .........Id : 2 .........Key : null .........Filter : null .........QoS : Guaranteed AEP SMA |
XPLATFORM-1537 - Emit Inbound and Outbound Message Logger Statistics |
This release adds inbound and outbound message logger stats to heartbeats and stats trace: [Inbound Message Logger] <No Inbound Message Logger> [Outbound Message Logger] Put {0(0 0)} Updates {0(0 0)} Removes {0(0 0)} Sends {979,561(0 11,000)} Commits {534,306(0 6,000)} Forks {0(0 0)} Disruptor (SingleThreaded, BusySpin) {[0 of 1,024] 0%} ...[o2p] [sample=65536, min=0 max=93 mean=26 median=26 75%ile=45 90%ile=51 99%ile=57 99.9%ile=64 99.99%ile=78] Writer {qsize=0} These stats are emitted in server heartbeats as part of engine stats. See Also:
AEP STATS AND ALERTS |
XPLATFORM-1538 - Enhance StatsDumpTool to allow filtering of admin app stats. |
This release changes the StatsDumpTool to filter admin app stats by default when generating stats trace from hearbeat logs. Admin app stats can still be done by passing the '-a' flag to the tool. STATS AND ALERTS TOOLS |
XPLATFORM-1539 - Change StatsDumpTool to accept start and end time using query format. |
This release enhances the StatsDumpTool to allow accept the start and end time options in the same time formats accepted by transaction log tool queries. For example, one can now launch the tool with a time range such as: java -cp "libs/*" com.neeve.tools.StatsDumpTool -s "30 minutes ago" -e "20 minutes ago" rdat/server-hearbeats/myapp-heartbeats.log java -cp "libs/*" com.neeve.tools.StatsDumpTool -s 2:00pm -e 2:05pm rdat/server-hearbeats/myapp-heartbeats.log It is still possible to specify these arguments in the prior timestamp format such as: java -cp "libs/*" com.neeve.tools.StatsDumpTool -s "20161216-02:06:00" rdat/server-hearbeats/myapp-heartbeats.log STATS AND ALERTS TOOLS |
XPLATFORM-1540 - Emit 'o2p' (OfferToPoll) stats in heartbeart for persister and ICR sender |
These release now populates the o2p stats in store persister and icr sender stats emitted in heartbeats on the SrvMonDisruptorStats entity. STATS AND ALERTS TOOLS |
XPLATFORM-1545 - Add disk usage stats to server heartbeats. |
This release adds disk usage stats collection to server heartbeats which can be used to monitor low disk space issues. See: ISrvMonSysStats.html.getDiskStats() STATS AND ALERTS |
XPLATFORM-1547 - Enable direct serialization of object metadata into ODS Subheader buffer |
This release removes an intermediate copy that was incurred copying an object's metadata into the Store replication subheaders. |
XPLATFORM-1552 - Add SMA bus configuration flag to control whether sequence number related metadata is set by an SMA bus or not on inbound messages |
This release adds a new SMA bus configuration flag (set_sno_on_receipt specified on bus descriptor) to control whether sequence number related metadata is set by an SMA bus on inbound messages |
XPLATFORM-1553 - Add SMA bus configuration flag to control whether flow related metadata is set by an SMA bus or not on inbound messages |
This release adds a new SMA bus configuration flag (set_flow_on_receipt specified on bus descriptor) to control whether flow related metadata is set by an SMA bus on inbound messages |
XPLATFORM-1554 - Add AEP engine configuration flag to control whether support related ROG metadata is set by an AEP engine or not |
This release adds an AEP engine configuration flag (setSupportMetadata) to control whether support related ROG metadata is set by an AEP engine on inbound/outbound messages |
XPLATFORM-1555 - Add AEP engine configuration flag to control whether sequence number related ROG metadata is set by an AEP engine on outbound messages |
This release adds an AEP engine configuration flag (setOutboundSequenceNumbers) to control whether sequence number related ROG metadata is set by an AEP engine on outbound messages |
XPLATFORM-1562 - Create Goal Aliases for ADM Maven MOJOs |
Added aliases for ADM maven plugin goals: "generate" same as "adm-generate"; "generateTest" same as "adm-generateTest" ADM |
XPLATFORM-1563 - Support generating code for multiple models from maven plugins |
Maven plugin supports new parameter called 'modelFiles' which enables code generation for multiple models per single execution. Either 'modelFile' or 'modelFiles' can be used but not both. When 'modelFiles' is used, 'namespace' parameter is not applicable and will break the build. ADM |
XPLATFORM-1567 - Add model validation to prevent duplicate names of model elements |
This release improves ADM model validation to fail fast if the same name is used in a model for multiple entities, multiple messages or combination of them. With this release an error will be reported when unless -Dnv.adm.suppressduplicatechecks=true is set. This release introduces stricter code generation checks for duplicate type names in ADM models. |
XPLATFORM-1578 - Javadoc solace binding properties. |
This release provides javadoc for the solace binding and its properties. |
See http://build.neeveresearch.com/solace/javadoc/3.5
Change
Consequently applications' DDL xml should be updated from
to
This change is a recommended change ... the old schema is still supported, but is now deprecated and may be removed in a future release.
Compatibility Notes:
To support properties based DDL in a consistent fashion the DDL schema has been updated such that an applications configured buses are now listed under a buses element. While the old configuration schema is still valid applications are advised to update to the new format.
XPLATFORM-1577 - Upgrade Solace CCSMP to 7.2.4.8 and JCSMP to 7.2.2.100 to allow better support for authentication schemes. |
This release upgrades the dependent solace binaries as follows: |
- For JNI binding: CCSMP libraries were upgraded from 7.1.2.137 to 7.2.4.8
- For Java, sol-jcsmp and sol-common from 7.1.2.248 to 7.2.2.100.
The motivation for the upgrade was to allow better support for authentication schemes provided by solace in the newer binaries.
|
Bug
XPLATFORM-1535 - Increased serialization costs on engine thread for outbound logging. |
This release fixes a performance regression that can occur in detached outbound message logging. The regression was caused by the movement of some of the serialization logic for outbound message metadata to be done by the aep engine's thread along with the dispose call for the message being logged. The additional logic in the aep engine's thread can cause reduced throughput for applications that have heavy outbound message loads and are close to be cpu bound on the business logic thread. AEP |
XPLATFORM-1559 - preProcessingTs is not persisted or replicated when replicateInParallel is true |
This release fixes an issue with parallel cluster replication in which the pre processing timestamp was not being replicated to the backup. This was problematic because the pre processing timestamp is used to return the engine time during recovery and and on the backup to prevent divergence. TALON |
XPLATFORM-1568 - Host field not properly set on ESA packets |
This release fixes a bug in discovery packets in which the host name was not getting set. The host name is used for informational and support purposes by the platform and tools so this issue was a minor one. HUMMINGBIRD |
XPLATFORM-1574 - Talon distribution xconfig scripts fail with "X license verification failed [Heap limit exceeded" with free license |
This release sets the '-Xmx' variable in a talon distributions configure.sh script allow the distribution's environment to be safely configured when using a free X license which imposes maximum heap sizes. HUMMINGBIRD |
XPLATFORM-1575 - EXbufFieldCorruptException: deserialization error: corrupt data stream [unsupported field wire type code 4 |
This release fixes a wire corruption issue in Xbuf encoded entities that can occur with array fields with the following sequence of events:
This fix address a problem in step 4: if enough values are added to the array field that it changes the size of the value needed to encode the length of the array, corruption will occur. This issue is most likely to impact State Replication applications. For this to impact an application, the application would have to:
|
XPLATFORM-1576 - Deserialization of an Xbuf message with String arrays can require reserialization. |
This release fixes an issue with Xbuf repeated string fields that could cause a message to be reserialized if written after being deserialized which could slow down Event Sourcing applications that receive messages with String array fields. ADM PERFORMANCE XBUF |
XPLATFORM-1580 - NPE in Solace binding during close after messaging start failure. |
Fixes an issue in which a null pointer exception can be thrown during close() on a solace message bus binding when an exception is thrown starting messaging (for example, as a result of a subscription validator throwing an exception). SOLACE BINDING |
XPLATFORM-1587 - Solace reconnect attempt doesn't use all properties of original connect attempt |
When an attempt is made to open a solace bus binding, properties on the bus descriptor are normalized to remove solace agnostic properties and to rationalize between jcsmp and ccsmp binding format. Because this normalization process modifies the properties supplied in the bus descriptor, a second attempt using the same bus descriptor will have properties missing. This release fixes this issue. SOLACE BINDING |
XPLATFORM-1589 - NullPointerException resolving unqualified type import |
This release fixes a NullPointerException that can occur during ADM code generation when resolving an imported type from another model that is not fully qualified. java.lang.NullPointerException at com.neeve.adm.AdmModel.lookupType(AdmModel.java:2306) at com.neeve.adm.AdmXMLParser.createField(AdmXMLParser.java:1265) at com.neeve.adm.AdmXMLParser.getFieldRef(AdmXMLParser.java:1428) ADM |
XPLATFORM-1590 - XLinkedHashMap can erroneously return null retrieving an existing entry after another entry is removed |
This release fixes a bug in the XLinkedHashMap implementations that could result in null be returned when getting a value from the map after another element was removed. HUMMINGBIRD |