...
- There are > 0 commits in the engine's commit pipeline
- The number of completed commits in the pipeline has not changed for a configurable period of time
- A message or event handler is hung and preventing the engine's multiplexer thread from processing further events
- The stuck condition above remains present for 2 consecutive stuck engine checks. An alert will only be raised on the second stuck engine check which confirms that the condition is persistent.
- Per stuck engine check either a HungTransactionPipeline or HungEventHandler will be reported, not both, and HungTransactionPipeline takes precedence. Stuck alerts are meant to raise a warning that an administrator should investigate engine health, and make a deeper determination of possible root causes. Upon seeing that StuckAlertEvent has been raised, thread dumps should be taken, and stats and trace logs should be examined to determine if there is a serious issue.
AsynchronousEventHandler
StuckAlertEvents aren't dispatched via the engine's regular event handler because in this case the handler itself may be stuck. To receive AepStuckAlertEvents, and application must register an AsynchonousEventHandler with the engine via the setAsynchronousEventHandler method.
AepStuckAlertEvent
The following accessors are available on a AepStuckAlertEvent
Field | Description |
---|---|
Reason | Returns the reason for this alert. Possible values are: HungTransactionPipeline: AepEngine transactions complete asychronously in a pipelined fashion. The engine's pipeline is considered hung when there are oustanding commits that have not completed. HungEventHandler :Indicates that the AepEngine's event processor is hung. The event processor thread handles dispatching of application messages to their handlers along with other internal events. |
Message | An optional descriptive message intended for display in an alert message. |
Engine | The engine that is stuck. |
LastEventProcessedTimestamp | Gets the timestamp in millis of the last event fully processed by the engine's event handler or the time that the first event was received whichever is greater |
LastCommitCompletionTimestamp | Gets the timestamp in millis of the last completed engine transaction or the timestamp that the first event was received whichever is greater |
IncompleteCommitCount | Gets the number of commits that have not been completed. |
Abatement and Repeated Alerts
There currently is no corresponding abatement event for stuck alerts, abatement of a stuck alert condition can be determined by looking at numCommitsCompleted engine stat, if the number of commits completed is increasing then the stuck condition has abated.
AepStuckAlertEvents are not repeatedly dispatched for a given stuck condition. For a HungTransactionPipeline another stuck alert won't be generated until at least one commit has completed since the last alert. For a HungEventHandler, another stuck alert event won't be triggered until at least one event has been successfully processed since the last alert.