...
The use of a gateway app to integrate your apps' state with an external source is works well in situations where the gateway app can tap into the application's message stream to detect changes in state. In situations where an app's message stream does not clearly reflect the changes being made to the app's state, additional messages reflecting the changes to the application state would need to be published for the gateway app integration method to work. This can become cumbersome rather quickly and potentially pollute the architecture at play by leaking an application's private state to the outside world. In such situations, integrating the app's state with the external data source via CDC would be the preferred mechanism,
...
- Connect to the external data source on one side
- Start a transaction log CDC runner on the other side
- The CDC runner would "tail" the log and dispatch entries from the log as and when new entries are added to the log
- On receipt of entries from the CDC runner, use techniques specific to he the external data source to write the data in the received entries to the external source
| Note | ||
|---|---|---|
| ||
Eagle provides a CDC runner designed to CDC data to an RDBMS. It implements all the boilerplate code needed to CDC data to an RDBMS and can be used to integrate an app's state with an RDBMS |
...
The CDC runner automatically conflates data in the transaction log. The runner dispatches both the conflated objects and the individual changes to objects to the CDC processor. For example, if the app makes 10 updates to a Customer object in memory, the CDC runner will dispatch all 10 changes as well as the final (conflated) state of the object to the recipient of the runner change feedCDC processor. Therefore, the CDC runner's change feed can be used as both an audit trail of changes to the app state objects as well as a conflated stream of coarse grained changes to the app state objects. Since most external data sources, such as RDBMSs, analytical cubes and centralized storage engines have a performance capability that is generally much less than what a X application can process, the ability to only push the conflated changes to objects to the external data source while the app processes all fine grained changes allows for the integration of the app's state and the external source possible and stable. Without this capability, the external source would result in pushing back on the app resulting in queueing of data somewhere in the system effectively causing either a resource exhaustion or a slow down of the app.