These days, almost every action in an organization is reflected in some kind of a database. Invoices, purchase orders, shipping/receiving logs, production reports, etc. — all rely on an underlying database. By tapping into databases, it is possible to bring together events (data) from independent systems and benefit from the application of EP/CEP even within a simple IT infrastructure (no ESB, no enterprise architecture, systems added in an ad hoc manner).
There seems to be three basic ways for Coral8 to read from a database:
- Read from the Database input adapter;
- Poll from the Database input adapter;
- Read directly from a CCL query.
The first one is best suited for replay of historical data, or testing of algorithms against existing data. The adapter reads one row at a time, at a pre-set rate (rows per second).
The second adapter polls the database at a specified interval, and reads only rows that were added since the last read. The database must have a time-stamp — or an ID — column, that is unique to each row and increases in value. My preference here is to use an UTC time-stamp assigned by a client application — the one that inserted the data in the database — to preserve order of events entering a Coral8 stream. Using the ID column is convenient, but should be considered only when there is a single source of data and there is no way to swap records entering the database; or if the order of events is irrelevant.
The third method provides a specific syntax to create database sub-queries within a standard CCL query.
In the next few posts I will go through examples of each method.