As a htap database, ShannonBase has abilitiy of populating the changes in innodb to rapid in real-time, which makes it can do real time analytical workloads. Shannonbase use redo log to achieve this population. When a transaction do commit, it will write some redo logs. Therefore, we can have a copy of this redo log, and then parse it to apply these changes to rapid engine. Base the idea of above, we use a circular ring buffer to store these redo log. Only, the redo logs of insert, delete, and update operation will be stored into ring buffer. A backgroud thread will be launched after alter table xxx secondary_load statement executed. And, it will be stopped when all loaded tables are unloaded from rapid engine. If you want to know the status info of rapid engine, you can use show engine innodb status to list all informations including rapid engine. Here, we dont split rapid engine status information from innodb's because rapid is a sub-engine of innodb. The innodb engine is primary engine, the rapid is secondary one.

There are two approaches to propagate the changes from innodb to rapid engine. One is using notification hooks, another is using redo log. The notification hooks approach is to use innodb's notification hooks to notify rapid engine when there are changes in innodb, such as insert, delete, update operations. Rapid engine will then apply these changes to its own storage. The redo log approach is to parse the redo log to get the changes. Currently, ShannonBase uses a global variables, rapid_sync_mode, to indicate which approach is used.

After tables are loaded into ShannonBase Rapid Tables, data changes are automatically propagated from InnoDB tables to their counterpart tables in the Rapid. DML operations, INSERT, UPDATE, and DELETE, on the InnoDB Tables do not wait for changes to be propagated to the Rapid; that is, DML operations on the InnoDB Tables are not delayed by ShannonBase change propagation. Data changes on the InnoDB Tables are propagated to Rapid in batch transactions. Change propagation is initiated as follows: (1)Every 200ms. (2)When the change propagation buffer reaches its 64MB capacity. (3) When data updated by DML operations on the InnoDB Tables are read by a subsequent ShannonBase query.

A change propagation failure can cause tables in Rapid to become stale, and queries that access stale tables are not offloaded to Rapid for processing. Tables that have become stale due to change propagation failures resulting from out-of-code errors are automatically reloaded. A check for stale tables is performed periodically when the Rapid is idle. If change propagation failure has occurred for some other reason causing a table to become stale, you must unload and reload the table manually to restart change propagation for the table.