Architecture

Solution Diagram

Figure 1: Solution diagram

Connector Diagram

Figure 2: Connector diagram

Connector Flow Diagram

Figure 3: Connector Flow diagram

Incremental Overlap

To fetch data from the search API, an incremental overlap strategy is used. By default, the time interval included in each request is extended by one minute (default) backward before retrieving new data. This helps ensure that no data is missed.

In the following figure you can see, that after the connector start the data is download 1 day backwards by default. In the next iterations of the connector, the incremental overlap is applied and some of the data are processed again jus to be sure we did not missed any of them.

• Tokens
• Subscriptions
• Parameters
• Statistics