Advanced Mongo Data Source

Connections are the Service identifiers.

Select the connection name from the available list of connections, from where you would like to read the data.

Filtering criteria option to choose between All Data or Match Query.

If Match Query option is selected, provide the below detail:

Fetch filtered schema or read data from source as per the filter condition or query provided.

Note:

In Match Query, use only single quotes (‘’) where required.

Select the column for incremental read. The listed columns can be integer, long, date, timestamp, decimal, etc.

An offset value needs to be set for the incremental read that will be done on the selected column.

Only those column records with values greater than the offset value will be read.

Provides three options to control data fetched - None, Limit by Count, and Limit by Value.

None: All the records in the reference column with values greater than the Start Value will be read.

Limit by Count: Only the mentioned number of records in the reference column with their values greater than the Start Value will be read.

Provide No. of Records.

Limit by Value: All the records in the reference column with

values greater than the Start Value but less than/equal to the Column Value that you set will be read (Column Value is inclusive).

Set the Column Value.

For None and Limit by Count, it is recommended that the table should have data in sequential and sorted (increasing) order.

This option is checked by default during first time of the configuration. If the check box is unchecked, then it reads the records from Oplog collection as per the specified CDC configuration during the pipeline execution. It will get automatically disabled after pipeline is successfully executed.

If this option is unchecked, then provide the below field: