Advanced Mongo ETL Source

Advanced Mongo data source allows you to read data from MongoDB.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for MongoDB earlier. Or create one as explained in the topic - MongoDB Connection →

Database Name: Name of the MongoDB database source from which data is to be fetched should be selected or provided.

Collection Name: Name of the database collection that needs to be scanned should be selected or provided. The collection name is case-sensitive and the collection name entered in this field should exactly match with the collection name in the database.

Query: Option to select the filter criteria by choosing between All Data or Match Query.

If Query option is selected as Match Query, additional field will be displayed as given below:

Filter/Query: Option to exclusively fetch schema results or read data from source as per the filter condition or query provided.

If Query option is selected as All Data, then proceed by updating the following field.

Records Per Partition: Number of records to be read per partition. Default value is 100.

Add configuration: Additional properties can be added using Add Configuration link.

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Incremental Read #

Optionally, you can enable incremental read. For more details, see MongoDB Incremental Configuration →)

Pre Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Pre-Actions →)

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.