Cosmos Batch ETL Source

Azure Cosmos is a globally distributed, multi-model database service. It is designed to provide high availability, scalability, and low-latency access to data for modern applications.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Configure the data source parameters as explained below.

Connection Name #

Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Cosmos earlier. Or create one as explained in the topic - Cosmos Connection →

Use the Test Connection option to ensure that the connection with the Cosmos channel is established successfully.

A success message states that the connection is available. In case of any error in test connection, edit the connection to resolve the issue before proceeding further.

Database #

Specify the name of the Azure Cosmos database.

Container #

Identify the specific container within the Azure Cosmos database.

Enable Change Feed #

Activate Change Feed to capture and process changes.

ChangeFeed From Beginning #

Set to ‘True’ to start processing Change Feed events from the beginning.

Choose ‘False’ to process only new events.

CosmosDb CheckPoint Directory #

Define the directory for storing Cosmos DB Change Feed checkpoints.

Add Configuration: Additional properties can be added using this option as key-value pairs.

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Pre Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Pre-Actions →.

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.