Mongo ETL Target

Target Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for MongoDB earlier. Or create one as explained in the topic - MongoDB Connection →

Database Name: The database name should be selected to write data.

Collection Name: The name of the database collection should be selected to write data.

Output Fields: The fields to be included in the output data should be selected from the drop-down list.

Extended BSON Types: When it is checked, this option enables the extended BSON types while writing the data to Mongo DB emitter.

Replace Document: Replace the whole document when saving datasets that contain an _id field. If unchecked, it will only update the fields in the document that match the fields in the dataset.

Local Threshold: Provide the threshold value (in milliseconds) for choosing a server from multiple Mongo DB servers.

Max Batch Size: The maximum batch size for bulk operations when saving data. The default value provided is 512.

Write Concern W: The w option requests for an acknowledgment that the write operation has propagated to a specified number of mongod instances or to mongod instances with specified tags.

Write Concern Journal: The j option requests acknowledgement from MongoDB that the write operation has been written to the on-disk journal.

Write Concern Timeout: This option specifies a time limit, in milliseconds, for the write concern. wtimeout is only applicable for w values greatet than 1.

Shard Key: MongoDB partitions data in the collection using ranges of shard key values. The field should be indexed and contain unique values.

Force Insert: Check the option to enable Force Insert to save inserts even if the datasets contain _IDs.

Ordered: This option can be enabled to allow setting the bulk operations ordered property.

Save Mode: Save Mode is used to specify the expected behavior of saving data to a data sink.

ErrorifExist: When persisting data, if the data already exists, an exception is expected to be thrown.

Append: When persisting data, if data/table already exists, contents of the Schema are expected to be appended to existing data.

Overwrite: When persisting data, if data/table already exists, existing data is expected to be overwritten by the contents of the source data.

Ignore: When persisting data, if data/table already exists, the save operation is expected to not save the contents of the source data and to not change the existing data.

This is similar to a CREATE TABLE IF NOT EXISTS in SQL.

Add Configuration: Additional properties can be added using this option as key-value pairs.

Post Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Post-Actions →

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.