Cloud SQL ETL Target

Target Configuration #

Configure the target parameters that are explained below.

Connection Name #

Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Cloud SQL earlier. Or create one as explained in the topic - Cloud SQL Connection →

Schema Name #

Select the schema; the dropdown menu will show all schemas associated with the “Connection” selected.

Table Name #

Select the table name; the dropdown menu will show all tables associated with the “Schema”.

Enable Batch #

If TRUE, batches multiple messages while emitting data and improves the write performance.

Batch Size #

Specify the number of records sent to the database in a single batch.

Note: Increasing batch size boosts performance by reducing database round trips, yet large batches may consume excessive memory.

Save Mode #

Save mode specifies how to handle any existing data in the target.

The options are:

Append: Appends data in the configured target.

Overwrite: Overwrites existing data if any, in the target.

Upsert: The existing data in the designated target table will be updated, and any new data will be inserted.

Update: Update the data rows in the target.

Ignore Values #

Enable this option to exclude specified values from the data.

Values to ignore #

Enter comma-separated values to exclude from the data. If a record contains any of these values in columns, even partially, then matching column values will be ignored and set as null.

Add Configuration: To add additional custom ADLS properties in a key-value pair.

Post Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Post-Actions →

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.