RDS MySQL Ingestion Target

RDS MySQL is Relational Database service on Cloud. RDS MySQL emitter allows you to write to the supported RDS MySQL Database.

Target Configuration #

Fetch From Target/Upload Schema File

The data source records needs to be emitted to a RDS MySQL target table.

In case if the Gathr application has access to a target table in the RDS MySQL database, choose the option Fetch From Target.

In case if Gathr application does not have access to a RDS MySQL target table, you can choose the option Upload Schema File in order to map the RDS MySQL table columns with the source columns during design-time and confirm the data type for each column. In such cases you can run the application in a registered environment, that has access to all the required resources. During run-time, the application will run on the registered cluster of your choice picking up the configuration values as provided during application design.

When you select the Upload Schema File option, a Schema Results section will get displayed at the bottom of the configuration page.

You can then download the sample schema, provide RDS MySQL - Table Column Name against mapping values and verify the data type.

Once it is updated, you can upload the saved file to see a sample of how the records from a source column will be written into the corresponding mapped RDS MySQL column.

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for RDS MySQL earlier. Or create one as explained in the topic - RDS Connection →

Use the Test Connection option to make sure that the connection with RDS MySQL target is established successfully.

A success message states that the connection is available. In case of any error in test connection, edit the connection to resolve issue before proceeding further.

Table Name: Existing table name of the specified database.

Add Configuration: Additional properties can be added using this option as key-value pairs.

More Configurations

Enable Batch: Enable parameter to process batch multiple messages and improve write performance.

If Enable Batch field is selected as True, additional field will be displayed as given below:

Batch Size: Batch Size determines how many rows to insert per round trip. This helps the performance on JDBC drivers. This option applies only to writing. The default value is 1000.

If Enable Batch field is selected as False, then proceed by updating the following fields.

Save Mode: Save Mode is used to specify the expected behavior of saving data to the target RDS MySQL table.

Append: When persisting data, if data/table already exists, contents of the Schema are expected to be appended to existing data.

ErrorifExists: When persisting data, if the data already exists, an exception is expected to be thrown.

Ignore: When persisting data, if data/table already exists, the save operation is expected to not save the contents of the Data and to not change the existing data.

Overwrite: When persisting data, if data/table already exists, existing data is expected to be overwritten by the contents of the Data.

Update: The existing data in the designated target table will be updated, and any new data will be inserted.

If Save Mode field is selected with Update option, additional fields will be displayed as given below:

Update Type: Option to update the latest data out of existing and source data with a choice to either overwrite or keep the latest data with version.

Join Columns: The columns that need to be joined should be selected from the list.