RDS Emitter
RDS emitter allows you to write to RDS (un)Secured DB Engine. RDS is Relational Database service on Cloud.
RDS Emitter Configuration
To add RDSÂ emitter to your pipeline, drag the emitter onto the canvas and connect it to a Data Source or processor.
The configuration settings are as follows:
| Field | Description |
|---|---|
| Connection Name | All OpenJms connections will be listed here. Select a connection for connecting to the OpenJMS. |
| Message Name | The name for the message configuration which will act as metadata for the actual data. |
| Schema Name | Existing database Schema Names whose tables are fetched. (for MSSQL, DB2 and POSTGRES) |
| Table Name | Existing tablename of the specified database. |
| Is Batch Enable | Enable parameter to process batch multiple messages and improve write performance. |
| Batch Size | Batch Size determines how many rows to insert per round trip. This helps the performance on JDBC drivers. This option applies only to writing. It defaults to 1000. |
| Connection Retries | Number of retries for component connection |
| Delay Between Connection Retries | Defines the retry delay intervals for component connection in millis. |
| Checkpoint Storage Location | Select the checkpointing storage location. Available options are HDFS, S3, and EFS. |
| Checkpoint Connections | Select the connection. Connections are listed corresponding to the selected storage location. |
| Checkpoint Directory | It is the path where Spark Application stores the checkpointing data. For HDFS and EFS, enter the relative path like /user/hadoop/, checkpointingDir system will add suitable prefix by itself. For S3, enter an absolute path like: S3://BucketName/checkpointingDir |
| Time-Based Check Point | Select checkbox to enable timebased checkpoint on each pipeline run i.e. in each pipeline run above provided checkpoint location will be appended with current time in millis. |
| Output Mode | Output Mode to be used while writing the data to data sink. Select the output mode from the given three options: Append: Output Mode in which only the new rows in the streaming data will be written to the sink Complete Mode: Output Mode in which all the rows in the streaming data will be written to the sink every time there are some updates Update Mode: Output Mode in which only the rows that were updated in the streaming data will be written to the sink every time there are some updates. |
| Enable Trigger | Trigger defines how frequently a streaming query will be executed. |
| Processing Time | It will appear only when Enable Trigger checkbox is selected. Processing Time is the trigger time interval in minutes or seconds. |
| ADD CONFIGURATION | Enables to configure additional properties. |
If you have any feedback on Gathr documentation, please email us!