Kinesis ETL Target

Kinesis emitter emits data to Amazon Kinesis stream. Supported data type formats of the output are Json and Delimited.

Target Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Kinesis earlier. Or create one as explained in the topic - Kinesis Connection →

Stream Name: Name of the Kinesis Stream.

Shards Count: Number of shards required to create the stream, in case stream is not present.

Region: Name of the region. For example, us-west-2

Partition Key: Used by Amazon Kinesis to distribute data across shards.

Output Format: Datatype format of the output.

Output Fields: Fields in the message that needs to be a part of the output data.

Output Mode: Mode to be used while writing the data to Streaming sink.

Select the output mode from the given three options:

Append: Output Mode in which only the new rows in the streaming data will be written to the sink.

Complete: Output Mode in which all the rows in the streaming data will be written to the sink every time there are some updates.

Update: Output Mode in which only the rows that were updated in the streaming data will be written to the sink every time there are some updates.

Enable Trigger: Trigger defines how frequently a streaming query should be executed.

Processing Time: It will appear only when Enable Trigger checkbox is selected.

Processing Time is the trigger time interval in minutes or seconds.

Add Configuration: Enables to configure additional properties.

Post Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Post-Actions →

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.