SQS ETL Source

SQS Channel allows you to read data from different SQS queues. The Queue types supported are Standard and FIFO. Along with providing maximum throughput you can achieve scalability using the buffered requests independently.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for SQS earlier. Or create one as explained in the topic - SQS Connection →

Queue Type: Select type of the queue: Standard or FIFO.

Queue Name: Name of the queue where data will be published.

Visibility Timeout: The length of the time (in seconds) that a message received from a queue will be invisible to other receiving components.

Message Retention Period (in seconds): The amount of time Amazon SQS will retain a message, if it does not gets deleted.

Maximum Message Size (in bytes): Maximum message size (in bytes) accepted by Amazon SQS.

Receive Message Wait Time (in seconds): The maximum amount of time that a long polling receive call will wait for a message to become available before returning an empty house.

Delivery Delay (in seconds): The amount of time to delay the first delivery of all messages added to this queue.

Add configuration: Additional properties can be added using Add Configuration link.

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.