MQTT ETL Target
MQTT emitter emits data to MQTT queue or topic. Supported output formats are Json, and Delimited.
MQTT supports wireless network with varying levels of latency.
MQTT Emitter Configuration
To add MQTT emitter to your pipeline, drag the emitter onto the canvas, connect it to a Data Source or processor, and right-click on it to configure it.
Connection Name All MQTT connections will be listed here. Select a connection for connecting to MQTT.
Queue Name/Topic Name Queue or topic name to which messages will be published.
Output Format Datatype format of the output. JSON and Delimited. If Delimited is selected, provide a delimiter as well.
Output Fields Fields in the message that needs to be a part of the output data.
Checkpoint Directory It is the path where Spark Application stores the checkpointing data.
For HDFS and EFS, enter the relative path like /user/hadoop/, checkpointingDir system will add suitable prefix by itself.
For S3, enter an absolute path like: S3://BucketName/checkpointingDir
Time-Based Check Point Select checkbox to enable timebased checkpoint on each pipeline run i.e. in each pipeline run above provided checkpoint location will be appended with current time in millis.
Output Mode Output mode to be used while writing the data to Streaming sink.
Append Output Mode in which only the new rows in the streaming data will be written to the sink.
Complete Mode Output Mode in which all the rows in the streaming data will be written to the sink every time there are some updates
Update Mode
Output Mode in which only the rows that were updated in the streaming data will be written to the sink every time there are some updates.
Enable Trigger Trigger defines how frequently a streaming query should be executed.
Processing Time Trigger time interval in minutes or seconds.
Priority Priority defines the execution order of the emitters.
ADD CONFIGURATION Enables to configure additional properties.
Click on the Add Notes tab. Enter the notes in the space provided.
Click SAVE for saving the configuration.
If you have any feedback on Gathr documentation, please email us!