EventBridge Emitter

Gathr user has an option to write data into Amazon EventBridge. This emitter supports both batch as well as streaming data sources.

Configuration details are provided below:

Field	Description
Connection Name	Connection name for creating connection.
Option to specify Source	A string that identifies the source of the event. Note: - Static Value: You will be required to provide a value for source. - Select a Field: Select a field that will be used to evaluate source values.
Option to specify Detail Type	Use Static Value option to assign a constant string. Use Select a Field option to assign value from a field. Note: - Static Value: You will be required to provide a value for source. - Select a Field: Select a value that will be used to evaluate source values.
Specify Event Bus	An event bus is a pipeline that receives events.
Specify the event bus where you want to publish data.Use ‘Static Value’ to assign constant String or ‘Select a Field’ to assign value from a field. Available options are: - Static Value: You will be required to provide the input value for event bus. - Select Field: You will be required to select a field which will be used to evaluate resource values.
Specify Resource	Use Static Value option to assign a constant string or use Select a Field option to assign value from a field.
Constant Value	You will be required to provide the input string value for resource.
Specify Time	Use None option to leave it blank or use Current Time option to assign a current GMT time or use Select a Field option to assign value from a field. 👉 Selected field time field should be long/epoch format. Example: 1672916671
Select a Field	You will be required to select a field which will be used to evaluate Resource values. The selected field must be of string json array type. Example: [“resource1”, “resource2”]
Detail	Fields in the message that needs to be a part of the output data.
Convert Detail fields to JSON	Select String fields in the message that needs to be parsed as JSON objects.
Skip Field Name as Key in Detail	Upon checking this option, the field name will be excluded in Detail as key. Example: If the field name is ‘event’ then, Detail will be {payload}.
If this option remains unchecked then, Detail will be {event : {payload}}.
Throttle Control
Specify Resource	Use Static Value option to assign a constant string or use Select a Field option to assign value from a field.
Maximum Parallel Events	The maximum number of events that can be published by the application in parallel.
Maximum entries in a PutEvent Request	Maximum number of events that can be published in a single putevent request.
Write Modes (for streaming pipeline only)
Output Mode	The output mode specifies how to write the data. In case of streaming source, the available options are: - Append - Complete
Checkpoint Storage Location	DBFS checkpoint storage is not supported when the pipeline is configured on EMR.
Checkpoint Connections	Select the connection that are listed corresponding to the selected storage location.
Override Credentials	Override credential for user specific actions. Upon checking this option you will need to provide username through which the Hadoop service is running.You can test the connection by clicking at the TEST CONNECTION button.
Checkpoint Directory	It is the path where the Spark application stores the checkpointing data. For HDFS and EFS (AWS environment), enter the relative path like /user/hadoop/ checkpointing/Dir system will add suitable prefix by itself. For S3, enter an absolute path like S3://BucketName/checkpointingDir
Time-based Check Point	Check the option to enable the time-based checkpoint on each pipeline run i.e., in each pipeline run the above provided checkpoint location will be appended with current time in milliseconds.
Enable Trigger	Check this option to define how frequently a streaming query should be executed. Upon checking this option, provide Trigger Type as explained below.
Trigger Type	Select one of the options available from the drop-down: - One-Time Micro-Batch - Fixed Interval Micro-Batches

Upon selecting the Fixed Interval Micro-Batches, provide the below details:


Processing Time	Provide the value for processing time in Hour(s)/Minute(s)/Seconds.
Retry on failure
Enable Retry Mechanism	Check the option to retry the job after any interruption while writing data. The job will resume from the point where it was interrupted.
Retry Count	Specify the number of times to retry jobs in case of any interruption.
Retry Delay	Specify the retry delay in milliseconds. This is the time that application should wait before retrying.
Stop on Failure	Once checked, application will fail after completing retries (if any). If unchecked, application will continue to publish further events in input dataset.
ADD CONFIGURATION	User can add further configurations (Optional).
Environment Params	Click the + ADD PARAM button to add further parameters as key-value pair.

If you have any feedback on Gathr documentation, please email us!