Webhook ETL Source

The Webhook connector in Gathr enables seamless data ingestion by configuring a Webhook data source with a corresponding Webhook Endpoint, capturing real-time events from the associated web application.

• To get a quick understanding about Webhook, see the topic, Introduction to Webhook →.

• To learn how to create an endpoint in Gathr, see the topic, Webhook Endpoints →.

Schema Type #

Fetch From Source/Upload Data File #

To infer schema during the application design phase, you can either fetch the sample data from Webhook events published to Gathr or upload a sample data file in one of the supported formats to see the schema details.

Upload Data File #

To design the application, please upload a data file containing sample records in a format supported by Gathr.

The sample data provided for application design should match the source web application’s schema.

If Upload Data File method is selected to design the application, provide the below details.

File Format #

Select the format of the sample file depending on the file type.

Gathr-supported file formats for Webhook data sources are CSV, JSON, and TEXT.

JSON format is selected by default.

For CSV file format, select its corresponding delimiter.

Header Included #

Enable this option to read the first row as a header if your Workday sample data file is in CSV format.

Upload #

Please upload the sample file as per the file format selected above.

Fetch From Source #

If Fetch From Source method is selected to design the application, then the schema details will be fetched using data from Webhook events published to Gathr.

Continue to configure the data source.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Configure the data source parameters as explained below.

Endpoint Name #

Select the endpoint mapped to your web application where the events are triggered.

Dedup Path #

Specify the field path to identify duplicate records for an event.

This unique identifier ensures that if duplicate records are captured for an event, they will be identified using this path and won’t be processed more than once.

Dedup path format example: .

Retrieve Events By #

Select ‘Earliest’ to read all existing events or ‘Date’ to read events from a specific date onwards.

Offset #

Provide the offset date and time. Events from this point onward will be retrieved.

Add Configuration: Additional properties can be added using this option as key-value pairs.

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Pre Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Pre-Actions →.

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.