HTTP Data Source
The HTTP data source can fetch data based on requests sent to HTTP and HTTPS resources via Gathr.
Information that will be useful while configuring the data source:
Type of the source data (for example, CSV, JSON, or Text)
Resource URI to access the source data.
Authentication type of the resource and details required for Gathr to access the data (for example, if Auth Type is Basic, then, Username and Password will be needed to access the resource)
Further, the HTTP data source configuration is also equipped to handle SSL-enabled resources.
Configuring HTTP Data Source
To add an HTTP data source to your pipeline, drag the Data Source to the canvas and right-click on it to configure.
| Field | Description |
|---|---|
| URI | The HTTP or HTTPS URI should be provided to send requests to a resource via the HTTP data source. The URI should contain all the details that the HTTP data source will need to find the API resource it is requesting. Example: |
| Request Method | The HTTP request method to fetch data from a source should be selected out of GET or POST. The method selected determines how the configured parameters in the data source will be submitted to the HTTP resource. GET: Use the GET method if form data is included in the URI, appended as query string parameters. POST: Use the POST method if the form data is to be given in the request body of the HTTP data source. |
| Request Body | Using the request body option, a data payload can be sent to an HTTP resource in the body of the request. The Request Body field will appear only for post requests. |
Request Body Example
{
"from":1,
"size":10,
"query":{"match_all":{}}
}
| Header | Use the header field(s) to provide additional requests to the HTTP resource via the HTTP data source. HTTP headers have a key-value pair format. Example
|
| Accept Type | Content type that URI accepts. Example: JSON, CSV and Text. |
| Auth Type | Specify the authorization type that should be used to authenticate the HTTP resource. The supported authentication types are: None, Basic, Token, and OAuth2. Each type is explained below in detail. |
| Path to message | Path to data is a JSON path expression that points to arrays. Path to Data filed will appear only when the type of source data is specified as JSON in the File Format option. Example: Consider fetching the schema details of a source for the sample JSON file illustrated below:
If you keep the Path to Data field value as $ (i.e., default value), then as per the data source configuration the entire JSON file will get fetched.
But, if you provide the Path to message field value as $.data, then only the attributes of the
|
Auth Types
The supported authentication types are: None, Basic, Token, and OAuth2.
Each type is explained below in detail.
Auth Type - None
Choose this option to access an HTTP resource without needing any authentication.
Auth Type - Basic
If Auth Type is selected as Basic, proceed after providing the below parameters:
Username: Enter the user name for accessing the HTTP resource.
Password: Enter the password for accessing the HTTP resource.
Auth Type - Token Based
If Auth Type is selected as Token, proceed after providing the below parameters:
Token ID: The key with which the token is referred in the request.
Token: Token to access the HTTP resource.
Auth Type - Oauth2
If Auth Type is selected as OAuth2, proceed after providing the below parameters:
Auth Headers: The headers associated with Auth URL should be provided as key-value pairs, through which the authorization code is generated.
Client ID: The client identifier given during the application registration process should be provided.
Secret Key: The secret key given to the client during the application registration process should be provided.
Auth URL: The endpoint for the authorization server, which retrieves the authorization code should be provided.
Configure Pre-Action in Source →
If you have any feedback on Gathr documentation, please email us!