Vertica Data Source
This component is supported in Gathr on-premise. VERTICA Channel supports Oracle, Postgres, MYSQL, MSSQL, DB2 connections.
You can configure and connect above mentioned DB-engines with JDBC. It allows you to extract the data from DB2 and other sources into your data pipeline in batches after configuring JDBC channel.
Prerequisite: Upload an appropriate driver jar as per the RDBMS used in JDBC Data Source. Use the upload jar option.
For using DB2, create a DB2 Connection.
Configuring SQS - Vertica Data Source
To add an SQS Vertica Data Source into your pipeline, drag the Data Source to the canvas and right-click on it to configure.
| Field | Description |
|---|---|
| Connection Name | Connection Name for creating connection. |
| Query | Hive compatible SQL query to be executed in the component. |
Metadata
Enter the schema and select table. You can view the Metadata of the tables:
| Field | Description |
|---|---|
| Table | Select table of which you want to view Metadata. |
| Column Name | Name of the column generated from the table. |
| Column Type | Type of the column, for example: Text, Int |
| Nullable | If the value of the column could be Nullable or not. |
Once the Metadata is selected, Click Next and detect schema to generate the output with Sample Values. The next tab is Incremental Read.
Incremental Read
Enter the schema and select table. You can view the Metadata of the tables.
| Field | Description |
|---|---|
| Enable Incremental Read | Check this check-box to enable incremental read support. |
| Column to Check | Select a column on which incremental read will work. Displays the list of columns that has integer, long, date, timestamp, decimal types of values. |
| Start Value | Mention a value of the reference column, only the records whose value of the reference column is greater than this value will be read. |
| Read Control Type | Provides three options to control data to be fetched -None, Limit By Count, and Maximum Value. None: All the records with value of reference column greater than offset will be read. Limit By Count: Mentioned no. of records will be read with the value of reference column greater than offset will be read. Maximum Value: All the records with value of reference column greater than offset and less than Column Value field will be read. For None and Limit by count it is recommended that table should have data in sequential and sorted (increasing) order. |
Click Done to save the configuration.
Configure Pre-Action in Source →
If you have any feedback on Gathr documentation, please email us!