Snowflake Ingestion Source

The Snowflake cloud-based data warehouse system can be used as a data source in Gathr for configuring Ingestion applications.

Data Source Configuration #

Fetch From Source/Upload Data File

For designing the application, you can either fetch the sample data from the Snowflake source by providing the data source connection details or upload a sample data file in one of the supported formats to see the schema details during the application design.

If Upload Data File is selected to fetch sample data, provide the below details.

File Format: Select the sample file format (file type) depending on the data type.

Gathr-supported file formats for Snowflake data source are CSV, JSON, TEXT, Parquet and ORC.

For CSV file format, select its corresponding delimiter.

Header Included: Enable this option to read the first row as a header if your Snowflake data is in CSV format.

Upload: Please upload the sample file as per the file format selected above.

If Fetch From Source is selected, continue configuring the data source.

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Snowflake earlier. Or create one as explained in the topic - Snowflake Connection →

Use the Test Connection option to ensure that the connection with the Snowflake channel is established successfully.

A success message states that the connection is available. In case of any error in test connection, edit the connection to resolve the issue before proceeding further.

Configure the Snowflake data source by providing the details for each field as per the table below:

Warehouse Name: You should provide the warehouse name if Upload Data File option was selected for fetching schema details.

Schema Name: Snowflake schema name is to be selected or provided that will be used for fetching schema details.

Query: The user is required to write the Snowflake SQL query.

Add Configuration: Additional properties can be added using this option as key-value pairs.

More Configurations

Inspect Query: Query used to fetch limited records during Application design. Used only during schema detection and inspection.

Pre Action Query: A semicolon separated list of SQL commands that are executed before reading data.

Post Action Query: A semicolon separated list of SQL commands that are executed after reading data.

Create Warehouse: Option to create a warehouse.

Warehouse Configuration

Warehouse Name: Preferred unique name for the new warehouse should be provided.

Warehouse Size: Preferred warehouse size should be selected.

Maximum Cluster Count: The maximum number of clusters required for the warehouse should be specified.

Scaling Policy: The scaling policy should be selected out of Standard or Economy.

Auto Suspend: Auto suspend value for the warehouse should be provided (in seconds).

Auto Resume: Specifies whether to automatically resume a warehouse when a SQL statement (e.g. query) is submitted to it.

Comments: If needed, the user can mention specific comments.

Schema #

Check the populated schema details. For more details, see Schema Preview →