Snowflake ETL Source

The snowflake cloud-based data warehouse system can be used as a data source in Gathr for configuring ETL pipelines.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Snowflake earlier. Or create one as explained in the topic - Snowflake Connection →

Warehouse Name: You can select a warehouse from the existing warehouse list. Or, the warehouse name should be provided if create warehouse option is selected.

Schema Name: Snowflake schema list of the database of the selected connection is to be mentioned here.

Query: The user is required to write the Snowflake SQL query.

Inspect Query: Query used to fetch limited records during Application design. Used only during schema detection and inspection.

Pre Action Query: A semicolon separated list of SQL commands that are executed before reading data.

Post Action Query: A semicolon separated list of SQL commands that are executed after reading data.

Create Warehouse: You can create a new warehouse by selecting the check-box. It is required to choose upload file option during detect schema step to avail this option.

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Warehouse Configuration #

If the create warehouse option is chosen, the warehouse configuration details should be provided as explained in Snowflake Warehouse Configuration → topic.

Pre Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Pre-Actions →

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.