GCS Batch ETL Source

Gathr provides GCS (Google Cloud Storage) Batch channel.

Permissions Required in GCS #

The user should have cloud storage permission to access and read data from Buckets of GCS Batch data source.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Connection Name: Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for GCS Batch earlier. Or create one as explained in the topic - GCS Connection →

Bucket Name: Provide path of the file for Google storage bucket name.

Path: Provide value for the end path with * in case of directory. Example: outdir*

Detect Schema #

Check the populated schema details. For more details, see Schema Preview →

Incremental Read #

Optionally, you can enable incremental read. For more details, see GCS Incremental Configuration →

Pre Action #

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Pre-Actions →).

Notes #

Optionally, enter notes in the Notes → tab and save the configuration.