Cloud SQL Data Source
Cloud SQL is a fully-managed database service that helps you set up, maintain, manage, and administer your relational databases on Google Cloud Platform.
You can use Cloud SQL with PostgreSQL and MSSQL Server in Gathr.
Configuring Cloud SQL Data Source
Add the Cloud SQL Data Source to your pipeline by selecting it from the components panel. Click on the component to configure it.
Under the Schema Type tab, select Fetch From Source, Upload Data File or Use Existing Dataset.
Under the Configuration tab, the below fields are available:
| Field | Description |
|---|---|
| Connection Name | Connections are the Service identifiers. Select the connection name from the drop-down list. The connection must already be created from the Connections page. For details, click Create Connection. |
| Override Credential | Select the override credentials option check-box for overriding the credentials. |
| Username | Provide the database username that has access to data. |
| Password | Provide the database user password. |
| Query | Hive compatible SQL query to be executed in the component. |
| Inspect Query | Provide a Hive compatible SQL query (as provided above) to be executed in the component with a limit in record count. This will be used only during Inspect and Schema Detection. |
| Enable Query Partitioning | Tables will be partitioned and loaded in RDDs if this check-box is enabled. This option enables parallel reading of data from the table. |
| ADD CONFIGURATION | Click the ADD CONFIGURATION button to add further configuration details as key-value pair. |
| Environment Params | Option to add environment parameters by clicking at the ADD PARAM button is available. |
| Metadata | The schema and table of the selected database are reflected. |
| Schema | Schema available within the selected database is available in the drop-down list. |
| Table | Table available within the selected schema is available. |
If you have any feedback on Gathr documentation, please email us!