Gathr help center
Browse docs
    • Gathr Installation Introduction
      • On Prem Installation Introduction
      • Component Versions Supported
      • Gathr Prerequisites
      • Embedded Gathr
      • Enable SSL on Gathr Webstudio
      • Enable HA on Gathr Webstudio
        • Gathr Setup
        • Cluster Configuration
        • Cluster Configuration for Apache
        • Cluster Configuration for CDP Using Cloudera Manager
        • Configure Gathr with HTTPS
        • Gathr Settings
        • Database
        • Messaging Queue
        • Elasticsearch
        • Cassandra
        • Version Control
      • Configure Gathr for Kerberos (Optional)
      • Restart Gathr
        • CDC Application Prerequisites
        • Enable CDC for MYSQL
        • Enable CDC for Postgres
        • Enable CDC for SQL Server
        • Enable CDC for Oracle
        • AWS RDS Postgres-Kafka-CDC
        • SMTP Server Configuration
        • Enable SSL on Kafka
        • Nexus Configuration
        • Start/Stop H2O Server
        • Python Configuration
        • Apache Airflow Installation
        • Kudu Installation
        • SSL Components Setup
        • Installing Jupyter, IDEs and Sparkmagic on Centos/RHEL
        • Python Environment
      • Gathr Manual Deployment on IBM
      • Gathr Automated Deployment on IBM
      • Gathr Deployment on Azure
      • Setup Gathr on AWS - Manual Deployment
      • Setup Gathr through AWS Marketplace - Automated Deployment
      • Gathr Deployment on GCP - Manual
      • Gathr Deployment on GCP - BYOL
      • Gathr Enterprise/Medium/Small - Deployment on GCP
    • About Gathr
    • Administration Introduction
      • Manage Workspace
      • Create a Workspace
      • Enter a Workspace Administration
      • Cluster List View
      • Manage Databricks Clusters
      • Manage EMR Clusters
      • Manage Google Cloud Dataproc Clusters
      • Manage IBM Spectrum Conductor
      • Register Cluster
      • Registered Clusters Listing
      • Steps to Register Cluster
      • Register Container Image
      • Registered Container Images Listing
      • Steps to Register Container Image
      • Manage Setup
      • Manage Configuration Introduction
      • Web Studio
      • Processing Engine
      • Messaging Queue
      • NoSQL
      • Indexing Store
      • Metrics Store
      • Hadoop
      • Others
      • Default
      • Manage Connections
      • Manage Superuser Connections
      • Create Connections
      • Auto Update Connection
      • Manage Connections of Workspace
      • Audit Trail
      • Event Search
      • Visualizing Audit Results
      • Audit Table Glossary
      • Configurations
      • Manage Users and Roles
      • Manage Users (main-menu)
      • Manage Users (Workspace-menu)
      • Template
      • Components
      • Connections
      • Credit Points Consumption
      • Help
    • Projects Introduction
      • Datasets Introduction
      • Create Datasets
      • Datasets List
      • Reuse Datasets
      • Import Export Datasets
      • Register Models Introduction
      • Register Spark ML Model
      • Register H2O Models
      • Register Scikit Model
      • Model Versions
      • Applications Introduction
      • Create CDC Application
      • Actions on CDC Applications
      • Data Validation Introduction
      • Create Jobs
      • Processor Group Introduction
      • Steps to Create a Processor Group
      • Steps to Use a Processor Group
      • Pipeline Introduction
      • Creating a Spark Pipeline
      • Deploy a Pipeline
      • Pipeline Management
      • Continuous Integration Continuous Delivery
      • Configuring Cloudera to Support Lineage
      • Pipeline Listing Page
      • Templates Introduction
      • Create Template
      • Template as Component in a Pipeline
      • Bulk Operations History
      • Workflow Introduction
      • Create a Workflow
      • Create a Template
      • Terminologies
      • Alerts
      • Introduction to Import Export Entities
      • ImportExport Workflow Job
      • Import Python Manifest
      • Action Window
      • Actions on Listing Page
      • Register Entities Introduction
      • Register Components
      • Functions
      • Variables
      • Calendars
      • Compute Environment
      • Sandboxes
      • Non Container based Sandbox
      • Container based Sandbox
      • Sandbox Listing Page
      • Creating Sandbox
      • Version Control Overview
      • Modified Files
      • Revert Notebooks
      • Tags
      • Overview
      • Project Activity
    • Monitoring and Alerts Overview
    • Pipeline Monitoring
    • Error Search
    • Resource Analyzer Report
    • Configuring Alerts
    • Components Introduction
      • Data Sources Introduction
      • Pre-Actions
      • Auto Schema
      • ADLS Data Source - Batch and Streaming
      • Advanced Mongo Data Source
      • Attunity Data Source
      • AWS IoT Data Source
      • Azure Blob Batch Data Source
      • Azure Blob Stream Data Source
      • Batch Cosmos Data Source
      • BigQuery Data Source
      • Cassandra Data Source
      • CDC Data Source
      • Container Data Source
      • Custom Channel Data Source
      • Data Generator Data Source
      • Dataset Channel Data Source
      • Delta (Batch and Streaming) Data Source
      • Delta SQL Data Source
      • Dummy Data Source
      • File Reader Data Source
      • GCS (Batch and Streaming) Data Source
      • Google Spreadsheet Data Source
      • HDFS Data Source
      • Hive Data Source
      • HTTP Data Source
      • HTTPV2 Data Source
      • HTTPV2 Pagination
      • HTTPV2 Incremental Configuration
      • IBM MQ
      • Impala Data Source
      • JDBC Data Source
      • Jira Data Source
      • Kafka Data Source
      • Kinesis Data Source
      • Kudu Data Source
      • MQTT Data Source
      • Native DFS Receiver Data Source
      • Native File Reader Data Source
      • Neo4j
      • OpenJMS Data Source
      • Pub/Sub Data Source
      • RabbitMQ Data Source
      • RDS Data Source
      • Redshift Data Source
      • S3 Batch Data Source
      • S3 Streaming Data Source
      • Salesforce Data Source
      • SFTP Data Source
      • Snowflake Data Source
      • Socket Data Source
      • SQS Data Source
      • Stream Cosmos Data Source
      • Streaming Delta Data Source
      • Tibco Data Source
      • Vertica Data Source
      • VSAM Data Source
      • Processors Introduction
      • Advanced Sort Processor
      • Aggregation Processor
      • Alert Processor
      • App ID Generator Processor
      • Binary Avro Parser Processor
      • Cache Processor
      • Container Processor
      • Custom Processor
      • Data Cleansing Processor
      • Data Quality Management Processor
      • Decoder Processor
      • Decryption Processor
      • Dedup Processor
      • Distinct Processor
      • Drools Processor
      • Drop Processor
      • Encoder Processor
      • Encryption Processor
      • Eviction Processor
      • Expression Evaluator Processor
      • Expression Filter Processor
      • Field Converter Processor
      • Field Flattener Processor
      • Field Replacer Processor
      • Field Splitter Processor
      • Filter Processor
      • Functions Processor
      • Hashing Processor
      • HTTP Processor
      • JDBC Container Processor
      • Join Processor
      • Jolt Processor
      • JSON Parser Processor
      • Limit Processor
      • Masking Processor
      • NA Processor
      • PII Masking Processor
      • Pivot Processor
      • Processor Group Processor
      • Python Processor
      • Rank Processor
      • Redis Lookup
      • Register as Table Processor
      • Rename Processor
      • Repartition Processor
      • Router Processor
      • Sagemaker Processor
      • Scala Processor
      • Schema Flattener Processor
      • Schema Transformer Processor
      • Select Processor
      • Sequence Generator Processor
      • Snowflake Processor
      • Sort Processor
      • SQL Processor
      • Stored Procedure Processor
      • TopNRecords Processor
      • Union Processor
      • Turnpike Processor
      • Watermark Processor
      • XML Parser Processor
      • XSLT Processor
      • Functions Introduction
      • Date Functions
      • Lookup Functions
      • String Functions
      • Math Functions
      • Array Functions
      • Miscellaneous Functions
      • Data Science Introduction
      • ML Models
      • PMML Models
      • H2O Models
      • Scikit Models
      • Models Listing Page
      • Emitters Introduction
      • Post Actions
      • ADLS Emitter
      • Advanced HDFS Emitter
      • Advanced Kafka Emitter
      • Advanced Redshift Emitter
      • AWS IOT Emitter
      • Azure Blob Emitter
      • Batch Emitter
      • BigQuery Emitter
      • Cassandra Emitter
      • Container Emitter
      • Cosmos Emitter
      • Cloud SQL Emitter
      • Delta Emitter
      • Dummy Emitter
      • Elasticsearch Emitter
      • EventBridge Emitter
      • File Writer Emitter
      • GCS Emitter
      • HBase Emitter
      • Hive Emitter
      • HTTP Emitter
      • JDBC Emitter
      • Kafka Emitter
      • Kinesis Emitter
      • Kudu Emitter
      • Mongo Emitter
      • MQTT Emitter
      • NativeHDFS Emitter
      • Neo4j
      • OpenJMS Emitter
      • Pub/Sub Emitter
      • RabbitMQ Emitter
      • RDS Emitter
      • Redshift Emitter
      • S3 Emitter
      • Salesforce Emitter
      • Snowflake Emitter
      • Solr Emitter
      • SQS Emitter
      • Streaming Emitter
      • Vertica Emitter

Gathr

      • Latest (v5.3.3)
      • v5.3.2
      • v5.3.1
      • v5.3.0
      • v5.2.1
      • v5.2.0
      • v5.0.0
      • v4.9.3
      • v4.9.2
      • v4.9.1
      • v4.9.0
      • v4.8.1
      • v4.8.0
      • v4.7.1
      • v4.7.0
      • All versions
        • Gathr Installation Introduction
          • On Prem Installation Introduction
          • Component Versions Supported
          • Gathr Prerequisites
          • Embedded Gathr
          • Enable SSL on Gathr Webstudio
          • Enable HA on Gathr Webstudio
            • Gathr Setup
            • Cluster Configuration
            • Cluster Configuration for Apache
            • Cluster Configuration for CDP Using Cloudera Manager
            • Configure Gathr with HTTPS
            • Gathr Settings
            • Database
            • Messaging Queue
            • Elasticsearch
            • Cassandra
            • Version Control
          • Configure Gathr for Kerberos (Optional)
          • Restart Gathr
            • CDC Application Prerequisites
            • Enable CDC for MYSQL
            • Enable CDC for Postgres
            • Enable CDC for SQL Server
            • Enable CDC for Oracle
            • AWS RDS Postgres-Kafka-CDC
            • SMTP Server Configuration
            • Enable SSL on Kafka
            • Nexus Configuration
            • Start/Stop H2O Server
            • Python Configuration
            • Apache Airflow Installation
            • Kudu Installation
            • SSL Components Setup
            • Installing Jupyter, IDEs and Sparkmagic on Centos/RHEL
            • Python Environment
          • Gathr Manual Deployment on IBM
          • Gathr Automated Deployment on IBM
          • Gathr Deployment on Azure
          • Setup Gathr on AWS - Manual Deployment
          • Setup Gathr through AWS Marketplace - Automated Deployment
          • Gathr Deployment on GCP - Manual
          • Gathr Deployment on GCP - BYOL
          • Gathr Enterprise/Medium/Small - Deployment on GCP
        • About Gathr
        • Administration Introduction
          • Manage Workspace
          • Create a Workspace
          • Enter a Workspace Administration
          • Cluster List View
          • Manage Databricks Clusters
          • Manage EMR Clusters
          • Manage Google Cloud Dataproc Clusters
          • Manage IBM Spectrum Conductor
          • Register Cluster
          • Registered Clusters Listing
          • Steps to Register Cluster
          • Register Container Image
          • Registered Container Images Listing
          • Steps to Register Container Image
          • Manage Setup
          • Manage Configuration Introduction
          • Web Studio
          • Processing Engine
          • Messaging Queue
          • NoSQL
          • Indexing Store
          • Metrics Store
          • Hadoop
          • Others
          • Default
          • Manage Connections
          • Manage Superuser Connections
          • Create Connections
          • Auto Update Connection
          • Manage Connections of Workspace
          • Audit Trail
          • Event Search
          • Visualizing Audit Results
          • Audit Table Glossary
          • Configurations
          • Manage Users and Roles
          • Manage Users (main-menu)
          • Manage Users (Workspace-menu)
          • Template
          • Components
          • Connections
          • Credit Points Consumption
          • Help
        • Projects Introduction
          • Datasets Introduction
          • Create Datasets
          • Datasets List
          • Reuse Datasets
          • Import Export Datasets
          • Register Models Introduction
          • Register Spark ML Model
          • Register H2O Models
          • Register Scikit Model
          • Model Versions
          • Applications Introduction
          • Create CDC Application
          • Actions on CDC Applications
          • Data Validation Introduction
          • Create Jobs
          • Processor Group Introduction
          • Steps to Create a Processor Group
          • Steps to Use a Processor Group
          • Pipeline Introduction
          • Creating a Spark Pipeline
          • Deploy a Pipeline
          • Pipeline Management
          • Continuous Integration Continuous Delivery
          • Configuring Cloudera to Support Lineage
          • Pipeline Listing Page
          • Templates Introduction
          • Create Template
          • Template as Component in a Pipeline
          • Bulk Operations History
          • Workflow Introduction
          • Create a Workflow
          • Create a Template
          • Terminologies
          • Alerts
          • Introduction to Import Export Entities
          • ImportExport Workflow Job
          • Import Python Manifest
          • Action Window
          • Actions on Listing Page
          • Register Entities Introduction
          • Register Components
          • Functions
          • Variables
          • Calendars
          • Compute Environment
          • Sandboxes
          • Non Container based Sandbox
          • Container based Sandbox
          • Sandbox Listing Page
          • Creating Sandbox
          • Version Control Overview
          • Modified Files
          • Revert Notebooks
          • Tags
          • Overview
          • Project Activity
        • Monitoring and Alerts Overview
        • Pipeline Monitoring
        • Error Search
        • Resource Analyzer Report
        • Configuring Alerts
        • Components Introduction
          • Data Sources Introduction
          • Pre-Actions
          • Auto Schema
          • ADLS Data Source - Batch and Streaming
          • Advanced Mongo Data Source
          • Attunity Data Source
          • AWS IoT Data Source
          • Azure Blob Batch Data Source
          • Azure Blob Stream Data Source
          • Batch Cosmos Data Source
          • BigQuery Data Source
          • Cassandra Data Source
          • CDC Data Source
          • Container Data Source
          • Custom Channel Data Source
          • Data Generator Data Source
          • Dataset Channel Data Source
          • Delta (Batch and Streaming) Data Source
          • Delta SQL Data Source
          • Dummy Data Source
          • File Reader Data Source
          • GCS (Batch and Streaming) Data Source
          • Google Spreadsheet Data Source
          • HDFS Data Source
          • Hive Data Source
          • HTTP Data Source
          • HTTPV2 Data Source
          • HTTPV2 Pagination
          • HTTPV2 Incremental Configuration
          • IBM MQ
          • Impala Data Source
          • JDBC Data Source
          • Jira Data Source
          • Kafka Data Source
          • Kinesis Data Source
          • Kudu Data Source
          • MQTT Data Source
          • Native DFS Receiver Data Source
          • Native File Reader Data Source
          • Neo4j
          • OpenJMS Data Source
          • Pub/Sub Data Source
          • RabbitMQ Data Source
          • RDS Data Source
          • Redshift Data Source
          • S3 Batch Data Source
          • S3 Streaming Data Source
          • Salesforce Data Source
          • SFTP Data Source
          • Snowflake Data Source
          • Socket Data Source
          • SQS Data Source
          • Stream Cosmos Data Source
          • Streaming Delta Data Source
          • Tibco Data Source
          • Vertica Data Source
          • VSAM Data Source
          • Processors Introduction
          • Advanced Sort Processor
          • Aggregation Processor
          • Alert Processor
          • App ID Generator Processor
          • Binary Avro Parser Processor
          • Cache Processor
          • Container Processor
          • Custom Processor
          • Data Cleansing Processor
          • Data Quality Management Processor
          • Decoder Processor
          • Decryption Processor
          • Dedup Processor
          • Distinct Processor
          • Drools Processor
          • Drop Processor
          • Encoder Processor
          • Encryption Processor
          • Eviction Processor
          • Expression Evaluator Processor
          • Expression Filter Processor
          • Field Converter Processor
          • Field Flattener Processor
          • Field Replacer Processor
          • Field Splitter Processor
          • Filter Processor
          • Functions Processor
          • Hashing Processor
          • HTTP Processor
          • JDBC Container Processor
          • Join Processor
          • Jolt Processor
          • JSON Parser Processor
          • Limit Processor
          • Masking Processor
          • NA Processor
          • PII Masking Processor
          • Pivot Processor
          • Processor Group Processor
          • Python Processor
          • Rank Processor
          • Redis Lookup
          • Register as Table Processor
          • Rename Processor
          • Repartition Processor
          • Router Processor
          • Sagemaker Processor
          • Scala Processor
          • Schema Flattener Processor
          • Schema Transformer Processor
          • Select Processor
          • Sequence Generator Processor
          • Snowflake Processor
          • Sort Processor
          • SQL Processor
          • Stored Procedure Processor
          • TopNRecords Processor
          • Union Processor
          • Turnpike Processor
          • Watermark Processor
          • XML Parser Processor
          • XSLT Processor
          • Functions Introduction
          • Date Functions
          • Lookup Functions
          • String Functions
          • Math Functions
          • Array Functions
          • Miscellaneous Functions
          • Data Science Introduction
          • ML Models
          • PMML Models
          • H2O Models
          • Scikit Models
          • Models Listing Page
          • Emitters Introduction
          • Post Actions
          • ADLS Emitter
          • Advanced HDFS Emitter
          • Advanced Kafka Emitter
          • Advanced Redshift Emitter
          • AWS IOT Emitter
          • Azure Blob Emitter
          • Batch Emitter
          • BigQuery Emitter
          • Cassandra Emitter
          • Container Emitter
          • Cosmos Emitter
          • Cloud SQL Emitter
          • Delta Emitter
          • Dummy Emitter
          • Elasticsearch Emitter
          • EventBridge Emitter
          • File Writer Emitter
          • GCS Emitter
          • HBase Emitter
          • Hive Emitter
          • HTTP Emitter
          • JDBC Emitter
          • Kafka Emitter
          • Kinesis Emitter
          • Kudu Emitter
          • Mongo Emitter
          • MQTT Emitter
          • NativeHDFS Emitter
          • Neo4j
          • OpenJMS Emitter
          • Pub/Sub Emitter
          • RabbitMQ Emitter
          • RDS Emitter
          • Redshift Emitter
          • S3 Emitter
          • Salesforce Emitter
          • Snowflake Emitter
          • Solr Emitter
          • SQS Emitter
          • Streaming Emitter
          • Vertica Emitter

      Pipelines

      Pipeline Introduction →
      Creating a Spark Pipeline →
      Deploy a Pipeline →
      Pipeline Management →
      Continuous Integration Continuous Delivery →
      Configuring Cloudera to Support Lineage →
      Pipeline Listing Page →
      • © 2024 Gathr, All right reserved
      • Cookie Policy
      • Privacy Policy
      Top