Actions on CDC Applications

Once you have created the application, you can view it on the Applications dashboard.

You may start or stop the CDC application by clicking on the Start or Stop button.

As you click the ellipsis on the right-bottom corner of the tile you will be able to Edit, Delete, Schedule and View the application.

Configure Job

Click the gear icon to Configure Job. A window appears to configure the job of the particular pipeline. Click Gear icon under Action Column to proceed further. User also has an option to Replicate Configuration is available under Action Column.

Start

Once you click on Start button, you get an option to choose the pipeline created within the application and start them. You will see only the stopped pipeline(s) here before you start the operation.

Stop

Likewise, you may stop the running application by clicking on the Stop button or choose a specific pipeline to stop running it. You will only see the started pipeline here to perform the stop operation.

Edit

Once you click on the Edit button, you will be able to make edits in the application. You will be transported to the Source panel from where you can start making the required changes in the existing CDC application from the beginning.

Delete

If you need to Delete the application, click on the ellipsis on the left side of the CDC application tile. Click Delete to delete the existing CDC application.

Schedule

In the schedule application window you can view the scheduled and unscheduled pipelines. To schedule a pipeline, select one and click the Schedule button. Similarly, you can un-schedule and re-schedule the pipelines.

View

As you click on the View button to navigate across the application dashboard, you can view three default widgets available on application monitoring dashboard.

Default Widgets

Once you open the application dashboard, you see three default widgets on dashboard page:

In the Application components widget, you can view the components including Data Pipelines, Source Tables, Target Tables and the Audit activity.

Data Pipelines

As you click the View Data Pipelines button, you will be able to see all Data Pipelines created by the application.

Here, you can Monitor your CDC application, view audit activity, explore history, do scheduling, download version, submit pipeline logs and edit.

Monitor

Upon clicking monitor, you will be able to view the error metrics of a pipeline, search the Host, select the time unit and refresh intervals of the application. The error search tab lets you search errors using keywords and filters.

View Audit Activity

Upon clicking the View Audit Activity, a window displays the audit activities performed on the pipeline.

History

This feature lets you view the History of the pipeline, that is pipeline details including the start time, end time, status and the application id.

Stats Notification

This tab enables you to send emails with the statistics of the data pipeline, to a user along with the customized details.

Click on Stats Notification from the pipeline tile of the Data Pipeline homepage.

This page allows you to enter the email address, on which you will receive the emails. Here, choose the duration of the emails.

Next to Configuration is Metrics. It shows all the metrics of the pipeline that will be sent in the email.

Download Version

You can also download a version of the pipeline.

Pipeline Submission Logs

Logs of the data pipeline can be viewed by either clicking on Application ID or the Pipeline Submission Logs.

For more details about each section you can refer the Pipeline Submission Logs → section.

View Source Tables

Within the application components you can view the source tables. You can also search a specific source in the search bar.

View Target Tables

The user can view target tables enlisted or search a particular target table in the search bar.

View Audit

The Audit captures and presents all important activities and events in the platform for auditing.

Interaction events include pipeline creation, pipeline inspection, application creation, application edit, license uploads, test-cases execution, configuration updates, connection updates, notebook creation, model validation and all other interactions possible within Gathr. The “View Audit” button available within Application dashboard shows the audit events performed on that particular application.

In the Data Pipelines Status widget you can click and view the total data pipelines, running pipelines, stopped and failed pipelines. You can also view the run history.

The user can write the logical description about the CDC application and save it. User can also upload an image in description. The description which user provides gets visible on Applications page within tile.

Custom Widgets

Four types of customized widgets can also be added in application. These are explained below:

You can add the widget by clicking on the + ADD WIDGET icon.

In this widget you can view the various operations (insert, update and delete) that were performed across the application at data pipeline level.

In order to create widget, you have to follow the below mentioned steps:

Field	Description
Title	You will have to mention a title for the widget.
Data Pipeline	You will have to choose all or the individual data pipeline from drop-down for the widget creation.
Operation	You can choose one of the four options: All: You can choose the All option when you want to see the stats for all of the operations. Insert: You can choose the Insert option when you want to see the stats for insert operation. Update: You can choose the Update option when you want to see the stats for insert operation. Delete: You can choose the Delete option when you want to see the stats for insert operation.
Execution	You can choose one of the two options: All: You can choose the All option when you want to have stats for all execution. Last Run: You can choose the Last Run option when you want to have stats for last run only.
Graph	You can choose one of the two options: Pie: This graph option will be available only when you have selected Last Run in execution option. Column: This Graph option will be used for all other cases when you haven’t chosen the Last Run option in the execution input.
Refresh Interval	You can configure the refresh interval of widget from this configuration. Widgets will get auto-refreshed after each configured interval.

You can view the insert, update and delete operations performed at the run time of application.

In order to create widget, you have to follow the below mentioned steps:

Field	Description
Title	You will have to mention a title for the widget.
Target	You will have to choose all or the individual source to target table mapping from drop-down for the widget creation.
Operation	You can choose one of the four options: All: You can choose the All option when you want to see the stats for all of the operations. Insert: You can choose the Insert option when you want to see the stats for insert operation. Update: You can choose the Update option when you want to see the stats for insert operation. Delete: You can choose the Delete option when you want to see the stats for insert operation.
Execution	You can choose one of the two options: All: You can choose the All option when you want to have stats for all execution. Last Run: You can choose the Last Run option when you want to have stats for last run only.
Graph	You can choose one of the two options: Pie: This graph option will be available only when you have selected Last Run in execution option. Column: This Graph option will be used for all other cases when you haven’t chosen the Last Run option in the execution input.
Refresh Interval	You can configure the refresh interval of widget from this configuration. Widgets will get auto-refreshed after each configured interval.

In this widget, you can view the details of the source table which is mapped to the target table along with the configured SCD type.

Data Flow Scheduling

Add this widget by Add Widget button. In this widget you can view all the pipeline that are scheduled and at what frequency.