Confluent Cloud Connection

Kafka Confluent Cloud is a fully managed event streaming platform that simplifies the deployment and operation of Apache Kafka for real-time data streaming applications.

This topic describes how to authenticate to Confluent Cloud and configure any necessary connection properties in the Confluent Cloud connection connector.

Prerequisites #

• Ensure you have an active account on Confluent Cloud with the necessary access permissions.

• Obtain the Kafka cluster credentials, including the Bootstrap Servers (brokers), API Key, and API Secret.

To obtain the Bootstrap Servers (brokers), API Key, and API Secret, follow these steps within the Kafka Confluent Cloud platform:

1. Log in to Confluent Cloud:

• Access the Confluent Cloud platform by visiting the Confluent Cloud website and logging in to your account.

2. Create or Access a Cluster:

• If you don’t have an existing cluster, create one within the Confluent Cloud dashboard. If you already have a cluster, select it from the dashboard.

3. Access Cluster Settings:

• Within the cluster dashboard, look for settings or configuration options related to cluster credentials and security.

4. Generate or Retrieve API Key and API Secret:

• Typically, you’ll find a section for “API Keys” or “Credentials” within the cluster settings.

  If you haven’t generated API credentials before, you may need to create a new API Key. This often involves providing a name for the key.

  Once the API Key is generated, you will usually be shown the API Key and API Secret.

  Make sure to copy and securely store these credentials, as they are sensitive and will not be displayed again.

5. Bootstrap Servers (Brokers):

• The bootstrap servers, which are the Kafka broker endpoints, are typically provided in the Confluent Cloud documentation or cluster settings.

  Look for the list of bootstrap servers or broker endpoints, and ensure you have them handy.

6. Kafka Topic

• You will need a Kafka topic which is set up with a message value schema to create the Confluent Kafka connection.

Connection Configuration #

Each connection property available in the Confluent Cloud connector is explained below.

Connection Name #

A unique name for creating the connection is to be given.

Scope #

Define the connection scope to customize their accessibility.

• Organization: Accessible to organization users across all Gathr projects for usage in applications.

• Project: Accessible to organization users limited to projects that are specified by the connection owner for usage in applications.

• Private: Accessible only to the connection owner for usage in applications.

Bootstrap Servers #

List of Kafka nodes against which connections need to be created.

API Key #

API Key for your Confluent Cloud cluster.

To read about the steps required to create a Cloud API key, click here.

API Secret #

API Secret for your Confluent Cloud cluster

Schema Registry URL #

The URL pointing to your Confluent Schema Registry instance should be provided.

Schema Registry API Key #

The security credential used for authentication when interacting with the Confluent Schema Registry should be provided.

Schema Registry API Secret #

The security credential used for authentication when interacting with the Confluent Schema Registry should be provided.

Add Configuration: Additional properties can be added using this option as key-value pairs.

After entering all the details, click on the TEST button.

If the connection service identification and authentication details are provided correctly, a success message stating “connection available” is generated. Click on the CREATE button to save the changes.

If the details are incorrect or the server is down, you will get a message “Connection unavailable”.