-
Notifications
You must be signed in to change notification settings - Fork 709
Private link connection #22192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shiyuhang0
wants to merge
32
commits into
pingcap:release-8.5
Choose a base branch
from
shiyuhang0:private-link-connection
base: release-8.5
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,243
−0
Open
Private link connection #22192
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
2f7c7ba
add plc
shiyuhang0 77059dc
opt
shiyuhang0 0479792
some fix
shiyuhang0 992882e
some fix
shiyuhang0 ca4dad3
add toc
shiyuhang0 4be169d
some fix by ai
shiyuhang0 f4c6bf0
Apply suggestions from code review
shiyuhang0 ca176a2
fix
shiyuhang0 ba55775
fix lint
shiyuhang0 a5d2ad4
fix according to the preview
shiyuhang0 ccfd4fe
Apply suggestions from code review
shiyuhang0 a0ef99a
Update Alibaba Cloud RDS private link setup docs
hfxsd aed869e
Update tidb-cloud/serverless-private-link-connection-to-alicloud-rds.md
hfxsd f6fe625
Update serverless-private-link-connection-to-alicloud-rds.md
hfxsd 203e2eb
Update serverless-private-link-connection-to-alicloud-rds.md
hfxsd 56e443e
Update serverless-private-link-connection-to-alicloud-rds.md
hfxsd fc113a1
Update docs for Private Link Connection setup
hfxsd 6466583
remove subnet group in alicloud rds
shiyuhang0 c2f9643
add link for subnet grup
shiyuhang0 156c135
add alicloud self-hosted kafka
shiyuhang0 4c5a5fa
some opt
shiyuhang0 ff6d3c0
Update private link connection docs for clarity and accuracy
hfxsd cd7c7ab
Merge branch 'private-link-connection' of https://github.com/shiyuhan…
hfxsd adfe661
Revise and clarify private link connection guides
hfxsd 5b53ba7
Update serverless-private-link-connection-to-self-hosted-kafka-in-aws.md
hfxsd 319e2f6
Update serverless-private-link-connection-to-self-hosted-kafka-in-aws.md
hfxsd 470d9ab
Update Alibaba Cloud and AWS private link docs for clarity
hfxsd d097da9
some opts for confluent
shiyuhang0 102aa34
some opts for confluent
shiyuhang0 fdb5c9d
fix provider in alibaba kafka
shiyuhang0 2b4e737
fix
shiyuhang0 d3c26e9
fix
shiyuhang0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
129 changes: 129 additions & 0 deletions
129
tidb-cloud/serverless-private-link-connection-to-alicloud-rds.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| --- | ||
| title: Connect to Alibaba Cloud ApsaraDB RDS for MySQL via a Private Link Connection | ||
| summary: Learn how to connect to an Alibaba Cloud ApsaraDB RDS for MySQL instance using an Alibaba Cloud Endpoint Service private link connection. | ||
| --- | ||
|
|
||
| # Connect to Alibaba Cloud ApsaraDB RDS for MySQL via a Private Link Connection | ||
|
|
||
| This document describes how to connect to an Alibaba Cloud ApsaraDB RDS for MySQL instance using an Alibaba Cloud Endpoint Service private link connection. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - ApsaraDB RDS for MySQL: ensure you have an existing ApsaraDB RDS for MySQL instance or the permissions required to create one. | ||
|
|
||
| - Alibaba Cloud permissions: verify that your account has the following authorizations to manage networking components: | ||
|
|
||
| - Manage load balancer | ||
| - Manage endpoint services | ||
|
|
||
| - {{{ .essential }}} information: confirm that your {{{ .essential }}} is active in Alibaba Cloud. Retrieve and save the following details for later use: | ||
|
|
||
| - Account ID | ||
| - Availability Zones (AZ) | ||
|
|
||
| To view the the Alibaba Cloud account ID and available zones, do the following: | ||
|
|
||
| 1. In the [TiDB Cloud console](https://tidbcloud.com), navigate to the cluster overview page of the TiDB cluster, and then click **Settings** > **Networking** in the left navigation pane. | ||
| 2. On the **Private Link Connection For Dataflow**, click **Create Private Link Connection**. | ||
| 3. You can find the Alibaba Cloud account ID and available zones. | ||
|
|
||
| ## Step 1. Set up an ApsaraDB RDS for MySQL instance | ||
|
|
||
| Identify an Alibaba Cloud ApsaraDB RDS for MySQL you want to use, or [set up a new RDS](https://www.alibabacloud.com/help/en/rds/apsaradb-rds-for-mysql/step-1-create-an-apsaradb-rds-for-mysql-instance-and-configure-databases). | ||
|
|
||
| To ensure successful connectivity, your ApsaraDB RDS for MySQL instance must meet the following requirements: | ||
|
|
||
| - Region match: the instance must reside in the same Alibaba Cloud region as your {{{ .essential }}} cluster. | ||
| - AZ (Availability Zone) availability: the availability zones must overlap with those of your {{{ .essential }}} cluster. | ||
| - Network accessibility: the instance must be accessible within the VPC, with an appropriately configured IP allowlist. | ||
|
|
||
| > **Note** | ||
| > | ||
| > Cross-region connections for ApsaraDB RDS for MySQL are not supported. | ||
| ## Step 2. Expose the ApsaraDB RDS for MySQL instance as an endpoint service | ||
|
|
||
| ### 1. Set up the load balancer | ||
|
|
||
| Set up the load balancer in the same region of your ApsaraDB RDS for MySQL as follows: | ||
|
|
||
| 1. Go to [Server Groups](https://slb.console.alibabacloud.com/nlb/ap-southeast-1/server-groups) to create a server group. | ||
|
|
||
| - **Server Group Type**: select `IP` | ||
| - **VPC**: enter the VPC where your ApsaraDB RDS for MySQL is located | ||
| - **Backend Server Protocol**: select `TCP` | ||
|
|
||
| Click the created server group to add backend servers. Add the IP address of your ApsaraDB RDS for MySQL instance. You can ping the for MySQL RDS endpoint to get the IP address. | ||
|
|
||
| 2. Go to [NLB](https://slb.console.alibabacloud.com/nlb) to create a network load balancer. | ||
|
|
||
| - **Network Type**: select `Internal-facing` | ||
| - **VPC**: select the VPC where your ApsaraDB RDS for MySQL is located | ||
| - **Zone**: it must overlap with your {{{ .essential }}} cluster | ||
| - **IP Version**: select `IPv4` | ||
|
|
||
| Find the load balancer you created, and then click **Create Listener**: | ||
|
|
||
| - **Listener Protocol**: select `TCP` | ||
| - **Listener Port**: enter the database port, for example, `3306` for MySQL | ||
| - **Server Group**: choose the server group you created in the previous step | ||
|
|
||
| ### 2. Set up an endpoint service | ||
|
|
||
| Set up the endpoint service in the same region of your ApsaraDB RDS for MySQL: | ||
|
|
||
| 1. Go to [Endpoint service](https://vpc.console.alibabacloud.com/endpointservice) to create an endpoint service. | ||
|
|
||
| - **Service Resource Type**: select `NLB` | ||
| - **Select Service Resource**: select all zones that NLB is in, and choose the NLB that you created in the previous step | ||
| - **Automatically Accept Endpoint Connections**: it is recommended to choose `No` | ||
|
|
||
| 2. Go to the details page of the endpoint service, and copy the **Endpoint Service Name**, for example, `com.aliyuncs.privatelink.<region>.xxxxx`. You need to use it for TiDB Cloud later. | ||
|
|
||
| 3. On the detail page of the endpoint service, click the **Service Whitelist** tab, click **Add to Whitelist**, and then enter the TiDB Cloud account ID. For more information about how to get the account ID, see [Prerequisites](#prerequisites). | ||
|
|
||
| ## Step 3. Create a private link connection in TiDB Cloud | ||
|
|
||
| You can create a private link connection using the TiDB Cloud console or the TiDB Cloud CLI. | ||
|
|
||
| <SimpleTab> | ||
| <div label="Console"> | ||
|
|
||
| 1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. | ||
|
|
||
| > **Tip:** | ||
| > | ||
| > You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters. | ||
| 2. Click the name of your target cluster to go to its overview page, and then click **Settings** > **Networking** in the left navigation pane. | ||
|
|
||
| 3. In the **Private Link Connection For Dataflow** area, click **Create Private Link Connection**. | ||
|
|
||
| 4. In the **Create Private Link Connection** dialog, enter the required information: | ||
|
|
||
| - **Private Link Connection Name**: enter a name for the private link connection. | ||
| - **Connection Type**: select **Alibaba Cloud Endpoint Service**. If you cannot find this option, ensure that your cluster is created on Alibaba Cloud. | ||
| - **Endpoint Service Name**: enter the endpoint service name you obtained in [Set up an endpoint service](#2-set-up-an-endpoint-service). | ||
|
|
||
| 5. Click **Create**. | ||
|
|
||
| 6. Go back to the detail page of the endpoint service on [Alibaba Cloud console](https://account.alibabacloud.com/login/login.htm). In the **Endpoint Connections** tab, allow the endpoint connection request from TiDB Cloud. | ||
|
|
||
| </div> | ||
|
|
||
| <div label="CLI"> | ||
|
|
||
| To create a private link connection using the TiDB Cloud CLI: | ||
|
|
||
| 1. Run the following command: | ||
|
|
||
| ```shell | ||
| ticloud serverless private-link-connection create -c <cluster-id> --display-name <display-name> --type ALICLOUD_ENDPOINT_SERVICE --alicloud.endpoint-service-name <endpoint-service-name> | ||
| ``` | ||
|
|
||
| 2. Go back to the detail page of the endpoint service on [Alibaba Cloud console](https://account.alibabacloud.com/login/login.htm). In the **Endpoint Connections** tab, allow the endpoint connection request from TiDB Cloud. | ||
|
|
||
| </div> | ||
| </SimpleTab> | ||
|
|
||
| For more information, see [Create an AliCloud Endpoint Service Private Link Connection](/tidb-cloud/serverless-private-link-connection.md#create-an-alicloud-endpoint-service-private-link-connection). |
132 changes: 132 additions & 0 deletions
132
tidb-cloud/serverless-private-link-connection-to-aws-confluent.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| --- | ||
| title: Connect to AWS Confluent via a Private Link Connection | ||
| summary: Learn how to connect to an AWS Confluent instance using an AWS Confluent Endpoint Service private link connection. | ||
| --- | ||
|
|
||
| # Connect to Confluent Cloud via a Private Link Connection | ||
|
|
||
| > **Note** | ||
| > | ||
| > Only Confluent Cloud Dedicated clusters on AWS are supported. | ||
| This document describes how to connect to a Confluent Cloud Dedicated cluster on AWS using an AWS Endpoint Service private link connection. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - You have a Confluent Cloud account. | ||
|
|
||
| - Confirm that your {{{ .essential }}} is active in AWS. Retrieve and save the following details for later use: | ||
|
|
||
| - Account ID | ||
| - Availability Zones (AZ) | ||
|
|
||
| To view the the AWS account ID and available zones, do the following: | ||
|
|
||
| 1. In the [TiDB Cloud console](https://tidbcloud.com), navigate to the cluster overview page of the TiDB cluster, and then click **Settings** > **Networking** in the left navigation pane. | ||
| 2. On the **Private Link Connection For Dataflow**, click **Create Private Link Connection**. | ||
| 3. You can find the AWS account ID and available zones. | ||
|
|
||
| ## Step 1. Set up a Confluent Cloud network | ||
|
|
||
| Identify a Confluent Cloud network you want to use, or [create a new Confluent Cloud network on AWS](https://docs.confluent.io/cloud/current/networking/ccloud-network/aws.html#create-ccloud-network-aws). | ||
|
|
||
| The Confluent Cloud network must meet the following requirements: | ||
|
|
||
| - Type: the network must be a privatelink network. | ||
| - Region match: the instance must reside in the same AWS region as your {{{ .essential }}} cluster. | ||
| - AZ (Availability Zone) availability: the availability zones must overlap with those of your {{{ .essential }}} cluster. | ||
|
|
||
| To get the unique name of the Confluent Cloud network: | ||
|
|
||
| 1. On the `Network overview` page, obtain the `DNS subdomain` of the Confluent Cloud network. | ||
| 2. Extract the unique name from it. For example, if the `DNS subdomain` is `use1-az1.domnprzqrog.us-east-1.aws.confluent.cloud`, then the unique name is `domnprzqrog.us-east-1`. | ||
| 3. Save the unique name for later use. | ||
|
|
||
| > **Note** | ||
| > | ||
| > The Confluent Cloud Dedicated cluster must be deployed under this network. | ||
| ## Step 2. Add a PrivateLink Access to the network | ||
|
|
||
| Add a PrivateLink Access to the network you identified or set up in Step 1. Refer to [Add a PrivateLink Access in Confluent Cloud](https://docs.confluent.io/cloud/current/networking/private-links/aws-privatelink.html#add-a-privatelink-access-in-ccloud). | ||
|
|
||
| During the process, you need to: | ||
|
|
||
| - Provide the TiDB Cloud AWS account ID that you obtain in [Prerequisites](#prerequisites). | ||
| - Save the `VPC Service Endpoint` provided by Confluent Cloud for later use, usually in the format of `com.amazonaws.vpce.<region>.vpce-svc-xxxxxxxxxxxxxxxxx`. | ||
|
|
||
| ## Step 3. Create a private link connection in TiDB Cloud | ||
|
|
||
| ### 1. Create the AWS Endpoint Service private link connection | ||
|
|
||
| You can create a private link connection using the TiDB Cloud console or the TiDB Cloud CLI. | ||
|
|
||
| <SimpleTab> | ||
| <div label="Console"> | ||
|
|
||
| 1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. | ||
|
|
||
| > **Tip:** | ||
| > | ||
| > You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters. | ||
| 2. Click the name of your target cluster to go to its overview page, and then click **Settings** > **Networking** in the left navigation pane. | ||
|
|
||
| 3. In the **Private Link Connection For Dataflow** area, click **Create Private Link Connection**. | ||
|
|
||
| 4. Enter the required information in the **Create Private Link Connection** dialog: | ||
|
|
||
| - **Private Link Connection Name**: enter a name for the private link connection. | ||
| - **Connection Type**: select **AWS Endpoint Service**. If you cannot find this option, ensure that your cluster is created on AWS. | ||
| - **Endpoint Service Name**: enter the `VPC Service Endpoint` you obtained in [Step 2](#step-2-add-a-privatelink-access-to-the-network). | ||
|
|
||
| 5. Click **Create**. | ||
|
|
||
| </div> | ||
|
|
||
| <div label="CLI"> | ||
|
|
||
| To create a private link connection using the TiDB Cloud CLI, run the following command: | ||
|
|
||
| ```shell | ||
| ticloud serverless private-link-connection create -c <cluster-id> --display-name <display-name> --type AWS_ENDPOINT_SERVICE --aws.endpoint-service-name <endpoint-service-name> | ||
| ``` | ||
|
|
||
| </div> | ||
| </SimpleTab> | ||
|
|
||
| You can also refer to [Create an AWS Endpoint Service Private Link Connection](/tidbcloud/serverless-private-link-connection#create-an-aws-endpoint-service-private-link-connection) for more details. | ||
|
|
||
| ### 2. Attach domains to the private link connection | ||
|
|
||
| You can create a private link connection using the TiDB Cloud console or the TiDB Cloud CLI. | ||
|
|
||
| <SimpleTab> | ||
| <div label="Console"> | ||
|
|
||
| 1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. | ||
|
|
||
| > **Tip:** | ||
| > | ||
| > You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters. | ||
| 2. Click the name of your target cluster to go to its overview page, and then click **Settings** > **Networking** in the left navigation pane. | ||
|
|
||
| 3. In the **Private Link Connection For Dataflow** area, choose the target private link connection and then click **...**. | ||
|
|
||
| 4. Click **Attach Domains**. | ||
|
|
||
| 5. In the **Attach Domains** dialog, choose the **Confluent Cloud** domain type, enter the Confluent unique name to generate the domains, and then click **Attach Domains** to confirm. | ||
|
|
||
| </div> | ||
|
|
||
| <div label="CLI"> | ||
|
|
||
| ```shell | ||
| ticloud serverless private-link-connection attach-domains -c <cluster-id> --private-link-connection-id <private-link-connection-id> --type CONFLUENT --unique-name <unique-name> | ||
| ``` | ||
|
|
||
| </div> | ||
| </SimpleTab> | ||
|
|
||
| For more information, see [Attach Domains to a Private Link Connection](/tidb-cloud/serverless-private-link-connection.md#attach-domains-to-a-private-link-connection). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.