文章目录

    • 前言
    • 技术积累
      • 什么是Helm
      • StorageClass
      • 使用的工具版本
    • helm 安装 MySQL 1主2从
      • 1. 添加 bitnami 的仓库
      • 2. 查询 MySQL 资源
      • 3. 拉取 MySQL chart 到本地
      • 4. 对chart 本地 values-test.yaml 修改
      • 5. 对本地 templates 模板 修改
      • 6. 安装 MySQL 集群
      • 7. 查看部署的 MySQL 集群
      • 8. 连接 MySQL 集群 验证服务
    • ingress-nginx开放主从端口

前言

K8S对于云原生部署有着至关重要的作用，几乎所有的应用和中间件都可以部署在K8S，让其帮助我们进行管理。一般情况下我们推荐数据库不使用容器化部署，但是在某些场合为了方便管理也可以采用该种方式。今天我们就用在K8S集群部署MySQL，仅当作学习学习，在生产环境谨慎使用。

技术积累

什么是Helm

Helm是K8S中包管理工具，包管理器类似于我们在 Ubuntu 中使用的apt、Centos中使用的yum一样，能快速查找、下载和安装软件包， 能够将一组K8S资源打包统一管理, 是查找、共享和使用为Kubernetes构建的软件的最佳方式。具体介绍和安装使用参见之前博文： Kubernetes包管理工具Helm简介及使用

StorageClass

由于数据库需要使用存储资源，K8S集群中我们可手动创建pv、pvc,StorageClass工具则可以自动车床件pvc。具体简介和使用参见之前博文：【实战】Kubernetes安装持久化工具NFS-StorageClass

使用的工具版本

软件版本
chart9.12.0
mysql8.0.34
kubernetesversion v1.27.2
helmversion v3.12.1

helm 安装 MySQL 1主2从

1. 添加 bitnami 的仓库

$ helm repo add bitnami https://charts.bitnami.com/bitnami

2. 查询 MySQL 资源

$ helm repo update

[root@master k8s]# helm search repo mysql
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/mysql 9.12.1 8.0.34 MySQL is a fast, reliable, scalable, and easy t…
bitnami/phpmyadmin 12.1.0 5.2.1 phpMyAdmin is a free software tool written in P…
bitnami/mariadb 13.1.2 11.0.3 MariaDB is an open source, community-developed …
bitnami/mariadb-galera 9.1.1 11.0.3 MariaDB Galera is a multi-primary database clus…

3. 拉取 MySQL chart 到本地

$ mkdir -p /k8s/mysql && cd /k8s/mysql
#拉取 chart 到本地 /root/mysql 目录
$ helm pull bitnami/mysql –version 9.12.1
$ tar -xvf mysql-9.12.1.tgz
$ cp mysql/values.yaml ./values-test.yaml

#查看当前目录层级
[root@master mysql]# tree -L 2
.
├── mysql
│ ├── Chart.lock
│ ├── charts
│ ├── Chart.yaml
│ ├── README.md
│ ├── templates
│ ├── values.schema.json
│ └── values.yaml
├── mysql-9.12.1.tgz
└── values-test.yaml

3 directories, 7 files

4. 对chart 本地 values-test.yaml 修改

查看集群 storageclasses
[root@master mysql]# kubectl get storageclass

NAMEPROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEnfs-storage (default) nfs-provisioner DeleteImmediate false10d

修改配置
$ vim values-test.yaml

#镜像修改image:registry: registry.cn-hangzhou.aliyuncs.comrepository: senfel/mysqltag: 8.0.34-debian-11-r31#架构改为集群复制architecture: replication#root密码 主从复制密码auth:rootPassword: "123456root"#主节点primary:configuration: |-[mysqld]default_authentication_plugin=caching_sha2_password ## 加密方式persistence: ## 储存方式storageClass: "nfs-storage"#从节点副本数secondary:replicaCount: 2#从节点secondary:configuration: |-[mysqld]default_authentication_plugin=caching_sha2_password## 加密方式read_only=1 ##限定普通用户只读super_read_only=on ##限定root只读persistence: ##储存方式storageClass: "nfs-storage"

5. 对本地 templates 模板 修改

mysql5会有以下问题，mysql8无需处理
[root@master mysql]# kubectl api-versions

# k8s v1.27.2 版本中，statefulsets 版本为：apps/v1而不是模板中 apps/v1beta1 的版本# 因此需要替换 master-statefulset.yaml 和 slave-statefulset.yaml 模板$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/master-statefulset.yaml$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/slave-statefulset.yaml未替换模板中 statefulsets 版本，直接安装服务会报错$ helm install mysql-cluster mysql -f values-test.yaml Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "StatefulSet" in version "apps/v1beta1"

6. 安装 MySQL 集群

#创建 mysql-cluster 名称空间
$ kubectl create ns mysql-cluster

#安装 MySQL 集群
$ helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml

##helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 mysql/values.yaml 文件中配置

[root@master mysql]# helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml

NAME: mysql-clusterLAST DEPLOYED: Mon Sep4 15:09:28 2023NAMESPACE: mysql-clusterSTATUS: deployedREVISION: 1TEST SUITE: NoneNOTES:CHART NAME: mysqlCHART VERSION: 9.12.1APP VERSION: 8.0.34** Please be patient while the chart is being deployed **Tip:Watch the deployment status using the command: kubectl get pods -w --namespace mysql-clusterServices:echo Primary: mysql-cluster-primary.mysql-cluster.svc.cluster.local:3306echo Secondary: mysql-cluster-secondary.mysql-cluster.svc.cluster.local:3306Execute the following to get the administrator credentials:echo Username: rootMYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace mysql-cluster mysql-cluster -o jsonpath="{.data.mysql-root-password}" | base64 -d)To connect to your database:1. Run a pod that you can use as a client:kubectl run mysql-cluster-client --rm --tty -i --restart='Never' --imageregistry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 --namespace mysql-cluster --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash2. To connect to primary service (read/write):mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"3. To connect to secondary service (read-only):mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"

7. 查看部署的 MySQL 集群

[root@master mysql]# kubectl get svc,pods -n mysql-cluster

NAME TYPECLUSTER-IPEXTERNAL-IP PORT(S)AGEservice/mysql-cluster-primaryClusterIP 10.102.253.38 <none>3306/TCP 2m2sservice/mysql-cluster-primary-headless ClusterIP None<none>3306/TCP 2m2sservice/mysql-cluster-secondaryClusterIP 10.102.8.228<none>3306/TCP 2m2sservice/mysql-cluster-secondary-headless ClusterIP None<none>3306/TCP 2m2sNAMEREADY STATUSRESTARTS AGEpod/mysql-cluster-primary-0 1/1 Running 02m2spod/mysql-cluster-secondary-0 1/1 Running 02m2spod/mysql-cluster-secondary-1 1/1 Running 080s

#查看 pvc
[root@master mysql]# kubectl get pvc -n mysql-cluster

NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEdata-mysql-cluster-primary-0 Boundpvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107 8GiRWOnfs-storage4m12sdata-mysql-cluster-secondary-0 Boundpvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32 8GiRWOnfs-storage4m12sdata-mysql-cluster-secondary-1 Boundpvc-90072614-6cbb-48f2-8cdf-a8816557e46b 8GiRWOnfs-storage3m30s

#查看 pv
[root@master mysql]# kubectl get pv | grep mysql

pvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107 8GiRWODelete Bound mysql-cluster/data-mysql-cluster-primary-0nfs-storage 4m45spvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32 8GiRWODelete Bound mysql-cluster/data-mysql-cluster-secondary-0nfs-storage 4m45spvc-90072614-6cbb-48f2-8cdf-a8816557e46b 8GiRWODelete Bound mysql-cluster/data-mysql-cluster-secondary-1nfs-storage 4m3spvc-a5673e1b-e3b2-415f-b2bb-c649eb4cd33d 8GiRWODelete Bound default/data-mysql-cluster-secondary-0nfs-storage 32mpvc-a683be89-289c-4088-b46f-f2af4d322a1a 8GiRWODelete Bound default/data-mysql-cluster-primary-0nfs-storage 32m

8. 连接 MySQL 集群 验证服务

#启动一个临时容器
$ kubectl run mysql-cluster-client –rm –tty -i –restart=‘Never’ –image registry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 –namespace mysql-cluster –env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD –command – bash

##登陆 MySQL Master节点
$ mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p

Enter password: 123456rootmysql> show databases;+--------------------+| Database |+--------------------+| information_schema || my_database|| mysql|| performance_schema || sys|+--------------------+5 rows in set (0.01 sec)

#查看主从状态
#查看File和Position的值，在从库配置中会显示。

mysql> show master status\G

*************************** 1. row *************************** File: mysql-bin.000003 Position: 157 Binlog_Do_DB: Binlog_Ignore_DB:Executed_Gtid_Set:1 row in set (0.00 sec)

##登陆从库，查看主从同步状态
$ mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p
Enter password: 123456root

mysql> show slave status\G;

*************************** 1. row *************************** Slave_IO_State: Waiting for source to send eventMaster_Host: mysql-cluster-primaryMaster_User: replicatorMaster_Port: 3306Connect_Retry: 10Master_Log_File: mysql-bin.000003Read_Master_Log_Pos: 157 Relay_Log_File: mysql-relay-bin.000006Relay_Log_Pos: 373Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: YesSlave_SQL_Running: YesReplicate_Do_DB:Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table:Replicate_Wild_Do_Table:Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0Exec_Master_Log_Pos: 157Relay_Log_Space: 799Until_Condition: None Until_Log_File:Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path:Master_SSL_Cert:Master_SSL_Cipher: Master_SSL_Key:Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: NoLast_IO_Errno: 0Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error:Replicate_Ignore_Server_Ids: Master_Server_Id: 313Master_UUID: ff169bc1-4af1-11ee-b795-f6df04eaa47a Master_Info_File: mysql.slave_master_infoSQL_Delay: 0SQL_Remaining_Delay: NULLSlave_SQL_Running_State: Replica has read all relay log; waiting for more updates Master_Retry_Count: 86400Master_Bind:Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set:Executed_Gtid_Set:Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: Master_public_key_path:Get_master_public_key: 0Network_Namespace:1 row in set, 1 warning (0.00 sec)

至此，使用Helm在K8S集群安装MySQL主从实战演示完成。

ingress-nginx开放主从端口

ingress-nginx是K8S开放端口的工具，可以直接将端口映射到集群外部。
#修改ingress-nginx tcp配置
$ vim values.yaml

#格式 "ns":"svc:port"tcp: {"56379":"redis-cluster/redis-cluster:6379","53306": "mysql-cluster/mysql-cluster-primary:3306","54406": "mysql-cluster/mysql-cluster-secondary:3306"}

#升级releas
$ helm upgrade ingress-nginx -n ingress-nginx .

#验证服务器端口是否开启
[root@node2 ~]# netstat -nplt | grep 06

tcp00 0.0.0.0:53306 0.0.0.0:* LISTEN16944/nginx: mastertcp00 0.0.0.0:54406 0.0.0.0:* LISTEN16944/nginx: mastertcp6 00 :::53306:::*LISTEN16944/nginx: mastertcp6 00 :::54406:::*LISTEN16944/nginx: master

查看最终的nginx.conf
#获取ingress-nginx pods
[root@node2 ~]# kubectl get pods -n ingress-nginx

NAME READY STATUSRESTARTS AGEingress-nginx-controller-75x86 1/1 Running 08m51singress-nginx-controller-jgzks 1/1 Running 08m29s

#查看最终的nginx.conf
[root@master ingress-nginx]# kubectl exec ingress-nginx-controller-75×86 -n ingress-nginx – cat /etc/nginx/nginx.conf

至此TCP端口已暴露完成，对于UDP也是一样的修改value.yaml配置即可

外网navicat连接验证

两台ingress-nginx服务物理机都开放了端口，都可以连接！！！