category
tags
type
status
slug
date
summary
icon
password
前言
我们团队在服务器环境中中统一使用KubeSphere作为k8s的默认可视化界面,它可以方便地管理k8s环境中的各个元素。包括每个节点的内存、硬盘、CPU用量等等。
![notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2F6c096b44-beb9-48ee-8f92-1efdde47f3a3%2Fe5aa01db-7618-46a4-8871-dec3d3162a71%2FUntitled.png?table=block&id=0e7bbcb9-1e10-48ab-8dbc-8110e6a70bfd&t=0e7bbcb9-1e10-48ab-8dbc-8110e6a70bfd&width=1896&cache=v2)
创建各个中间件
实际生产时,以下组件都需要做HA高可用,这里仅演示一个单机测试版本。
1. MySQL
核心三要素
部署方式(StatefulSet),数据存储 (PVC),配置文件(ConfigMap)。
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F132126b3-bc93-46e0-8b23-7c377e5ba22d%2FUntitled.png?table=block&id=c08b9b28-d66b-40f6-bb20-e4958ab5fd41&t=c08b9b28-d66b-40f6-bb20-e4958ab5fd41&width=432&cache=v2)
开始前准备
可在官方镜像文档描述查看mysql容器启动的相关配置
传统 Docker的启动方式
创建配置configmap
- 在配置项中添加配置
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F3c146322-3599-441a-801a-87cbccf95b31%2FUntitled.png?table=block&id=0ea36590-a57b-4b30-aaba-0b07f30af144&t=0ea36590-a57b-4b30-aaba-0b07f30af144&width=384&cache=v2)
- 添加一个key-value配置,key就是配置文件的文件名 这里是
my.cnf
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F1cac6b69-a5de-4321-a9b3-cdac66f48858%2FUntitled.png?table=block&id=2c8d98c3-ec2d-4c2b-8379-eb52c3cce69d&t=2c8d98c3-ec2d-4c2b-8379-eb52c3cce69d&width=384&cache=v2)
my.cnf的内容,填写的是数据库服务端的配置
创建 挂载卷
- 创建卷
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F32ccbdd5-f9b1-4a29-8330-d22e4f291140%2FUntitled.png?table=block&id=9d78e7ba-5db8-451e-98ca-8ebcda78e9a3&t=9d78e7ba-5db8-451e-98ca-8ebcda78e9a3&width=336&cache=v2)
- 设置存储类型和访问模式以及卷容量
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F49dc5371-1b31-4e6f-8cfc-b2718182cb1b%2FUntitled.png?table=block&id=4bd73d5b-c998-4619-8273-36e735522ce5&t=4bd73d5b-c998-4619-8273-36e735522ce5&width=384&cache=v2)
添加工作负载
- 在部署工作负载中选择有状态负载
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fde3158c0-faf8-4d63-913a-579fd9f13e18%2FUntitled.png?table=block&id=e34f4073-ef26-4f6a-aa25-d0a38d926987&t=e34f4073-ef26-4f6a-aa25-d0a38d926987&width=384&cache=v2)
- 镜像 配置为dockerhub中的mysql,并使用镜像默认端口设置
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F4106f53f-e00d-40be-bb87-7b2b6ac067d0%2FUntitled.png?table=block&id=c8802b64-7668-470b-8c8c-1df152bade16&t=c8802b64-7668-470b-8c8c-1df152bade16&width=384&cache=v2)
- 镜像环境变量,根据mysql镜像的官方要求,需要添加一个
MYSQL_ROOT_PASSWORD
;创建后点击下一步
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fc29e4b8b-12f6-428b-92cc-8ccb8e388dc5%2FUntitled.png?table=block&id=c037f2db-3302-4dd2-9670-77f29f699bcf&t=c037f2db-3302-4dd2-9670-77f29f699bcf&width=384&cache=v2)
- 挂载存储卷
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F8d499241-741a-4c8f-a22e-8a1256cd8b87%2FUntitled.png?table=block&id=4c7fc905-664b-4956-9004-8719a07d8ef6&t=4c7fc905-664b-4956-9004-8719a07d8ef6&width=384&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F0929ee82-b0e6-4717-a002-ef03f368e5af%2FUntitled.png?table=block&id=077495d6-10f4-4137-959f-60c90046cc0a&t=077495d6-10f4-4137-959f-60c90046cc0a&width=384&cache=v2)
- 挂载配置卷
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F09225016-e24d-4ae1-8c9d-d6e26d51d92d%2FUntitled.png?table=block&id=16d79375-bb91-4d2c-8eb3-212e16b692fd&t=16d79375-bb91-4d2c-8eb3-212e16b692fd&width=384&cache=v2)
指定配置卷的挂载目录
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F3c951129-5845-444c-af8a-92e336c227ff%2FUntitled.png?table=block&id=7391d7da-543e-4520-b133-31979339bd31&t=7391d7da-543e-4520-b133-31979339bd31&width=384&cache=v2)
对外 提供服务
- KS默认会创建一个内网服务,可以选择修改,或重新创建一个NodePort对外服务
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F41efb218-7003-49c6-81ef-ad83c15630d4%2FUntitled.png?table=block&id=c3c0dfa6-cd42-4ac7-804d-fca739066cf7&t=c3c0dfa6-cd42-4ac7-804d-fca739066cf7&width=384&cache=v2)
- NodePort类型表示可以从集群中的任意节点IP进行访问
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F8aa475fc-2c47-48e9-b4b4-2793b6d486df%2FUntitled.png?table=block&id=c10b2133-8d3e-4aab-a962-0edf2c725832&t=c10b2133-8d3e-4aab-a962-0edf2c725832&width=384&cache=v2)
- 访问地址,如图冒号后的地址就是外部访问的端口号
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fe5d6794e-a609-4358-91be-0cb01beba065%2FUntitled.png?table=block&id=1441bb72-ed60-4492-b4f1-588824d37d68&t=1441bb72-ed60-4492-b4f1-588824d37d68&width=384&cache=v2)
- 访问客户端测试
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F92eb58a8-086d-4fc5-b030-c7bfd2a6eb1c%2FUntitled.png?table=block&id=5e370e23-aeae-4423-a991-e69ddae3fc96&t=5e370e23-aeae-4423-a991-e69ddae3fc96&width=384&cache=v2)
2. Nacos
3. Redis
4. SEATA
部署前准备
官方文档的Docker部署方式
seata 连接到Nacos,由nacos托管 seata的配置;同时客户端是通过nacos的注册中心中获取seata的地址。
创建配置文件CM
我这里需要将SEATA注册到nacos中,使用nacos进行配置,用mysql进行undo_log的存储,需要修改
application.yml
文件,配置参考如下:创建有状态副本
- 查找镜像
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Ff5d9114c-1592-498f-bbfc-842a4cffb8df%2FUntitled.png?table=block&id=e01a6785-ef3a-4e1f-a720-48011647c03e&t=e01a6785-ef3a-4e1f-a720-48011647c03e&width=336&cache=v2)
- 开放端口
7091是面板的地址 ,8091是 服务端口地址
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F874aaff1-4bfb-4466-9cfc-98b448da031b%2FUntitled.png?table=block&id=83bf4a81-3d52-4e3f-bd44-607cc0d15671&t=83bf4a81-3d52-4e3f-bd44-607cc0d15671&width=336&cache=v2)
- 挂载配置文件,注意,我们这里只修改
application.yml
配置文件,为了不覆盖容器中/seata-server/resources
目录下的其它配置文件,这里要使用子路径配置。
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb86a6b86-5c09-4f00-8980-66d5f51cfc7b%2FUntitled.png?table=block&id=250cbea8-4c29-45e6-9c9b-c43bd5eb952d&t=250cbea8-4c29-45e6-9c9b-c43bd5eb952d&width=336&cache=v2)
暴露服务
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F16bc64a1-b77d-4cd4-8668-a87b68ba4357%2FUntitled.png?table=block&id=099c6d8d-913b-405b-b049-29f8a45c2221&t=099c6d8d-913b-405b-b049-29f8a45c2221&width=336&cache=v2)
此方案部署后,seata-server虽然部署成功,但是注册到Nacos上的IP是Kubernetes内部集群IP,seata客户端应用是无法访问到seata-server服务的,除非nacos和seata客户端应用也部署在Kubernetes集群内部,但显然目前无法进行这么大的变革。
所以解决问题的方法就是让seata-server的注册IP是宿主机IP.
定义SEATA的外部IP
定义SEATA的外部端口
k8s如果用NodePort方式暴露端口的话端口必须是30000-32765之间的端口号,但是在容器中默认配置的端口是8091,这就会导致连接不上,要从外部访问的折中之计就是先利用hostPort将容器的端口直接在宿主机上暴露。
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F97267182-951b-416c-a801-c5af9b1a5bad%2FUntitled.png?table=block&id=f7fb832c-dbe9-428a-b2c5-e4035ed08c80&t=f7fb832c-dbe9-428a-b2c5-e4035ed08c80&width=1162&cache=v2)
这样做会占用了宿主机唯一的8091端口,要注意实例数不能大于节点机数,否则多出来的会无法部署。
通过Mysql存储实现Seata的高可用
开始前准备
创建seata的数据库,并且执行官方的初始化脚本 mysql.sql , 数据库建完效果如下,我目前用的是1.5.2版本的脚本:
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb1c850a2-bc0c-424c-bb04-7c2665dc9732%2FUntitled.png?table=block&id=10793fc1-57f4-4d91-be0c-88077e172db9&t=10793fc1-57f4-4d91-be0c-88077e172db9&width=1552&cache=v2)
在NACOS中配置Seata的Store类型为DB
创建
/config.txt
,并填写数据库配置如下:创建一个nacos配置导入脚本 /conf/nacos-config.sh
内容点击展开:
执行脚本,后面跟上nacos的ip和端口,例如:
./nacos-config.sh -h192.168.31.191 -p31864
![成功执行的结果](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F3bcbf71b-1f6d-4e2c-959f-55258b0e0cfb%2FUntitled.png?table=block&id=ae616cbd-977e-4b28-aac7-c30f77879c6a&t=ae616cbd-977e-4b28-aac7-c30f77879c6a&width=2766&cache=v2)
配置添加成功的效果如下:当然这些配置你也可以在NACOS中逐一添加
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fcc7e2e0f-861e-402c-bafa-b7fe8f3f89e8%2FUntitled.png?table=block&id=b1473aea-2039-44cb-b7d6-a5e696f729e8&t=b1473aea-2039-44cb-b7d6-a5e696f729e8&width=2922&cache=v2)
- 导入配置后,重新启动Seata,若日志结尾出现
{dataSource-1} inited
表示Seata 成功连接mysql
相关
- 作者:Tangly
- 链接:https://blog.tangly1024.com/article/kubesphere-k8s-middleware-mysql-redis-nacos-sentinel-seata
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议,转载请注明出处。