TOC
前言
longhorn 是 rancher 公司开源并贡献给 CNCF 的一个开源分布式存储项目,可用来作为 kubernetes 的 CSI 存储.
本文介绍 helm 和 yaml 两种方式部署 longhorn 并部署一个有状态服务 Gogs 应用.
官方文档: https://longhorn.io/docs/1.1.0/deploy/install/install-with-kubectl/
安装要求
- open-iscsi
- kubernetes 1.16+ 官方推荐1.17+ (本文使用1.18.8)
- curl、findmnt、grep、awk、blkid、lsblk. 在编写本文时没有显示的去安装这些工具,也许是机器已经具备相关环境.
- 启用了
mount propagation. 本文部署没有显示启用这个功能,也没有遇到相关问题.
helm部署longhorn
添加longhorn的helm repo
helm repo add longhorn https://charts.longhorn.io
使用helm部署longhorn:
helm install longhorn longhorn/longhorn --namespace longhorn-system
查看部署情况
部署中:
$ watch kubectl get po -n longhorn-system
NAME READY STATUS RESTARTS AGE
longhorn-driver-deployer-76c8fd69fb-fvtg7 0/1 Init:0/1 0 27s
longhorn-manager-l2mrs 0/1 Running 0 27s
longhorn-ui-78547884d8-zb2sc 1/1 Running 0 27s
最终部署完成的效果:
$ kubectl get po -n longhorn-system
NAME READY STATUS RESTARTS AGE
csi-attacher-74db7cf6d9-djzd2 1/1 Running 0 81s
csi-attacher-74db7cf6d9-lcxbb 1/1 Running 0 81s
csi-attacher-74db7cf6d9-rdxls 1/1 Running 0 81s
csi-provisioner-d444fb7c9-6545x 1/1 Running 0 78s
csi-provisioner-d444fb7c9-96n9g 1/1 Running 0 78s
csi-provisioner-d444fb7c9-k7ks7 1/1 Running 0 78s
csi-resizer-58458969c9-64fnz 1/1 Running 0 73s
csi-resizer-58458969c9-cdqj2 1/1 Running 0 72s
csi-resizer-58458969c9-fhvg4 1/1 Running 0 72s
csi-snapshotter-6d856448d-4n2nk 1/1 Running 0 69s
csi-snapshotter-6d856448d-p4mbz 1/1 Running 0 69s
csi-snapshotter-6d856448d-pm7kx 1/1 Running 0 69s
engine-image-ei-cf743f9c-xjl5k 1/1 Running 0 115s
instance-manager-e-296a3f7e 1/1 Running 0 115s
instance-manager-r-40e113e5 1/1 Running 0 115s
longhorn-csi-plugin-pnj85 2/2 Running 0 67s
longhorn-driver-deployer-76c8fd69fb-fvtg7 1/1 Running 0 2m28s
longhorn-manager-l2mrs 1/1 Running 0 2m28s
longhorn-ui-78547884d8-zb2sc 1/1 Running 0 2m28s
查看storageClass:
$ kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
longhorn (default) driver.longhorn.io Delete Immediate true 2m23s
另外也可以使用helm list -n longhorn-system来查看helm中的资源,这里不再演示.
接下来部署一个StatefulSet资源来试试通过longhron storageClass自动化创建PV和PVC的效果.演示应用使用Gogs.
yaml方式部署longhorn
yaml的方式部署的话就一行代码的事
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.1.0/deploy/longhorn.yaml
如果github网络问题无法下载可以试试这个地址:https://res.cloudinary.com/lyp/raw/upload/v1617962359/hugo/blog.github.io/cncf/longhorn1.1.yaml
效果和helm部署是一样的,除了helm list看不到资源外(helm list只能看到通过helm方式部署的资源)
部署Gogs
创建yaml文件gogs.yaml
文件内容如何:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: gogs
labels:
app: gogs
spec:
serviceName: gogs
volumeClaimTemplates:
- metadata:
name: gogs-data
spec:
storageClassName: longhorn
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
updateStrategy:
type: RollingUpdate
replicas: 1
selector:
matchLabels:
app: gogs
template:
metadata:
labels:
app: gogs
spec:
containers:
- name: gogs
image: gogs/gogs:0.12
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3000
name: gogs
- containerPort: 22
name: ssh
volumeMounts:
- name: gogs-data
mountPath: /data
---
apiVersion: v1
kind: Service
metadata:
name: gogs
spec:
type: NodePort
ports:
- name: gogs
protocol: TCP
port: 3000
targetPort: 3000
nodePort: 30026
- name: ssh
protocol: TCP
port: 22
targetPort: 22
nodePort: 30022
selector:
app: gogs
开始部署:
kubectl apply -f gogs.yaml
一切正常的话可以看到 gogs 的 pod 已经 running 状态了:
$ k get po
NAME READY STATUS RESTARTS AGE
gogs-0 1/1 Running 0 3h49m
我这里是已经部署好了,因此 AGE 显示时间比较长. 再来看看 PV 和 PVC
$ k get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
gogs-data-gogs-0 Bound pvc-3078b16e-4d20-48f2-a32e-6fc6b14bb266 5Gi RWO longhorn 3h50m
$ k get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-3078b16e-4d20-48f2-a32e-6fc6b14bb266 5Gi RWO Delete Bound default/gogs-data-gogs-0 longhorn 3h50m
可以看到 PV 和 PVC 都已经处于正常的使用状态了.说明 longhorn storageClass 是 working 的.
FAQ
需要提前安装open-iscsi
如果没有提前安装好 open-iscsi 可能会报下面的错误:
[root@devmaster helmrepo]# kubectl logs -f -n longhorn-system longhorn-manager-8smtj
time="2020-06-14T07:35:39Z" level=error msg="Failed environment check, please make sure you have iscsiadm/open-iscsi installed on the host"
time="2020-06-14T07:35:39Z" level=fatal msg="Error starting manager: Environment check failed: Failed to execute: nsenter [--mount=/host/proc/932/ns/mnt --net=/host/proc/932/ns/net iscsiadm --version], output , stderr, nsenter: failed to execute iscsiadm: No such file or directory\n, error exit status 1"
...
下面给出几个系统安装 iscsi 工具的命令:
Debian/Ubuntu:
sudo apt-get install open-iscsi
Redhat/Centos/Suse:
yum install iscsi-initiator-utils
微信公众号
扫描下面的二维码关注我们的微信公众号,第一时间查看最新内容。同时也可以关注我的Github,看看我都在了解什么技术,在页面底部可以找到我的Github。
扫码下面二维码加我微信 :)
