部署指南
- 1: 部署环境要求
- 2: All In One 部署与卸载
- 2.1: All In One 最小化部署
- 2.2: KubeCube 卸载
- 3: 在已有K8s上安装KubeCube
- 3.1: 通过helm安装KubeCube
- 3.2: 通过helm纳管计算集群
- 4: 使用 hotplug 进行拓展
- 4.1: 热插拔 hotplug 介绍
- 4.2: 通过 hotplug 启用 loggie
- 5: 使用脚本安装 K8s 和 KubeCube(已废弃)
- 5.1: 在已有k8s集群中部署KubeCube
- 5.2: 添加计算集群
- 5.3: 添加节点
- 5.4: 多节点高可用部署
- 6: 已有系统接入
- 6.1: Prometheus
- 6.2: ElasticSearch
- 6.3: 第三方认证系统
1 - 部署环境要求
在进行 All In One 或者多节点部署之前,请按照以下内容确认环境要求
系统版本及硬件要求
操作系统 | 最低要求 |
---|---|
Ubuntu 16.04, 18.04 | CPU:4 核,内存:8 G,磁盘空间:20 G |
Debian Buster, Stretch | CPU:4 核,内存:8 G,磁盘空间:20 G |
CentOS 7.x | CPU:4 核,内存:8 G,磁盘空间:20 G |
Kylin v10 | CPU:4 核,内存:8 G,磁盘空间:20 G |
OS ARCH | 硬件要求 |
---|---|
AMD 64 | Intel 系列,AMD 系列 |
ARM 64 | Phytium 2000(国产化arm64芯片理论上都支持,但未经过充分测试) |
以上系统配置要求适用于 KubeCube 默认最小化 All In One 模式安装,如需启动更多可插拔组件和拓展功能,建议机器配置为 8 核 CPU 和 16 G 内存
依赖说明
KubeCube | Kubernetes | Docker | Containerd |
---|---|---|---|
v1.0 ~ v1.4 | v1.18 ~ v1.23 | 19.3.12+ | 1.5.x |
容器运行时
KubeCube 支持的 cri 跟随 Kubernetes 标准
KubeCube 部署脚本支持的 cri 如下:
支持的容器运行时 | 版本 |
---|---|
Docker | 19.3.12+ |
Containerd | 1.5.x |
节点上若无容器运行时,部署脚本将自动安装 containerd 1.5.5 作为容器运行时
网络插件
KubeCube 支持的 cni 跟随 Kubernetes 标准
KubeCube 部署脚本的 cni 目前支持 calico
Kubernetes 版本
KubeCube 支持的 k8s 版本为 v1.18 ~ v1.26 KubeCube 部署脚本支持的 k8s 版本为 v1.18.20、v1.19.13、v1.20.9、v1.21.2、v1.22.2、v1.23.5、v1.24.7、v1.25.3、v1.26.0
关于 K8s 集群安装
KubeCube 的 All In One 安装脚本提供单节点的 K8s 集群安装。如果需要自定义安装 K8s 集群,可以使用 kubeadm、kubez-ansible 等开源工具,之后 kubez-ansible 中也会集成 KubeCube。
前置准备
在使用 All In One 部署脚本开始 KubeCube 的安装时,脚本会检测环境,并提示需要安装缺失的依赖
监控组件说明
KubeCube 会默认安装 Prometheus 等监控组件,如果选择在已有k8s集群中部署 KubeCube,并且集群中已安装 Mertics Server,安装 KubeCube 后 Prometheus 会和 Mertics Server 产生冲突,导致监控功能不可用。需要在安装 KubeCube 后执行以下步骤:
点击页面右上角【切换到控制台】,点击任意空间,进入到控制台页面;
在左侧菜单栏点击【自定义资源CRD】,进入到集群级别 CRD 列表,可以点击右上方输入 “hotplug” 进行搜索,找到 “hotplugs.hotplug.kubecube.io” CRD,点击【v1】版本进入 CRD 详情页;
选择 common 实例,点击【设置YAML】,找到 spec.component. name=kubecube-monitoring,添加环境变量 prometheusAdapter.enabled=false,如:
- env: |
grafana:
enabled: false
prometheus:
prometheusSpec:
externalLabels:
cluster: "{{.cluster}}"
remoteWrite:
- url: http://10.173.32.129:31291/api/v1/receive
prometheusAdapter:
enabled: false
name: kubecube-monitoring
namespace: kubecube-monitoring
pkgName: kubecube-monitoring-15.4.10.tgz
status: enabled
2 - All In One 部署与卸载
2.1 - All In One 最小化部署
对于想要快速开始、快速体验的用户来说,All In One 是最佳的安装方式。All In One 部署脚本将会在执行该脚本的节点上,安装一个单节点的 K8s 集群,并在该 K8s 集群上部署 KubeCube。
v1.8.x
在 Linux 上部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
export KUBECUBE_VERSION=v1.8
curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/release/v1.3/entry.sh | bash
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.4.x
在 Linux 上部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.4
curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.2.x
在 Linux 上部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.2
curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.1.x
在 Linux 上部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.1
curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.0.x
在 Linux 上部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.0
curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
2.2 - KubeCube 卸载
KubeCube 提供一键卸载脚本
⚠️ 注意,卸载脚本仅用于使用 All In One 部署脚本安装的环境,不应该用于任何正式环境
下载卸载脚本
curl -o cleanup.sh https://kubecube.nos-eastchina1.126.net/hack/cleanup.sh
一键卸载
一键卸载按顺序卸载 kubecube, kubernetes 和 docker
/bin/bash cleanup.sh all
指定卸载
卸载脚本可以指定卸载对象,但需注意卸载依赖
卸载 KubeCube
删除 kubecube chart release,并清理 kubecube 在本机上的相关文件
/bin/bash cleanup.sh kubecube
卸载 Kubernetes
卸载 kubernetes 集群,并清理 kubernetes 在本机上的相关文件
/bin/bash cleanup.sh k8s
卸载 Docker
卸载 docker,并清理 docker 在本机上的相关文件
⚠️注意:需在卸载 kubernetes 后才能卸载 docker,不然会造成 kubernetes 集群异常
/bin/bash cleanup.sh docker
3 - 在已有K8s上安装KubeCube
3.1 - 通过helm安装KubeCube
通过 helm 的方式在管控集群中部署 KubeCube
v1.8.x
要求
- 已有 K8s 集群环境
- helm v3 客户端
下载 KubeCube helm chart 包
KUBECUBE_VERSION=v1.8
curl -s https://kubecube.nos-eastchina1.126.net/kubecube-chart/${KUBECUBE_VERSION}/kubecube-chart.tar.gz | tar -xz
通过 helm 在管控集群上安装 KubeCube
创建 pivot-value.yaml 文件并填写必要的 value 值
# pivot-value.yaml
global:
# 管控集群的 Node IP,用来暴露 KubeCube 的 NodePort service
nodeIP: x.x.x.x
dependencesEnable:
ingressController: "false" # 如果集群中没有部署 ingress controller,请将此设置为 "true"
localPathStorage: "false" # 如果集群中没有部署 local path storage,请将此设置为 "true"
metricServer: "false" # # 如果集群中没有部署 metric server,请将此设置为 "true"
# 如果要启动日志功能,请假以下值设置为 "enabled"
hotPlugEnable:
pivot:
logseer: "disabled"
logagent: "disabled"
elasticsearch: "disabled"
localKubeConfig: xx # 管控集群的 kubeconfig 的 base64
pivotKubeConfig: xx # 管控集群的 kubeconfig 的 base64
warden:
containers:
warden:
args:
cluster: "pivot-cluster" # 管控集群名
部署 KubeCube
helm install kubecube -n kubecube-system --create-namespace ./kubecube-chart -f ./pivot-value.yaml
卸载管控集群中的 KubeCube
注意:not found
错误可以被忽略
- 手动清理 webhook
kubectl delete validatingwebhookconfigurations kubecube-validating-webhook-configuration warden-validating-webhook-configuration kubecube-monitoring-admission
- 卸载 KubeCube chart
helm uninstall kubecube -n kubecube-system
- 清理残留资源
kubectl delete sa/kubecube-pre-job -n kubecube-system
kubectl delete clusterRole/kubecube-pre-job
kubectl delete clusterRoleBinding/kubecube-pre-job
kubectl delete ns kubecube-system hnc-system kubecube-monitoring
3.2 - 通过helm纳管计算集群
使用 helm 的方式在计算集群中安装 Warden(KubeCube 的计算集群 agent),安装完成后,Warden 将主动向管控集群的 KubeCube 注册计算集群信息,完成计算集群的纳管。
v1.8.x
要求
- 已有 K8s 集群环境
- helm v3 客户端
下载 KubeCube helm chart 包
KUBECUBE_VERSION=v1.8
curl -s https://kubecube.nos-eastchina1.126.net/kubecube-chart/${KUBECUBE_VERSION}/kubecube-chart.tar.gz | tar -xz
通过 helm 在计算集群上安装 Warden
创建 member-value.yaml 文件并填写必要的 value 值
# member-value.yaml
global:
# 管控集群的 Node IP,用来暴露 KubeCube 的 NodePort service
nodeIP: x.x.x.x
dependencesEnable:
ingressController: "false" # 如果集群中没有部署 ingress controller,请将此设置为 true
localPathStorage: "false" # 如果集群中没有部署 local path storage,请将此设置为 true
metricServer: "false" # # 如果集群中没有部署 metric server,请将此设置为 true
# 以下为计算集群指定安装组件,请不要修改
componentsEnable:
kubecube: "false"
warden: "true"
audit: "false"
webconsole: "false"
cloudshell: "false"
frontend: "false"
# 如果要启动日志功能,请假以下值设置为 "enabled"
hotPlugEnable:
common:
logagent: "disabled"
localKubeConfig: xx # 当前集群的 kubeconfig 的 base64
pivotKubeConfig: xx # 管控集群的 kubeconfig 的 base64
warden:
containers:
warden:
args:
inMemberCluster: true
cluster: "member-cluster" # 集群集群名
部署 Warden
helm install warden -n kubecube-system --create-namespace ./kubecube-chart -f ./pivot-value.yaml
卸载计算集群中的 warden
注意:not found
错误可以被忽略
- 在管控面删除 cluster cr
kubectl delete cluster {计算集群名}
- 在计算集群手动清理 webhook
kubectl delete validatingwebhookconfigurations kubecube-validating-webhook-configuration warden-validating-webhook-configuration kubecube-monitoring-admission
- 在计算集群卸载 warden chart
helm uninstall warden -n kubecube-system
- 再计算集群清理残留资源
kubectl delete sa/kubecube-pre-job -n kubecube-system
kubectl delete clusterRole/kubecube-pre-job
kubectl delete clusterRoleBinding/kubecube-pre-job
kubectl delete ns kubecube-system hnc-system kubecube-monitoring
4 - 使用 hotplug 进行拓展
4.1 - 热插拔 hotplug 介绍
热插拔是通过修改配置文件实现不停机更新监控、日志、审计等组件的启停和配置。
热插拔
热插拔的实现是基于 helm ,因此集群需要预先安装好 helm3 版本。
登录管控 k8s 集群,执行命令可以查看热插拔配置
# kubectl get hotplug
NAME PHASE AGE
common running 23h
pivot-cluster running 16d
其中:
common
表示公共的热插拔配置,pivot-cluster
表示 pivot-cluster 这个 k8s 集群的热插拔配置。允许自定义各个集群的热插拔配置覆盖 common 热插拔配置实现个性化配置 k8s 集群组件热插拔。
热插拔配置的
.metadata.name
要求与k8s集群名称一致,所有 k8s 集群信息查看命令为kubectl get cluster
。
Common配置说明
目前,默认的 common 配置包括以下几个组件:
logseer
:日志管理组件,仅在管控集群安装
logagent
:日志采集代理组件
kubecube-monitoring
:监控 prometheuse 组件
kubecube-thanos
:监控 thanos 组件,仅在管控集群安装
示例如下:
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
annotations:
kubecube.io/sync: "true" ## 同步信号,kubecube会将这个配置同步到各个集群
name: common ## 公共配置为cmmon,其余集群特殊配置为集群名字
spec:
component:
- name: audit ## 审计日志
status: disabled
- name: logseer ## 日志管理组件
namespace: logseer
pkgName: logseer-v1.0.0.tgz
status: disabled ## 启停标识:这里disabled为禁用
- name: logagent ## 日志采集代理组件
namespace: logagent
pkgName: logagent-v1.0.0.tgz
status: enabled ## 启停标识:这里enabled为启用
env: | ## 环境变量
clustername: "{{.cluster}}" ## {{.cluster}} 程序会自动注入集群名字替换
- name: kubecube-monitoring ## 监控组件
namespace: kubecube-monitoring
pkgName: kubecube-monitoring-15.4.7.tgz
status: enabled
env: |
grafana:
enabled: false
prometheus:
prometheusSpec:
externalLabels:
cluster: "{{.cluster}}"
remoteWrite:
- url: http://10.173.32.42:31291/api/v1/receive
- name: kubecube-thanos ## 监控thanos组件
namespace: kubecube-monitoring
pkgName: thanos-3.18.0.tgz
status: disabled
status: ## message显示各个组件运行状态,phase显示总体运行状态
message: '{"kubecube-monitoring":"release is running","kubecube-thanos":"release
is running","logagent":"release is running","logseer":"release is running"}'
phase: running
每一个组件基本包含5个要素,在 spec.component
下:
name:组件名称
namespace:指定组件部署的命名空间,若指定的命名空间不存在,会自动以该字段值去创建一个命名空间。
pkgName:安装包名称,安装包默认存放路径为 warden 容器里的 /root/helmchartpkg,以 emptydir 形式存在。
status:组件是否启用
env:环境变量配置
每一个要素都可以使用集群独特配置进行覆盖,未覆盖的要素则依然使用 common 里的配置。
管控集群配置说明
Pivot-cluster 配置是管控集群配置,即 KubeCube 所在的集群。集群独特的配置会与 common 的配置结合,用于个性化配置集群组件,结合时遇到相同字段pivot-cluster 优先。
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
annotations:
kubecube.io/sync: "true" ## 同步信号,kubecube会将这个配置同步到其他集群
name: pivot-cluster ## 与集群名字一致,指明这是pivot-cluster这个集群的热插拔配置
spec:
component:
- name: logseer ## 日志管理组件
status: enabled ## 结合common设置为disabled,这里设置为enabled,标识其余集群不启用,pivot-cluster集群启用
- name: kubecube-monitoring
env: |
grafana:
enabled: true
prometheus:
prometheusSpec:
externalLabels:
cluster: "{{.cluster}}"
remoteWrite:
- url: http://thanos-receive:19291/api/v1/receive
- name: kubecube-thanos
status: enabled
env: |
receive:
replicaCount: 1
replicationFactor: 1
status:
message: '{"kubecube-monitoring":"release is running","kubecube-thanos":"release
is running","logagent":"release is running","logseer":"release is running"}'
phase: running
4.2 - 通过 hotplug 启用 loggie
KubeCube 使用 hotplug 热插拔方式集成 Loggie,用户可以通过打开 hotplug 中关于日志部分的开关来开启 Loggie。我们有两个时间点可以用来开启 Loggie。
在使用 Helm 安装 KubeCube 时
在使用 Helm 安装 KubeCube 时,我们可以设置以下 values 来开启 Loggie。
安装管控集群时的参数如下:
# pivot-value.yaml
...
global:
# set "enabled" if wanna open log application.
hotPlugEnable:
pivot:
logseer: "enabled"
logagent: "enabled"
elasticsearch: "enabled"
...
安装计算集群时的参数如下:
# member-value.yaml
...
global:
# set "enabled" if wanna open log application.
hotPlugEnable:
common:
logagent: "enabled"
...
在使用过程中
在使用过程中,我们也可以通过直接修改 hotplug 的方式来开启 Loggie。
修改管控集群的 hotplug 如下:
kubectl edit hotplug pivot-cluster
# pivot-cluster
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
name: pivot-cluster
spec:
component:
- name: elasticsearch
namespace: elasticsearch
pkgName: elasticsearch-7.8.1.tgz
status: enabled # 将该值设为 enabled 来开启日志
...
- name: logseer
status: enabled # 将该值设为 enabled 来开启日志
- name: logagent
status: enabled # 将该值设为 enabled 来开启日志
修改计算集群的 hotplug 如下:
kubectl edit hotplug common
# common
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
name: common
spec:
component:
...
- env: |
clustername: "{{.cluster}}"
elasticsearch:
address: x.x.x.x:32200 # 填写管控集群的 es 的 nodeport svc 访问地址,一般为 {nodeIP}:32200
name: logagent
namespace: logagent
pkgName: logagent-1.3.0.tgz
status: enabled # 将该值设为 enabled 来开启日志
5 - 使用脚本安装 K8s 和 KubeCube(已废弃)
5.1 - 在已有k8s集群中部署KubeCube
v1.4.x
在 Kubernetes 集群中部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.2.x
在 Kubernetes 集群中部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.1.x
在 Kubernetes 集群中部署 KubeCube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_MEMBER=“false”
MASTER_IP="${node ip}"
${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
v1.0.x
在 Kubernetes 集群中部署 KubeCube
⚠️修改 Kubernetes API-Server 配置
必要性
KubeCube 对多集群提供统一的认证和鉴权服务,需要使用 k8s api-server 的 auth-webhook 能力来做拓展。
KubeCube 提供对 k8s-apiserver 日志进行审计的能力,这需要为 k8s api-server 指定审计服务后端。
修改操作
如果您的 k8s api-server 服务是以 deployment 形式运行的,请直接修改 deployment ;如果您的 k8s api-server 服务是以 static pod 形式运行的,您需要修改对应的 manifest 文件,它的文件路径通常为 /etc/kubernetes/manifests/kube-apiserver.yaml
,修改内容如下:
apiVersion: v1
kind: Pod
metadata:
name: kube-apiserver
namespace: kube-system
spec:
containers:
- command:
- kube-apiserver
- --audit-log-format=json
- --audit-log-maxage=10
- --audit-log-maxbackup=10
- --audit-log-maxsize=100
- --audit-log-path=/var/log/audit
- --audit-policy-file=/etc/cube/audit/audit-policy.yaml
- --audit-webhook-config-file=/etc/cube/audit/audit-webhook.config
- --authentication-token-webhook-config-file=/etc/cube/warden/webhook.config
name: kube-apiserver
volumeMounts:
- mountPath: /var/log/audit
name: audit-log
- mountPath: /etc/cube
name: cube
readOnly: true
volumes:
- hostPath:
path: /var/log/audit
type: DirectoryOrCreate
name: audit-log
- hostPath:
path: /etc/cube
type: DirectoryOrCreate
name: cube
开始安装
在 Linux 机器上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_MEMBER=“false”
MASTER_IP="${node ip}"
${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="false"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="x.x.x.x"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
等待部署完成
KubeCube 部署完成后,请根据提示信息登陆 console 管理页面
使用 admin 账户登陆 console
⚠️请在登陆后修改 admin 用户的密码
5.2 - 添加计算集群
KubeCube 可以添加其它集群作为计算集群,前提是,计算集群能够访问管控集群的 k8s api-server 和 KubeCube,默认情况下 KubeCube 使用 NodePort 对外暴露服务,用户可自行使用 ingress 进行暴露
v1.4.x
方式一:部署新集群并添加
在 linux 机器上,需要构建 Kubernetes 集群并安装 KubeCube 依赖项
开始安装
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_PIVOT=“false”
INSTALL_KUBECUBE_MEMBER=“true”
INSTALL_KUBERNETES=“true”
MEMBER_CLUSTER_NAME=“member-1”
MASTER_IP="${node ip}"
KUBECUBE_HOST="${pivot node ip}"
MEMBER_CLUSTER_NAME 表示计算集群的名字,注意,不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip,用于向 KubeCube 注册集群
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"
方式二:纳管已有集群
添加已有集群只需从 console 页面导入集群信息即可
在 console 页面中导入集群信息
在 console 中确认新集群
v1.2.x
方式一:部署新集群并添加
在 linux 机器上,需要构建 Kubernetes 集群并安装 KubeCube 依赖项
开始安装
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_PIVOT=“false”
INSTALL_KUBECUBE_MEMBER=“true”
INSTALL_KUBERNETES=“true”
MEMBER_CLUSTER_NAME=“member-1”
MASTER_IP="${node ip}"
KUBECUBE_HOST="${pivot node ip}"
MEMBER_CLUSTER_NAME 表示计算集群的名字,注意,不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip,用于向 KubeCube 注册集群
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
方式二:纳管已有集群
添加已有集群只需从 console 页面导入集群信息即可
在 console 页面中导入集群信息
在 console 中确认新集群
v1.1.x
方式一:部署新集群并添加
在 linux 机器上,需要构建 Kubernetes 集群并安装 KubeCube 依赖项
开始安装
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_PIVOT=“false”
INSTALL_KUBECUBE_MEMBER=“true”
INSTALL_KUBERNETES=“true”
MEMBER_CLUSTER_NAME=“member-1”
MASTER_IP="${node ip}"
KUBECUBE_HOST="${pivot node ip}"
MEMBER_CLUSTER_NAME 表示计算集群的名字,注意,不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip,用于向 KubeCube 注册集群
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
方式二:纳管已有集群
添加已有集群只需从 console 页面导入集群信息即可
在 console 页面中导入集群信息
在 console 中确认新集群
v1.0.x
方式一:部署新集群并添加
在 linux 机器上,需要构建 Kubernetes 集群并安装 KubeCube 依赖项
开始安装
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置安装脚本参数
该安装模式下,需要修改以下参数:
INSTALL_KUBECUBE_PIVOT=“false”
INSTALL_KUBECUBE_MEMBER=“true”
INSTALL_KUBERNETES=“true”
MEMBER_CLUSTER_NAME=“member-1”
MASTER_IP="${node ip}"
KUBECUBE_HOST="${pivot node ip}"
MEMBER_CLUSTER_NAME 表示计算集群的名字,注意,不能与已有的计算集群名称同名 ${node ip} 表示你运行脚本所在 node 机器的 ip,该 node 需要可操作 kubectl ${pivot node ip} 表示管控集群 node 机器的 ip,用于向 KubeCube 注册集群
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="x.x.x.x"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
方式二:纳管已有集群
添加已有集群需要从 console 页面获取添加集群的定制脚本
在 console 页面中获取添加集群的脚本
使用脚本添加集群
在集群的 node 机器上,使用从 console 中下载的脚本,该机器需要能够执行 kubectl
/bin/bash add_cluster.sh
等待集群添加完成
在 console 中确认新集群
5.3 - 添加节点
KubeCube 提供为已有集群添加节点的能力,同时也支持使用 kubeadm 的原生方式添加节点
⚠️ 注意通过 KubeCube 的脚本添加节点时,node 机器需要能够通过 ssh 访问 master 机器,支持公钥和密码两种 ssh 方式,执行脚本前可以在 node 上 ssh 到 master 测试连通性
v1.4.x
向集群添加工作节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入集群
- MASTER_IP 为 master 节点 ip
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"
向集群的 control-plane 添加 master 节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入 control-plane
- MASTER_IP 需要填已有的 master 节点 ip
- NODE_MODE 当该模式为 node-join-control-plane 时,需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
- CONTROL_PLANE_ENDPOINT 为高可用 vip
- 可以根据需要选择适合自己的 ssh 方式
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
v1.2.x
向集群添加工作节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入集群
- MASTER_IP 为 master 节点 ip
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
向集群的 control-plane 添加 master 节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入 control-plane
- MASTER_IP 需要填已有的 master 节点 ip
- NODE_MODE 当该模式为 node-join-control-plane 时,需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
- CONTROL_PLANE_ENDPOINT 为高可用 vip
- 可以根据需要选择适合自己的 ssh 方式
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
v1.1.x
向集群添加工作节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
tip: 你可以通过预先下载离线包和镜像来减少部署时间
export PRE_DOWNLOAD="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入集群
- MASTER_IP 为 master 节点 ip
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
向集群的 control-plane 添加 master 节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入 control-plane
- MASTER_IP 需要填已有的 master 节点 ip
- NODE_MODE 当该模式为 node-join-control-plane 时,需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
- CONTROL_PLANE_ENDPOINT 为高可用 vip
- 可以根据需要选择适合自己的 ssh 方式
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
v1.0.x
向集群添加工作节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
tip: 你可以通过预先下载离线包和镜像来减少部署时间
export PRE_DOWNLOAD="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入集群
- MASTER_IP 为 master 节点 ip
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
向集群的 control-plane 添加 master 节点
在新节点上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待新节点加入 control-plane
- MASTER_IP 需要填已有的 master 节点 ip
- CONTROL_PLANE_ENDPOINT 为高可用 vip
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-control-plane"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
5.4 - 多节点高可用部署
本文提供 Kubernetes 的高可用部署和 KubeCube 的高可用部署方案,VIP 的实现需要用户自行提供
v1.4.x
主机规划
IP 地址 | 主机名 | 角色 |
---|---|---|
10.173.32.2 | lb1 | Keepalived & HAproxy |
10.173.32.3 | lb2 | Keepalived & HAproxy |
10.173.32.4 | master1 | master, etcd |
10.173.32.5 | master2 | master, etcd |
10.173.32.6 | master3 | master, etcd |
10.173.32.7 | worker1 | worker |
10.173.32.8 | worker2 | worker |
10.173.32.9 | worker3 | worker |
10.173.32.10 | vip 地址 |
⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1
部署高可用 Kubernetes
KubeCube 部署脚本提供部署高可用 k8s 的能力,当然,你也可以使用其他工具搭建高可用的 k8s 集群
开始安装
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 Kubernetes 安装完成,master2 和 master3 加入 control-plane 的方式与之相同
- CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip,在此我们用任意 master 节点的 ip 代替
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
worker1 作为工作节点加入集群
在 worker1 上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 worker1 加入集群,worker2 和 worker3 加入集群的方式与之相同
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"
部署高可用 KubeCube
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.4
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 KubeCube 部署完成
- install.conf
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"
v1.2.x
主机规划
IP 地址 | 主机名 | 角色 |
---|---|---|
10.173.32.2 | lb1 | Keepalived & HAproxy |
10.173.32.3 | lb2 | Keepalived & HAproxy |
10.173.32.4 | master1 | master, etcd |
10.173.32.5 | master2 | master, etcd |
10.173.32.6 | master3 | master, etcd |
10.173.32.7 | worker1 | worker |
10.173.32.8 | worker2 | worker |
10.173.32.9 | worker3 | worker |
10.173.32.10 | vip 地址 |
⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1
部署高可用 Kubernetes
KubeCube 部署脚本提供部署高可用 k8s 的能力,当然,你也可以使用其他工具搭建高可用的 k8s 集群
开始安装
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 Kubernetes 安装完成,master2 和 master3 加入 control-plane 的方式与之相同
- CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip,在此我们用任意 master 节点的 ip 代替
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
worker1 作为工作节点加入集群
在 worker1 上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 worker1 加入集群,worker2 和 worker3 加入集群的方式与之相同
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
部署高可用 KubeCube
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.2
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 KubeCube 部署完成
- install.conf
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# k8s cni, support now is calico only
CNI="calico"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"
v1.1.x
主机规划
IP 地址 | 主机名 | 角色 |
---|---|---|
10.173.32.2 | lb1 | Keepalived & HAproxy |
10.173.32.3 | lb2 | Keepalived & HAproxy |
10.173.32.4 | master1 | master, etcd |
10.173.32.5 | master2 | master, etcd |
10.173.32.6 | master3 | master, etcd |
10.173.32.7 | worker1 | worker |
10.173.32.8 | worker2 | worker |
10.173.32.9 | worker3 | worker |
10.173.32.10 | vip 地址 |
⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1
部署高可用 Kubernetes
KubeCube 部署脚本提供部署高可用 k8s 的能力,当然,你也可以使用其他工具搭建高可用的 k8s 集群
开始安装
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 Kubernetes 安装完成,master2 和 master3 加入 control-plane 的方式与之相同
- CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip,在此我们用任意 master 节点的 ip 代替
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP=""
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
worker1 作为工作节点加入集群
在 worker1 上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 worker1 加入集群,worker2 和 worker3 加入集群的方式与之相同
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
部署高可用 KubeCube
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.1
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 KubeCube 部署完成
- install.conf
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install k8s
INSTALL_KUBERNETES="false"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################
# +optional
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# the user who can access master node, it can be empty
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################
OFFLINE_INSTALL="false"
OFFLINE_PKG_PATH=""
- cube.conf
将kubecube_replicas
设置为3,使得 KubeCube 使用 3 副本部署,并且由于podAntiAffinity
# custom values for kubecube
kubecube_replicas=3
kubecube_args_logLevel="info"
v1.0.x
主机规划
IP 地址 | 主机名 | 角色 |
---|---|---|
10.173.32.2 | lb1 | Keepalived & HAproxy |
10.173.32.3 | lb2 | Keepalived & HAproxy |
10.173.32.4 | master1 | master, etcd |
10.173.32.5 | master2 | master, etcd |
10.173.32.6 | master3 | master, etcd |
10.173.32.7 | worker1 | worker |
10.173.32.8 | worker2 | worker |
10.173.32.9 | worker3 | worker |
10.173.32.10 | vip 地址 |
⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1
部署高可用 Kubernetes
开始安装
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 Kubernetes 安装完成
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
master2 节点加入 control-plane
在 master2 上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 master2 加入 control-plane
master3 加入 control-plane 与此类似,仅需修改
LOCAL_IP
为10.173.32.6
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-control-plane"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
worker1 作为工作节点加入集群
在 worker1 上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 worker1 加入集群
worker2 和 worker3 加入集群的方式与之类似,仅需修改
LOCAL_IP
为本机 IP 即可
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="true"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
部署高可用 KubeCube
在 master1 上执行部署脚本
KUBECUBE_VERSION=v1.0
export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash
设置脚本参数,并按照提示继续运行安装脚本并等待 KubeCube 部署完成
- install.conf
# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"
# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"
# if install k8s
INSTALL_KUBERNETES="false"
# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"
# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""
# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"
# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""
# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"
# k8s version you want to install
KUBERNETES_VERSION="1.20.9"
# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"
# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22
# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""
# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"
- cube.conf
将kubecube_replicas
设置为3,使得 KubeCube 使用 3 副本部署,并且由于podAntiAffinity
,它们会运行在非controlPlane
的节点上,并且每个节点仅运行单个副本
# custom values for kubecube
kubecube_replicas=3
kubecube_args_logLevel="info"
6 - 已有系统接入
6.1 - Prometheus
KubeCube 在部署时会自动安装 Prometheus 等监控组件,以实现监控功能。本文档介绍了如何在 KubeCube 中接入用户已有的 Prometheus。
准备工作
在集群部署好 Prometheus,Prometheus 可以正常监控到集群资源的数据;
在集群中部署好 KubeCube;
说明:部署好 KubeCube 后,由于集群内已有 Prometheus operator,多个 operator 会导致集群内 Prometheus 相关功能不可用,需要卸载 KubeCube 监控组件或删除本地 operator。
以平台管理员角色登录 KubeCube 管控集群。
步骤
1、添加 Label
由于 KubeCube 需要实现多集群监控,因此在 KubeCube 查询监控数据时,都会在 query 表达式中添加 cluster={clusterName}
来进行集群过滤。用户需要在 Prometheus 的 exporter 中添加这一 label,前端查询监控数据时才能查询到结果。
2、卸载 KubeCube 监控组件
方式一 页面操作:
点击页面右上角【切换到控制台】,点击任意空间,进入到控制台页面;
在左侧菜单栏点击【自定义资源CRD】,进入到集群级别 CRD 列表,可以点击右上方输入 “hotplug” 进行搜索,找到 “hotplugs.hotplug.kubecube.io” CRD,点击【v1】版本进入 CRD 详情页;
选择 common 实例,点击【设置YAML】,找到 spec.component. name=kubecube-monitoring,将 “status” 改成 “disabled”,即卸载 KubeCube 自带监控组件。
方式二 命令行操作:
kubectl edit hotplug common
- 找到 spec.component. name=kubecube-monitoring,将 “status” 改成 “disabled”。
详细配置说明见 热插拔 。
3、部署 ServiceMonitor
查看集群资源
查看控制台内监控数据,需要部署两个 ServiceMonitor:kubelet 和 kube-state-metrics 的 ServiceMonitor,样例如下:
部署 kubecube-monitoring-kubelet
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: kubecube-monitoring-kubelet namespace: kubecube-monitoring spec: endpoints: - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token honorLabels: true port: https-metrics relabelings: - sourceLabels: - __metrics_path__ targetLabel: metrics_path scheme: https tlsConfig: caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecureSkipVerify: true - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token honorLabels: true path: /metrics/cadvisor port: https-metrics relabelings: - sourceLabels: - __metrics_path__ targetLabel: metrics_path scheme: https tlsConfig: caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecureSkipVerify: true - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token honorLabels: true path: /metrics/probes port: https-metrics relabelings: - sourceLabels: - __metrics_path__ targetLabel: metrics_path scheme: https tlsConfig: caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecureSkipVerify: true jobLabel: k8s-app namespaceSelector: matchNames: - kube-system selector: matchLabels: k8s-app: kubelet
部署 kubecube-monitoring-kube-state-metrics
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: kubecube-monitoring-kube-state-metrics namespace: kubecube-monitoring spec: endpoints: - honorLabels: true port: http selector: matchLabels: app.kubernetes.io/instance: kubecube-monitoring app.kubernetes.io/name: kube-state-metrics
查看组件监控
当前 KubeCube 平台支持对组件的监控视图可视化查询,详细说明见 平台组件监控 。接入外部监控后,用户可按需在集群内部署对应组件的 ServiceMonitor。
各个 ServiceMonitor 的 yaml 可参考 https://github.com/kubecube-io/charts/tree/main/kubecube-monitoring/templates/exporters。
4、部署 Dashboard
查看集群资源(控制台内监控数据),需要部署:
- cube-resource-cluster.yaml
- cube-resource-namespace.yaml
- cube-resource-node.yaml
- cube-resource-persistentvolume.yaml
- cube-resource-pod.yaml
- cube-resource-workload.yaml
- default-rolebinding.yaml
查看组件监控,可以部署对应的 Dashboard:
- component-control-plane-pods.yaml
- component-coredns.yaml
- component-etcd.yaml
- component-kube-apiserver.yaml
- component-kube-controller-manager.yaml
- component-kube-proxy.yaml
- component-kube-scheduler.yaml
- component-kubelet.yaml
- component-prometheus.yaml
- component-thanos.yaml
5、修改 Nginx 配置
使用命令行:
kubectl edit configmap nginx-config -n kubecube-system
找到原有的地址配置,修改为自有 Prometheus 地址
upstream monitoring { server kubecube-thanos-query.kubecube-monitoring:9090; }
即 将
kubecube-thanos-query.kubecube-monitoring:9090
替换为外部地址。重启 pod:
kubectl delete pod frontend-xxxxxx-xxxxx -n kubecube-system
6.2 - ElasticSearch
KubeCube 提供了日志服务和操作审计服务,默认关闭。用户在开启后,日志服务和操作审计服务均会将日志发送到 ElasticSearch 进行存储,由 ElasticSearch 对日志进行管理。用户可以在 热插拔 中修改配置,安装内部 ElasticSearch,也可以配置外部的 ElasticSearch 地址,对接已有的 ElasticSearch。下面分别介绍如何在这两个功能中接入外部 ElasticSearch。
日志
方式一 页面操作:
点击页面右上角【切换到控制台】,点击任意空间,进入到控制台页面;
在左侧菜单栏点击【自定义资源CRD】,进入到集群级别 CRD 列表,可以点击右上方输入 “hotplug” 进行搜索,找到 “hotplugs.hotplug.kubecube.io” CRD,点击【v1】版本进入 CRD 详情页;
选择 common 实例,点击【设置YAML】,找到 spec.component. name=logseer,添加环境变量,如:
- name: logseer
namespace: logseer
pkgName: logseer-v1.0.0.tgz
status: disabled
env: |
address: elasticsearch-master.elasticsearch.svc
方式二 命令行操作:
kubectl edit hotplug pivot-cluster
- 找到 spec.component. name=logseer,添加环境变量,同上。
详细配置说明见 热插拔 。
操作审计
kubectl edit deploy audit -n kubecube-system
添加环境变量:AUDIT_WEBHOOK_HOST、AUDIT_WEBHOOK_INDEX、AUDIT_WEBHOOK_TYPE,如
env: - name: AUDIT_WEBHOOK_HOST value: http://elasticsearch-master.elasticsearch:9200 - name: AUDIT_WEBHOOK_INDEX value: audit - name: AUDIT_WEBHOOK_TYPE value: logs
注:如果同时配置了内部和外部 ElasticSearch,审计日志将优先发到外部 ElasticSearch。
其他详细说明见:操作审计 。
6.3 - 第三方认证系统
外部认证系统接入
KubeCube 中包含一套自有的认证系统,同时也支持多种类型的外部认证系统的接入。本文档介绍了如何在 KubeCube 中接入 GitHub、Ldap 以及通用认证接口三种认证系统的操作步骤。
GitHub 认证
1. 登记应用信息
在 GitHub 注册一个 Oauth 应用,Homepage URL 和 Authorization callback URL 均填写 http://{kubecube_host}/#/login
,创建成功后 Github 生成一个 ClientId,再手动创建一个 Client secret。
2. 修改配置文件
在管控集群修改 configmap:kubectl edit cm kubecube-auth-config -n kubecube-system
,修改内容如下:
apiVersion: v1
kind: ConfigMap
data:
github: |
enabled: true
clientId: 80b802dc59eeb847ed00
clientSecret: 83dc8eb788f706de3449d45e61f45ebdca433de2
host: http://10.219.196.107:30080
参数说明如下:
参数 | 说明 | 类型 | 默认值 |
---|---|---|---|
enabled | 是否开启 Github 登录 | boolean | false |
clientId | GitHub 授权的 ClientId | string | |
clientSecret | GitHub 授权的 Client secret | string | |
host | KubeCube 服务器地址 | string |
3. 访问前端登录页面
访问 KubeCube 前端登录页面,选择使用GitHub账号登录;
授权应用,点击 Authorize xxxapp,即可登录到 KubeCube。
KubeCube 会使用 Github 返回的信息自动在集群中创建该用户,并标记该用户的 “LOGINTYPE” 为 “github”。
Ldap 认证
KubeCube 支持接入用户已部署的 LDAP 来进行认证。具体步骤如下。
1. 添加启动参数
为 KubeCube Deployment 添加启动参数,参数说明如下:
参数 | 说明 | 类型 | 是否必填 | 默认值 | 示例 |
---|---|---|---|---|---|
ldap-is-enable | 是否开启 Ldap 登录 | boolean | 是 | false | true |
ldap-server | Ldap 服务器地址 | string | 是 | 10.219.196.107 | |
ldap-port | Ldap 服务器端口号 | string | 否 | 389 | 389 |
ldap-base | Ldap 查询分区 | string | 是 | dc=example,dc=com | |
ldap-admin-user-account | Ldap 管理员账号 | string | 是 | cn=admin,dc=example,dc=com | |
ldap-admin-password | Ldap 管理员密码 | string | 是 | admin123456 | |
ldap-object-class | Ldap 对象类 | string | 否 | person | person |
ldap-login-name-config | 用户名所在配置 | string | 否 | uid | cn |
ldap-object-category | Ldap objectcategory | string | 否 | dc=example,dc=com |
示例如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubecube
namespace: kubecube-system
spec:
template:
spec:
containers:
- args:
- -ldap-is-enable=true
- -ldap-object-class=person
- -ldap-server=10.219.196.107
- -ldap-base=dc=example,dc=com
- -ldap-admin-user-account=cn=admin,dc=example,dc=com
- -ldap-admin-password=admin123456
- -ldap-login-name-config=cn
2. 请求登录
接口路径
/api/v1/cube/login
接口方法
POST
请求参数
参数名称 | 参数说明 | 参数来源 | 参数类型 | 是否必须 | 备注 |
---|---|---|---|---|---|
name | 用户名 | body | string | 是 | |
password | 密码 | body | string | 是 | |
loginType | 登录方式 | body | string | 是 | ldap |
响应
如果 Ldap 返回认证成功,KubeCube 在集群中创建该用户,并标记该用户的 “LOGINTYPE” 为 “ldap”。
响应码 | 状态 | 描述 | 响应体 |
---|---|---|---|
200 | OK | 请求成功,返回在集群中创建的User信息 | User |
数据模型
User
参数名称 | 参数说明 | 参数类型 | 是否必须 | 备注 |
---|---|---|---|---|
kind | User | string | 是 | |
apiVersion | user.kubecube.io/v1 | string | 是 | |
metadata | 元数据 | Metadata | 是 | |
spec | UserSpec | 是 | ||
status | UserStatus | 是 |
Metadata
参数名称 | 参数说明 | 参数类型 | 是否必须 | 备注 |
---|---|---|---|---|
name | 该用户在集群中的名称 | string | 是 | |
labels | 标签 | map[string]string | 否 | Ldap返回的真实用户名保存在标签中 |
creationTimestamp | 创建此对象时的时间戳 | Time | 否 | |
generation | 所需状态的特定生成的序列号 | integer | 否 | |
uid | 资源对象在集群中的唯一标识 | string | 否 | |
selfLink | 资源关联的url | string | 否 |
UserSpec
参数名称 | 参数说明 | 参数类型 | 是否必须 | 备注 |
---|---|---|---|---|
loginType | 登录方式 | string | 是 | ldap |
UserStatus
参数名称 | 参数说明 | 参数类型 | 是否必须 | 备注 |
---|---|---|---|---|
lastLoginTime | 上次登录时间 | Time | 否 | |
lastLoginIP | 上次登录IP | string | 否 |
请求示例
curl https://0.0.0.0:7443/api/v1/cube/login -X POST -d '{"name": "test123","password":"123456","loginType":"ldap"}' --header "Content-Type: application/json"
返回示例
{
"kind": "User",
"apiVersion": "user.kubecube.io/v1",
"metadata": {
"labels": {
"name": "test123"
},
"spec": {
"loginType": "ldap",
},
"status": {
"lastLoginTime": "2022-07-06T06:25:37Z",
"lastLoginIP": "10.219.196.107"
}
}
通用认证
通用认证指的是通过请求第三方接口的方式进行认证。KubeCube 提供了一种通用的接入方式,但对接口的返回有一定要求。具体步骤如下。
1. 接口准备
第三方认证接口要求:
- 请求 url 为配置的固定值,KubeCube 会将请求携带的 header 进行转发,因此第三方接口对 header 进行认证;
- 接口返回格式为
map[string]interface{}
,其中包含一组值为 key=“name”,value=“{用户名}”。
2. 添加启动参数
为 KubeCube Deployment 添加启动参数,参数说明如下:
参数 | 说明 | 类型 | 是否必填 | 默认值 | 示例 |
---|---|---|---|---|---|
generic-auth-is-enable | 是否开启通用认证方式 | boolean | 是 | false | true |
generic-auth-url | 第三方认证url | string | 是 | https://kubecube123.com/api/v1/demo/auth | |
generic-auth-method | 第三方认证请求方式 | string | 是 | GET | |
generic-auth-scheme | 请求协议 | string | 否 | http | https |
generic-auth-insecure-skip-verify | 是否跳过安全校验 | string | 否 | true | |
generic-auth-tls-cert | tls证书 | string | 否 | LS0NGc9PQotLS0tLUVOFURS0tLS0t | |
generic-auth-tls-key | tls密钥 | string | 否 | LS0NsfGcQotfdLS0tLUVOFURtLS0ta1 |
示例如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubecube
namespace: kubecube-system
spec:
template:
spec:
containers:
- args:
- -generic-auth-is-enable=true
- -generic-auth-url=https://kubecube123.com/api/v1/demo/auth
- -generic-auth-method=GET
- -generic-auth-scheme=https
- -generic-auth-insecure-skip-verify=true
开启通用认证方式后,用户可跳过登录直接访问 KubeCube。KubeCube 会将请求携带的 header 转发给配置的第三方认证平台,由第三方认证平台返回的结果决定请求是否通过认证。