【信创-k8s】重磅-鲲鹏arm+麒麟V10离线部署k8s1.30+kubesphere4.1.3

随着信创产业的推进，鲲鹏arm64架构得以快速发展。而由于信创领域的主要客户通常部署在内网环境中，这使得离线部署成为该架构方案实施过程中不可或缺的关键环节。

**环境涉及软件版本信息**

+ 服务器芯片: **鲲鹏920/飞腾2000(arm64)**
+ 操作系统：**银河麒麟V10**
+ Containerd: **1.7.13**
+ Kubernetes：**v1.30.12**
+ KubeSphere：**v4.1.3**
+ KubeKey: **v3.1.9(二开版)**
+ Docker: **24.0.9**
+ DockerCompose: **v2.26.1**
+ Harbor: **v2.7.1**
+ Prometheus: **v2.51.2**

**服务器基本信息**

```plain
[root@node1 ~]# uname -a
Linux node1 4.19.90-17.5.ky10.aarch64 #1 SMP Fri Aug 7 13:35:33 CST 2020 aarch64 aarch64 aarch64 GNU/Linux
[root@node1 ~]#
[root@node1 ~]# cat /etc/os-release
NAME="Kylin Linux Advanced Server"
VERSION="V10 (Tercel)"
ID="kylin"
VERSION_ID="V10"
PRETTY_NAME="Kylin Linux Advanced Server V10 (Tercel)"
ANSI_COLOR="0;31"

[root@node1 ~]#
```

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432543205-7984761a-4881-479a-b5ef-088c95454c57.png)

## 1.说明
作者使用`k8s`和`kubesphere`过程中已适配芯片和操作系统如下：

+ CPU芯片：
- 鲲鹏
- 飞腾
- 海光
- 兆芯
- 国际芯片:interl、amd等
+ 操作系统
- 银河麒麟V10
- 麒麟国防版
- 麒麟信安
- 中标麒麟V7
- 统信 UOS
- 华为欧拉 openEuler、移动大云
- 阿里龙蜥 Anolis OS
- 腾讯 TencentOS
- 国际操作系统：centos、ubuntu、debian等

本文由 [编码如写诗-天行1st] 原创编写，有任何问题可添加作者微信 [sd_zdhr] 获取帮助。

关于我：

+ 主要从事后端开发，兼具前端、运维及全栈工程师，热爱`Golang`、`Docker`、`kubernetes`、`KubeSphere`。
+ 信创服务器`k8s`&`KubeSphere`布道者、`KubeSphere`离线部署布道者

### 关于kt
`kt`是基于`kk`二次开发产物，主要为适配信创国产化环境和简化`arm`部署过程。

支持`arm64`和`amd64`架构国产操作系统，已适配芯片+操作系统如上。

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432543210-08ada6f3-910b-4700-9b48-ef97fe678013.png)

## 2.前提条件
参考如下示例准备至少三台主机，其中node1可省略，让master节点即是主节点也是工作节点

| **主机名** | **IP** | **架构** | **OS** | **用途** |
| --- | --- | --- | --- | --- |
| node | 192.168.3.249 | X86_64 | Ubuntu24.04 | 联网主机，用于制作离线包 |
| node1 | 192.168.0.80 | arm64 | 麒麟V10 | 离线环境主节点，镜像仓库节点 |

## 3.构建离线包
在node可联网节点上操作

### 3.1 上传kt
将`[kt_x86.tar.gz](https://pan.xunlei.com/s/VOSc93lCJRKUTy_L9U5DHzDhA1?pwd=3r38#。 "kt_x86版本")`上传至可联网节点解压后操作，即日起至`2025-07-09`免费使用。

### 3.2 创建 manifest 文件
```plain
export KKZONE=cn
./kt create manifest --with-kubernetes v1.30.12 --with-registry
```

### 3.3 编辑 manifest 文件
```plain
vi manifest-sample.yaml
```

```yaml
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Manifest
metadata:
name: sample
spec:
arches:
- arm64
operatingSystems: []
kubernetesDistributions:
- type: kubernetes
version: v1.30.12
components:
helm:
version: v3.14.3
cni:
version: v1.2.0
etcd:
version: v3.5.13
containerRuntimes:
- type: docker
version: 24.0.9
- type: containerd
version: 1.7.13
calicoctl:
version: v3.27.4
crictl:
version: v1.29.0
docker-registry:
version: "2"
harbor:
version: v2.7.1
docker-compose:
version: v2.26.1
images:
- registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.9
- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.30.12
- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.30.12
- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.30.12
- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.30.12
- registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.9.3
- registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.22.20
- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.27.4
- registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.27.4
- registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.27.4
- registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.27.4
# ks
- dockerhub.kubekey.local/gjing1st/kubesphere/ks-extensions-museum:v1.1.6
- dockerhub.kubekey.local/gjing1st/kubesphere/ks-controller-manager:v4.1.3
- dockerhub.kubekey.local/gjing1st/kubesphere/ks-apiserver:v4.1.3
- dockerhub.kubekey.local/gjing1st/kubesphere/ks-console:v4.1.3
- dockerhub.kubekey.local/gjing1st/kubesphere/kubectl:v1.27.16
registry:
auths: {}
```

### 3.4 导出离线制品
```plain
export KKZONE=cn
./kt artifact export -m manifest-sample.yaml -o artifact-arm-k8s13012-ks413.tar.gz
```

可以看到下载了`arm64` 版本的`harbor`，由于`harbor`官方不支持arm版本，因此`kk`也不支持`arm`版本`harbor`。此版本`harbor`和`kk`为自己制作,由于`harbor:v2.8.0`版本之后不支持扩展`helm`功能，我司需要用其`helm`管理应用，所以这里使用`v2.7.1`版本。

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432543321-c17561b9-f12b-4f6a-ad97-12dc12afe4cf.png)

### 3.5 下载 KubeSphere Core Helm Chart
安装helm

```plain
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
```

下载 KubeSphere Core Helm Chart

```plain
VERSION=1.1.3 # Chart 版本
helm fetch https://charts.kubesphere.io/main/ks-core-${VERSION}.tgz
```

## 4 离线部署准备
### 4.1 将安装包拷贝至离线环境
将 kt、制品 artifact 、Helm文件等通过介质拷贝至**node1节点**。

### 4.2 初始化操作系统
**所有节点**,上传`[kt_arm.tar.gz](https://pan.xunlei.com/s/VOSc8AKcsUM5K5Lxma9KBW2sA1?pwd=85gd#。 "kt_arm版本")`解压后执行`./kt init-os`，已适配操作系统和架构见`1.说明`

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432543367-4242960a-9c7e-43ae-b8c3-02d55350c38f.png)

### 4.3 修改配置文件
主要修改相关节点和harbor信息

```plain
vi config-sample.yaml
```

```yaml
kind: Cluster
metadata:
name: sample
spec:
hosts:
- {name: node1, address: 192.168.0.80, internalAddress: 192.168.0.80, user: root, password: "123456"}
roleGroups:
etcd:
- node1
control-plane:
- node1
worker:
- node1
registry:
- node1
controlPlaneEndpoint:
## Internal loadbalancer for apiservers
# internalLoadbalancer: haproxy

domain: lb.kubesphere.local
address: ""
port: 6443
kubernetes:
version: v1.30.12
clusterName: cluster.local
autoRenewCerts: true
containerManager: docker
etcd:
type: kubekey
network:
plugin: calico
kubePodsCIDR: 10.233.64.0/18
kubeServiceCIDR: 10.233.0.0/18
## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
multusCNI:
enabled: false
registry:
type: harbor
registryMirrors: []
insecureRegistries: []
privateRegistry: "dockerhub.kubekey.local"
namespaceOverride: "kubesphereio"
auths: # if docker add by `docker login`, if containerd append to `/etc/containerd/config.toml`
"dockerhub.kubekey.local":
username: "admin"
password: Harbor@123 # 此处可自定义，kk3.1.8新特性
skipTLSVerify: true # Allow contacting registries over HTTPS with failed TLS verification.
plainHTTP: false # Allow contacting registries over HTTP.
certsPath: "/etc/docker/certs.d/dockerhub.kubekey.local"
addons: []
```

### 4.4 创建镜像仓库
```plain
./kt init registry -f config-sample.yaml -a artifact-arm-k8s13012-ks413.tar.gz
```

可以看到使用了`arm64`版本的harbor

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432543267-fb588970-d341-485d-9647-529643d11a77.png)

### 4.5 创建harbor项目
说明：

Harbor 管理员账号：**admin**，密码：**Harbor@123**。密码同步使用配置文件中的对应password

harbor 安装文件在 `/opt/harbor` 目录下，可在该目录下对 harbor 进行运维。

```plain
vi create_project_harbor.sh
```

```bash
#!/usr/bin/env bash

url="https://dockerhub.kubekey.local" # 或修改为实际镜像仓库地址
user="admin"
passwd="Harbor@123"

harbor_projects=(
gjing1st
kubesphere
kubesphereio
)

for project in "${harbor_projects[@]}"; do
echo "creating $project"
curl -u "${user}:${passwd}" -X POST -H "Content-Type: application/json" "${url}/api/v2.0/projects" -d "{ \"project_name\": \"${project}\", \"public\": true}" -k # 注意在 curl 命令末尾加上 -k
done
```

#### 创建 Harbor 项目
```plain
chmod +x create_project_harbor.sh

./create_project_harbor.sh
```

#### 验证
![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544117-e64c6d10-4117-42d3-865a-6d48712d946b.png)

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544233-bc132fd3-033c-4d4b-890d-be6bbe774c43.png)

## 5 安装Kubernetes
执行以下命令创建 Kubernetes 集群：

```plain
./kt create cluster -f config-sample.yaml -a artifact-arm-k8s13012-ks413.tar.gz --with-local-storage
```

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544162-cc938cb6-f026-4abb-b4d9-1fb9fc0936e9.png)

等待大概两分钟左右看到成功消息

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544161-ab5eae7d-3212-4ad9-a50d-e4d99f92ec28.png)

## 6 安装 KubeSphere
```plain
helm upgrade --install -n kubesphere-system --create-namespace ks-core ks-core-1.1.5.tgz \
--set global.imageRegistry=dockerhub.kubekey.local/ks \
--set extension.imageRegistry=dockerhub.kubekey.local/ks \
--set ksExtensionRepository.image.tag=v1.1.5 \
--debug \
--wait
```

等待大概30秒左右看到成功消息

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544042-23f04a8e-3baf-454c-beeb-251e7a7ce98b.png)

## 7 验证
登录页面

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544992-6414b519-f841-4e7c-83fb-5ffdfdc754b9.png)

初次登录需要换密码，如果不想换也可以继续填写`P@88w0rd`，不过建议更换

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544851-0f21ba64-074f-4829-b261-2f464e20863d.png)

首页

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432545183-86d151c7-ec77-4c01-acb9-e124a743cbd4.png)

集群节点版本信息

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432545440-a44cd776-37a4-4f91-a7ae-5916f9c1b919.png)

概览

![](https://cdn.nlark.com/yuque/0/2025/png/12795725/1749432544952-a1cf5c48-9e4c-4a0d-a445-ec3db57c274f.png)