私有云架构2025
2025年一月，回顾最初构建「Cloud Atlas」，我购买的二手 HPE ProLiant DL360 Gen9服务器 已经使用了三年多，当初磕磕绊绊摸索实践的 私有云架构 ，有些已经实现、有些则一直在脑海中未曾落地。
作为项目重启，我准备在2025年重新出发，来完成 cloud-atlas.io 从0开始的实践:
二手 HPE ProLiant DL360 Gen9服务器 将继续发光发热，作为主力计算密集设备(这是我投资最多的二手企业级设备):
源码编译构建 LFS(Linux from scratch) 和 BLFS(Beyond Linux from scratch) 作为 KVM 虚拟化底座，用最小化的内核和OS来构建坚如磐石的Host主机
所有软件包都手工编译和维护，不断打磨以实现最优化
虚拟化部署: 通过 KVM嵌套虚拟化 构建模拟集群
-
Nvidia Tesla P10 GPU运算卡 和 Tesla T10 廉颇老矣尚能饭否: 学习部署和维护基于 Kubernetes 容器化的大规模集群
从0开始系统学习
虚拟化服务器分布
虚拟化层 |
主机IP |
主机名 |
cpu |
内存(G) |
磁盘(G) |
NUMA |
说明 |
|---|---|---|---|---|---|---|---|
1 |
192.168.6.51 |
z-o7k-m-1 |
4 |
16 |
0 |
OpenStack 管控 1 |
|
1 |
192.168.6.52 |
z-o7k-m-2 |
4 |
16 |
0 |
OpenStack 管控 2 |
|
1 |
192.168.6.53 |
z-o7k-m-3 |
4 |
16 |
0 |
OpenStack 管控 3 |
|
1 |
192.168.6.61 |
z-o7k-n-1 |
2 |
8 |
1 |
OpenStack Nova 1 |
|
1 |
192.168.6.62 |
z-o7k-n-2 |
2 |
8 |
1 |
OpenStack Nova 2 |
|
1 |
192.168.6.63 |
z-o7k-n-3 |
2 |
8 |
1 |
OpenStack Nova 3 |
|
1 |
192.168.6.64 |
z-o7k-n-4 |
2 |
8 |
1 |
OpenStack Nova 4 |
|
1 |
192.168.6.65 |
z-o7k-n-5 |
2 |
8 |
1 |
OpenStack Nova 5 |
|
1 |
192.168.6.66 |
z-o7k-n-6 |
2 |
8 |
1 |
OpenStack Nova 6 |
|
1 |
192.168.6.67 |
z-o7k-n-7 |
2 |
8 |
1 |
OpenStack Nova 7 |
|
1 |
192.168.6.68 |
z-o7k-n-8 |
2 |
8 |
1 |
OpenStack Nova 8 |
|
1 |
192.168.6.69 |
z-o7k-n-9 |
2 |
8 |
1 |
OpenStack Nova 9 |
|
1 |
192.168.6.70 |
z-o7k-n-10 |
2 |
8 |
1 |
OpenStack Nova 10 |
|
1 |
192.168.6.101 |
z-k8s-m-1 |
2 |
8 |
0 |
K8s 管控 1 |
|
1 |
192.168.6.101 |
z-k8s-api |
apiserver入口 |
||||
1 |
192.168.6.102 |
z-k8s-m-2 |
2 |
8 |
0 |
K8s 管控 2 |
|
1 |
192.168.6.102 |
z-k8s-api |
apiserver入口 |
||||
1 |
192.168.6.103 |
z-k8s-m-3 |
2 |
8 |
0 |
K8s 管控 3 |
|
1 |
192.168.6.103 |
z-k8s-api |
apiserver入口 |
||||
1 |
192.168.6.111 |
z-k8s-n-1 |
2 |
8 |
1 |
K8s node 1 |
|
1 |
192.168.6.112 |
z-k8s-n-2 |
2 |
8 |
1 |
K8s node 2 |
|
1 |
192.168.6.113 |
z-k8s-n-3 |
2 |
8 |
1 |
K8s node 3 |
|
1 |
192.168.6.114 |
z-k8s-n-4 |
2 |
8 |
1 |
K8s node 4 |
|
1 |
192.168.6.115 |
z-k8s-n-5 |
2 |
8 |
1 |
K8s node 5 |
|
1 |
192.168.6.121 |
z-okd-m-1 |
2 |
8 |
1 |
OKD(OpenShift) 管控 1 |
|
1 |
192.168.6.122 |
z-okd-m-2 |
2 |
8 |
1 |
OKD(OpenShift) 管控 2 |
|
1 |
192.168.6.123 |
z-okd-m-3 |
2 |
8 |
1 |
OKD(OpenShift) 管控 3 |
|
1 |
192.168.6.131 |
z-okd-n-1 |
2 |
8 |
1 |
OKD(OpenShift) node 1 |
|
1 |
192.168.6.132 |
z-okd-n-2 |
2 |
8 |
1 |
OKD(OpenShift) node 2 |
|
1 |
192.168.6.133 |
z-okd-n-3 |
2 |
8 |
1 |
OKD(OpenShift) node 3 |
|
1 |
192.168.6.134 |
z-okd-n-4 |
2 |
8 |
1 |
OKD(OpenShift) node 4 |
|
1 |
192.168.6.135 |
z-okd-n-5 |
2 |
8 |
1 |
OKD(OpenShift) node 5 |
|
1 |
192.168.6.151 |
z-http |
http服务案例IP |
||||
1 |
192.168.6.190 |
z-gfs-0 |
2 |
8 |
12块2G |
1 |
GlusterFS node 0 |
1 |
192.168.6.191 |
z-gfs-1 |
2 |
8 |
12块2G |
1 |
GlusterFS node 1 |
1 |
192.168.6.192 |
z-gfs-2 |
2 |
8 |
12块2G |
1 |
GlusterFS node 2 |
1 |
192.168.6.193 |
z-gfs-3 |
2 |
8 |
12块2G |
1 |
GlusterFS node 3 |
1 |
192.168.6.194 |
z-gfs-4 |
2 |
8 |
12块2G |
1 |
GlusterFS node 4 |
1 |
192.168.6.195 |
z-gfs-5 |
2 |
8 |
12块2G |
1 |
GlusterFS node 5 |
0 |
192.168.6.199 |
acloud |
4 |
16 |
0 |
Lenovo ThinkPad X220 |
|
0 |
192.168.6.200 |
zcloud |
48 |
64 |
512SSD |
HP DL360 Gen9 |
|
0 |
192.168.6.200 |
wpad |
WPAD服务 |
||||
0 |
192.168.6.200 |
proxy |
Proxy服务 |
||||
1 |
192.168.6.201 |
z-b-store-1 |
2 |
4 |
512HDD |
0(0 1) |
virtio-blk直接访问HDD(存储备份基础服务) |
1 |
192.168.6.202 |
z-b-store-2 |
2 |
4 |
512HDD |
0(2 3) |
virtio-blk直接访问HDD(存储备份基础服务) |
1 |
192.168.6.203 |
z-b-store-3 |
2 |
4 |
512HDD |
0(4 5) |
virtio-blk直接访问HDD(存储备份基础服务) |
1 |
192.168.6.204 |
z-b-data-1 |
4 |
16 |
1024NVMe |
0(24 25 26 27) |
iommu NVMe(数据基础服务) |
1 |
192.168.6.204 |
etcd |
基础etcd服务 |
||||
1 |
192.168.6.205 |
z-b-data-2 |
4 |
16 |
1024NVMe |
0(28 29 30 31) |
iommu NVMe(数据基础服务) |
1 |
192.168.6.205 |
etcd |
基础etcd服务 |
||||
1 |
192.168.6.206 |
z-b-data-3 |
4 |
16 |
1024NVMe |
0(32 33 34 35) |
iommu NVMe(数据基础服务) |
1 |
192.168.6.206 |
etcd |
基础etcd服务 |
||||
1 |
192.168.6.211 |
z-b-cache-1 |
2 |
8 |
0(6 7) |
近端访问 z-data(缓存基础服务) |
|
1 |
192.168.6.212 |
z-b-cache-2 |
2 |
8 |
0(8 9) |
近端访问 z-data(缓存基础服务) |
|
1 |
192.168.6.213 |
z-b-cache-3 |
2 |
8 |
0(10 11) |
近端访问 z-data(缓存基础服务) |
|
1 |
192.168.6.221 |
z-b-mon-1 |
2 |
4 |
基础监控 |
||
1 |
192.168.6.222 |
z-b-mon-2 |
2 |
4 |
基础监控 |
||
1 |
192.168.6.231 |
z-numa |
4 |
4 |
6 |
0 1 |
测试功能-numa |
1 |
192.168.6.232 |
z-iommu |
2 |
4 |
6 |
1 |
测试功能-iommu(fedora35) |
1 |
192.168.6.233 |
z-vgpu |
1 |
2 |
6 |
1 |
测试功能-vgpu(fedora35) |
1 |
192.168.6.234 |
z-udev |
1 |
2 |
6 |
1 |
编译(ubuntu20) |
1 |
192.168.6.235 |
z-kdev |
1 |
2 |
6 |
1 |
内核测试(fedora35) |
1 |
192.168.6.238 |
centos7-zfs |
1 |
2 |
4 |
1 |
模版(基于zfs存储卷的centos7) |
1 |
192.168.6.239 |
z-codeready |
2 |
2 |
6 |
OpenShift开发环境CodeReady |
|
1 |
192.168.6.240 |
z-devstack |
2 |
2 |
6 |
OpenStack开发环境DevStack |
|
1 |
192.168.6.241 |
z-centos6 |
1 |
2 |
6 |
模版 |
|
1 |
192.168.6.242 |
z-centos7 |
1 |
2 |
6 |
模版 |
|
1 |
192.168.6.243 |
z-centos8 |
1 |
2 |
6 |
模版 |
|
1 |
192.168.6.244 |
z-fedora35 |
1 |
2 |
6 |
模版 |
|
1 |
192.168.6.245 |
z-ubuntu18 |
1 |
2 |
6 |
模版 |
|
1 |
192.168.6.246 |
z-ubuntu20 |
1 |
2 |
6 |
模版(ubuntu 20.04) |
|
1 |
192.168.6.247 |
z-ubuntu20-rbd |
2 |
4 |
7 |
模版(ubuntu 20.04)Ceph存储 |
|
1 |
192.168.6.248 |
z-win7 |
2 |
4 |
7 |
Win7+Ceph存储 |
|
1 |
192.168.6.249 |
z-centos9-rbd |
2 |
4 |
7 |
模版(CentOS 9 Stream)Ceph存储 |
|
1 |
192.168.6.250 |
z-centos8-rbd |
2 |
4 |
7 |
模版(CentOS 8)Ceph存储 |
|
1 |
192.168.6.251 |
z-centos7-rbd |
2 |
4 |
7 |
模版(CentOS 7)Ceph存储 |
|
1 |
192.168.6.252 |
z-vdev |
2 |
4 |
6 |
1 |
fedora35(vgpu 8G) |
1 |
192.168.6.253 |
z-dev |
2 |
4 |
6 |
0 |
fedora35 |
0 |
192.168.6.254 |
dl360-ilo |
HP DL360 Gen9带外管理 |
||||
= |
= |
= |
= |
= |
= |
= |
= |
1 |
192.168.8.116 |
y-k8s-m-1 |
2 |
8 |
1 |
K8s 管控1 |
|
1 |
192.168.8.117 |
y-k8s-m-2 |
2 |
8 |
1 |
K8s 管控2 |
|
1 |
192.168.8.118 |
y-k8s-m-3 |
2 |
8 |
1 |
K8s 管控3 |
|
1 |
192.168.8.119 |
y-k8s-n-1 |
2 |
8 |
1 |
K8s node 1 |
|
1 |
192.168.8.120 |
y-k8s-n-2 |
2 |
8 |
1 |
K8s node 2 |