workstdtion模拟交换机划分vlan,以及物理服务器网卡bond设置

workstdtion设置

https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_c2444bc85394584e2798df97ef8b9549.png

一. 第一种ubuntu系统网卡配置

查看端口速率：

  • 判断是千兆网卡还是百兆网卡 查看支持的速率 > ethtool ens33

    1
    2
    3
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair

完整网卡配置如下：共接四根网线 bond0>>bond3 处网卡需要是up状态 物理服务器时只需要插入网线查看端口状态即可判断网口状态 （交换机关键参数交换机 trunk + LACP）

  • bond3 浮动IP 192.168.1.10/24 网口上 位于千兆口 接一根交换机上划分的一根同网段网线 用于远程连接

  • bond0 万兆口接一根交换机划分的 50 段网络 用于ceph osd 内部通信ceph（–cluster-network）网络

  • bond1 上面接交换机上对 该网口划分了vlan60 vlan70 vlan80 vlan90 的网线（交换机 trunk ）

  • bond2 上面接交换机上对 192.168.1.10/24 网段划分的浮动IP 池 的网线 例如：本文则需要为桥接模式模拟浮动IP 并且此网口不配置IP（下方文件测试已配置了IP 实际过程中自行注释）实验中单节点vmware环境bond2下网卡ens39桥接模式 部署单节点stack一切正常 但是双节点bond2下网卡ens39 双节点ens39网卡均配置桥接模式时 只有一个节点可用 >> 此时 只需要将双节点上方均没有划分bond的任意网卡桥接如ens41 修改global文件用于浮动IP池使用的网卡为ens41 再重新部署 即 正常。

    https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_6e28aee0f289c98393d974c614fc09c3.png

  • 三台服务器 此种配置完成后 是可以互通的无论ping那个网段都是通的
    ubuntu网卡bond接线与ceph及start集成揽图

    https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_cea9571e22249bca02d6cb339ed3c060.png

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33: {}
    ens37: {}
    ens38: {}
    ens39: {}
    ens40: {}
    ens41: {}
    ens42: {}
    ens43: {}
  bonds:
    bond0:
      addresses: ["10.56.50.11/24"]  # 此网口用万兆口 用于ceph osd间通信（--cluster-network）
      interfaces:
        - ens37
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond1:
      interfaces:  
        - ens38
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond2:
      interfaces:
        - ens39
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond3:
      addresses: ["192.168.1.10/24"]      # 此网口用千兆口 >> 电口 用于远程连接 及浮动IP使用
      gateway4: 192.168.1.1
      interfaces:
        - ens33
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
  vlans:
    vlan60:
      id: 60
      link: bond1
      addresses: ["10.56.60.11/24"]
    vlan70:
      id: 70
      link: bond1
      addresses: ["10.56.70.11/24"]
    vlan80:
      id: 80
      link: bond1
      addresses: ["10.56.80.11/24"]
    vlan90:
      id: 90
      link: bond1
      addresses: ["10.56.90.11/24"]
  version: 2

效果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
  System information as of Wed 03 Apr 2024 04:59:36 AM UTC

  System load:              0.02
  Usage of /:               26.9% of 39.07GB
  Memory usage:             11%
  Swap usage:               0%
  Processes:                261
  Users logged in:          1
  IPv4 address for bond0:   10.56.50.11
  IPv4 address for bond3:   192.168.1.10
  IPv6 address for bond3:   240e:36f:151f:a000:e407:31ff:fee2:70f9
  IPv4 address for docker0: 172.17.0.1
  IPv4 address for vlan60:  10.56.60.11
  IPv4 address for vlan70:  10.56.70.11
  IPv4 address for vlan80:  10.56.80.11
  IPv4 address for vlan90:  10.56.90.11
#-----------------------------------------------------------------
root@ceph1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond3 state UP group default qlen 1000
    link/ether e6:07:31:e2:70:f9 brd ff:ff:ff:ff:ff:ff
3: ens37: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 5a:c0:b2:8b:fe:e7 brd ff:ff:ff:ff:ff:ff
4: ens38: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond1 state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
5: ens39: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond2 state UP group default qlen 1000
    link/ether 76:88:82:b8:46:ba brd ff:ff:ff:ff:ff:ff
6: ens40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:b0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ab0/64 scope link 
       valid_lft forever preferred_lft forever
7: ens41: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:ba brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8aba/64 scope link 
       valid_lft forever preferred_lft forever
8: ens42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:c4 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ac4/64 scope link 
       valid_lft forever preferred_lft forever
9: ens43: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:ce brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ace/64 scope link 
       valid_lft forever preferred_lft forever
10: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5a:c0:b2:8b:fe:e7 brd ff:ff:ff:ff:ff:ff
    inet 10.56.50.11/24 brd 10.56.50.255 scope global bond0
       valid_lft forever preferred_lft forever
    inet6 fe80::58c0:b2ff:fe8b:fee7/64 scope link 
       valid_lft forever preferred_lft forever
11: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
12: bond2: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 76:88:82:b8:46:ba brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7488:82ff:feb8:46ba/64 scope link 
       valid_lft forever preferred_lft forever
13: bond3: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e6:07:31:e2:70:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global bond3
       valid_lft forever preferred_lft forever
    inet6 240e:36f:151f:a000:e407:31ff:fee2:70f9/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 198193sec preferred_lft 111793sec
    inet6 fe80::e407:31ff:fee2:70f9/64 scope link 
       valid_lft forever preferred_lft forever
14: vlan90@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.90.11/24 brd 10.56.90.255 scope global vlan90
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
15: vlan60@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.60.11/24 brd 10.56.60.255 scope global vlan60
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
16: vlan70@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.70.11/24 brd 10.56.70.255 scope global vlan70
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
17: vlan80@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.80.11/24 brd 10.56.80.255 scope global vlan80
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
18: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:98:b9:c1:30 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

此时便可配置openstack 与 ceph 见 >> | bond版opesnatck 。

扩展：

workstdtion物理环境配置bond1 （桥接模式模拟 实际是会断网的）实验时推荐使用下方nat模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33:
      addresses: ["10.0.0.10/24"]    # workstdtion nat网卡防止断连
    ens37: {}
    ens38: {}
  bonds:
    bond1:
      addresses: ["172.16.100.200/24"]  # 主备模式
      gateway4: 172.16.100.1
      interfaces:
        - ens37
        - ens38
      nameservers:
        addresses: [223.5.5.5]
      parameters:
        mode: active-backup
        primary: ens37 # 主备网卡
        mii-monitor-interval: 100
  version: 2
~

workstdtion物理环境配置bond1 nat模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33:
      addresses: ["10.0.0.10/24"]
    ens37: {}
    ens38: {}
  bonds:
    bond1:
      addresses: ["10.0.0.11/24"]  # 主备模式
      gateway4: 10.0.0.220
      interfaces:
        - ens37
        - ens38  
      nameservers:
        addresses: [223.5.5.5]        
      parameters:
        mode: active-backup
        primary: ens37 # 主备网卡
        mii-monitor-interval: 100
  version: 2

实际物理环境，物理服务器ubuntu配置bond 802.3ad如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# This is the network config written by 'subiquity'
network:
  ethernets:
     eno1: {}
     eno2: {}
     eno3: {}
     eno4:
        addresses: [58.49.220.166/24]
        gateway4: 58.49.220.1
  bonds:
    bond0:
      interfaces:  
        - eno3
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
  vlans:
    vlan60:
      id: 60
      link: bond0   # 即en03网卡即基于eno3网卡配置802.3ad
      addresses: ["192.168.30.100/24"]  
  version: 2

双网卡版本。

下方参考

https://blog.csdn.net/qq_50247813/article/details/143630081

环境ubuntu22.04

本次纯实验，不会讲解bond功能，配置bond mode 1 和 mode 4 如何配置

  1. 确定内核模块是否加载
    实验使用root用户权限，非root用户使用sudo 调用root权限
1
2
3
4
5
6
root@ubuntu22:~# lsmod | grep bonding
root@ubuntu22:~# modprobe bonding
root@ubuntu22:~# lsmod | grep bonding
bonding               200704  0
tls                   114688  1 bonding

实验网卡

ens37 ens38

bond mode4配置文件如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
network:
    version: 2
    renderer: networkd
    ethernets:
      ens37:
        addresses: []
        dhcp4: no
        optional: true
      ens38:
        addresses: []
        dhcp4: no
        optional: true
    bonds:
      bond4:
        interfaces: [ens37,ens38]
        parameters:
          mode: 802.3ad
          lacp-rate: fast
          mii-monitor-interval: 100
          transmit-hash-policy: layer3+4
        addresses: [192.168.119.130/24]
        routes:
          - to: 0.0.0.0/0
            via: 192.168.119.2
        nameservers:
          addresses: [114.114.114.114]

网络配置（network）

  • version: 2：指定Netplan配置文件的版本。
  • renderer: networkd：指定使用systemd-networkd作为网络服务的渲染器。

以太网接口（ethernets）

  • ens37 和 ens38：定义了两个以太网接口。
  • addresses: []：为空列表，表示这些接口没有静态分配的IP地址。
  • dhcp4: no：表示这些接口不使用DHCP来获取IPv4地址。
  • optional: true：表示这些接口是可选的，即使它们不存在或未配置成功，也不会影响系统的网络启动。

链路聚合接口（bonds）

  • bond4：定义了一个名为bond4的链路聚合接口。
  • interfaces: [ens37, ens38]：指定bond4由ens37和ens38两个接口组成。
  • parameters：定义了bond接口的参数。
  • mode: 802.3ad：设置bond模式为802.3ad，这是一种基于LACP（链路聚合控制协议）的动态链路聚合模式。
  • lacp-rate: fast：设置LACP的速率为fast，意味着LACP包会更快地发送，以便更快地建立和维护聚合。
  • mii-monitor-interval: 100：设置接口状态监测间隔为100毫秒，用于检测链路是否活动。
  • transmit-hash-policy: layer3+4：设置流量分配策略，基于层3（IP地址）和层4（端口号）信息来计算哈希值，以决定数据包通过哪个slave接口发送。
  • addresses: [192.168.119.130/24]：为bond4接口分配静态IPv4地址。
  • routes：
  • to: 0.0.0.0/0：定义默认路由。
  • via: 192.168.119.2：指定下一跳地址，即所有非本地目的地的流量都将通过这个地址转发。
  • nameservers：
  • addresses: [114.114.114.114]：指定DNS服务器地址，这里使用了公共DNS（114.114.114.114）。

bond mode1 配置文件如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
network:
    version: 2
    renderer: networkd
    ethernets:
      ens37:
        addresses: []
        dhcp4: no
        optional: true
      ens38:
        addresses: []
        dhcp4: no
        optional: true

    bonds:
      bond4:
        interfaces: [ens37,ens38]
        parameters:
          mode: active-backup
          mii-monitor-interval: 100
        addresses: [192.168.119.130/24]
        routes:
          - to: 0.0.0.0/0
            via: 192.168.119.2
        nameservers:
          addresses: [114.114.114.114]

  • mode: active-backup 主备模式
    使用 netplan apply 使配置生效