workstdtion模拟交换机划分vlan,以及物理服务器网卡bond设置

workstdtion设置

https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_c2444bc85394584e2798df97ef8b9549.png

一. 第一种ubuntu系统网卡配置

查看端口速率：

  • 判断是千兆网卡还是百兆网卡 查看支持的速率 > ethtool ens33

    1
    2
    3
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair

完整网卡配置如下：共接四根网线 bond0>>bond3 处网卡需要是up状态 物理服务器时只需要插入网线查看端口状态即可判断网口状态

  • bond3 浮动IP 192.168.1.10/24 网口上 位于千兆口 接一根交换机上划分的一根同网段网线 用于远程连接

  • bond0 万兆口接一根交换机划分的 50 段网络 用于ceph osd 内部通信ceph（–cluster-network）网络

  • bond1 上面接交换机上对 该网口划分了vlan60 vlan70 vlan80 vlan90 的网线

  • bond2 上面接交换机上对 192.168.1.10/24 网段划分的浮动IP 池 的网线 例如：本文则需要为桥接模式模拟浮动IP 并且此网口不配置IP（下方文件测试已配置了IP 实际过程中自行注释）实验中单节点vmware环境bond2下网卡ens39桥接模式 部署单节点stack一切正常 但是双节点bond2下网卡ens39 双节点ens39网卡均配置桥接模式时 只有一个节点可用 >> 此时 只需要将双节点上方均没有划分bond的任意网卡桥接如ens41 修改global文件用于浮动IP池使用的网卡为ens41 再重新部署 即 正常。

    https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_6e28aee0f289c98393d974c614fc09c3.png

  • 三台服务器 此种配置完成后 是可以互通的无论ping那个网段都是通的
    ubuntu网卡bond接线与ceph及start集成揽图

    https://github.com/zznn-cloud/zznn-cloud-blog-images/raw/main/Qexo/24/4/image_cea9571e22249bca02d6cb339ed3c060.png

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33: {}
    ens37: {}
    ens38: {}
    ens39: {}
    ens40: {}
    ens41: {}
    ens42: {}
    ens43: {}
  bonds:
    bond0:
      addresses: ["10.56.50.11/24"]  # 此网口用万兆口 用于ceph osd间通信（--cluster-network）
      interfaces:
        - ens37
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond1:
      interfaces:  
        - ens38
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond2:
      interfaces:
        - ens39
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
    bond3:
      addresses: ["192.168.1.10/24"]      # 此网口用千兆口 >> 电口 用于远程连接 及浮动IP使用
      gateway4: 192.168.1.1
      interfaces:
        - ens33
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
  vlans:
    vlan60:
      id: 60
      link: bond1
      addresses: ["10.56.60.11/24"]
    vlan70:
      id: 70
      link: bond1
      addresses: ["10.56.70.11/24"]
    vlan80:
      id: 80
      link: bond1
      addresses: ["10.56.80.11/24"]
    vlan90:
      id: 90
      link: bond1
      addresses: ["10.56.90.11/24"]
  version: 2

效果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
  System information as of Wed 03 Apr 2024 04:59:36 AM UTC

  System load:              0.02
  Usage of /:               26.9% of 39.07GB
  Memory usage:             11%
  Swap usage:               0%
  Processes:                261
  Users logged in:          1
  IPv4 address for bond0:   10.56.50.11
  IPv4 address for bond3:   192.168.1.10
  IPv6 address for bond3:   240e:36f:151f:a000:e407:31ff:fee2:70f9
  IPv4 address for docker0: 172.17.0.1
  IPv4 address for vlan60:  10.56.60.11
  IPv4 address for vlan70:  10.56.70.11
  IPv4 address for vlan80:  10.56.80.11
  IPv4 address for vlan90:  10.56.90.11
#-----------------------------------------------------------------
root@ceph1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond3 state UP group default qlen 1000
    link/ether e6:07:31:e2:70:f9 brd ff:ff:ff:ff:ff:ff
3: ens37: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 5a:c0:b2:8b:fe:e7 brd ff:ff:ff:ff:ff:ff
4: ens38: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond1 state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
5: ens39: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond2 state UP group default qlen 1000
    link/ether 76:88:82:b8:46:ba brd ff:ff:ff:ff:ff:ff
6: ens40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:b0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ab0/64 scope link 
       valid_lft forever preferred_lft forever
7: ens41: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:ba brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8aba/64 scope link 
       valid_lft forever preferred_lft forever
8: ens42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:c4 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ac4/64 scope link 
       valid_lft forever preferred_lft forever
9: ens43: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:45:8a:ce brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe45:8ace/64 scope link 
       valid_lft forever preferred_lft forever
10: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5a:c0:b2:8b:fe:e7 brd ff:ff:ff:ff:ff:ff
    inet 10.56.50.11/24 brd 10.56.50.255 scope global bond0
       valid_lft forever preferred_lft forever
    inet6 fe80::58c0:b2ff:fe8b:fee7/64 scope link 
       valid_lft forever preferred_lft forever
11: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
12: bond2: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 76:88:82:b8:46:ba brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7488:82ff:feb8:46ba/64 scope link 
       valid_lft forever preferred_lft forever
13: bond3: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e6:07:31:e2:70:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global bond3
       valid_lft forever preferred_lft forever
    inet6 240e:36f:151f:a000:e407:31ff:fee2:70f9/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 198193sec preferred_lft 111793sec
    inet6 fe80::e407:31ff:fee2:70f9/64 scope link 
       valid_lft forever preferred_lft forever
14: vlan90@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.90.11/24 brd 10.56.90.255 scope global vlan90
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
15: vlan60@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.60.11/24 brd 10.56.60.255 scope global vlan60
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
16: vlan70@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.70.11/24 brd 10.56.70.255 scope global vlan70
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
17: vlan80@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:ce:28:a9:28:0d brd ff:ff:ff:ff:ff:ff
    inet 10.56.80.11/24 brd 10.56.80.255 scope global vlan80
       valid_lft forever preferred_lft forever
    inet6 fe80::e0ce:28ff:fea9:280d/64 scope link 
       valid_lft forever preferred_lft forever
18: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:98:b9:c1:30 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

此时便可配置openstack 与 ceph 见 >> | bond版opesnatck 。

扩展：

实际物理环境，物理服务器ubuntu配置bond 802.3ad如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# This is the network config written by 'subiquity'
network:
  ethernets:
     eno1: {}
     eno2: {}
     eno3: {}
     eno4:
        addresses: [58.49.220.166/24]
        gateway4: 58.49.220.1
  bonds:
    bond0:
      interfaces:  
        - eno3
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        lacp-rate: fast
        transmit-hash-policy: layer3+4
  vlans:
    vlan60:
      id: 60
      link: bond0   # 即en03网卡即基于eno3网卡配置802.3ad
      addresses: ["192.168.30.100/24"]  
  version: 2

双网卡版本。