■vSphere(ESXi)6.5にVyOSを導入して複数のセグメントを追加する。
後述のポートグループ[VM Netwok]に属する仮想マシンの
Next-Hopのルーティング、DNS、NTPを許可する上位FWの定義が多くなってきた。
L2間通信はVLANで分離しているし、グループ化も一つの手段ではあるけれど、
L3境界ルータとしてVyOSを構成して、そもそもESXiの内部から出ないL2と出ても良いL3に分離するよう構成する。
Product: VMware ESXi
Version: 6.5.0
Build: Releasebuild-5310538
Update: 0
Patch: 19
■下記のコマンドは変わらず使える。
vyOS1.0.4をvmplayer、ESXiの仮想マシンに導入する。
http://labunix.hateblo.jp/?page=1406821520
■VyOSをダウンロード。
導入方法にはiso方式とova方式がある。
https://vyos.io/
User Guide
https://wiki.vyos.net/wiki/User_Guide
$ md5sum vyos-1.1.7-amd64*
ae2d610dc7123e2dda75aa9df2b2946e vyos-1.1.7-amd64-signed.ova
9a7f745a0b0db0d4f1d9eee2a437fb54 vyos-1.1.7-amd64.iso
■ovftoolを導入(入っていない場合)
VMware Open Virtualization Format Tool 3.5.0
VMware OVF Tool for Linux 64-bit
$ md5sum < VMware-ovftool-3.5.0-1274719-lin.x86_64.bundle
5f38d0025f79dac8fc55554c5b620477 -
$ chmod +x VMware-ovftool-3.5.0-1274719-lin.x86_64.bundle
$ sudo ./VMware-ovftool-3.5.0-1274719-lin.x86_64.bundle
■ovaのデフォルトの最小構成は1Core、1GBメモリ、4GBディスク、vmnet3のNIC1枚(DHCP/IPv4)
$ ovftool vyos-1.1.7-amd64-signed.ova | \
awk 'BEGIN {f=0}{if($1=="Virtual")f=1}{if(f==1){print $0}}'
Virtual Machines:
Name: VyOS-1.1.7
Operating System: otherlinux64guest
Virtual Hardware:
Families: vmx-09
Number of CPUs: 1
Cores per socket: 1
Memory: 1024.00 MB
Disks:
Index: 0
Instance ID: 9
Capacity: 4.00 GB
Disk Types: SCSI-lsilogic
NICs:
Adapter Type: VmxNet3
Connection: management.gs.es
IP Allocation Policy:
Schemes: dhcp
Protocols: IPv4
■ホストのNW構成の確認
ここで既にセグメント分離はされていて、ホストは直接外部と通信出来ない。
Interface,Port,Group/DVPort/Opaque,Network,IP,Family,IP,Address,Netmask,Broadcast,MAC,Address,MTU,TSO,MSS,Enabled,Type,NetStack,
vmk0,Management,Network,IPv4,XX.XX.XX.66,XX.XX.XX.0,XX.XX.XX.255,00:XX:XX:XX:XX:XX,1500,65535,true,STATIC,defaultTcpipStack,
vmk1,VMkernel,IPv4,XX.XX.XX.241,XX.XX.XX.240,XX.XX.XX.255,00:XX:XX:XX:XX:XX,1500,65535,true,STATIC,defaultTcpipStack,
Name PCI Driver Link Speed Duplex MAC Address MTU Description,
vmnic0,0000:XX:XX.0,e1000e Up,1000Mbps Full,00:XX:XX:XX:XX:XX,1500,Intel Corporation 82567LM-3 Gigabit Network Connection
vmnic1,0000:XX:XX.0,r8169 Up,10Mbps Half,34:XX:XX:XX:XX:XX,1500,Realtek Realtek 8169 Gigabit Ethernet
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 1536 5 128 1500 vmnic0
PortGroup Name VLAN ID Used Ports Uplinks
VM Network 0 1 vmnic0
Management Network 0 1 vmnic0
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 1536 5 128 1500 vmnic1
PortGroup Name VLAN ID Used Ports Uplinks
VLAN140 140 0 vmnic1
VLAN110 110 0 vmnic1
VLAN100 100 0 vmnic1
VLAN130 130 0 vmnic1
VLAN120 120 0 vmnic1
Trunk-Any 4095 1 vmnic1
VMkernel 0 1 vmnic1
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch2 1536 1 128 1500
PortGroup Name VLAN ID Used Ports Uplinks
VLAN015 15 0
VLAN014 14 0
VLAN013 13 0
VLAN012 12 0
VLAN011 11 0
VLAN010 10 0
■以下の3種類のネットワークがある。[vSwitch0]と[vSwitch2]の間のGWとなるように構成する。
物理NIC、仮想スイッチ、ポートグループの順。
※Web管理GUIのvSwitch トポロジでは、ポートグループ、仮想スイッチ、物理NICの順。
$ echo "[vmnic0] -- [vSwitch0] -- [Management Network(vmk0)],[VM Network]" | graph-easy --dot
[vmnic0] -- [vSwitch0] -- [Management Network(vmk0)],[VM Network]
$ echo "[vmnic1] -- [vSwitch1] -- [VMKernel(vmk1)],[Trunk-Any],[VLAN100],[VLAN110],[VLAN120],[VLAN130],[VLAN140]"
[vmnic1] -- [vSwitch1] -- [VMKernel(vmk1)],[Trunk-Any],[VLAN100],[VLAN110],[VLAN120],[VLAN130],[VLAN140]
$ echo "[none] -- [vSwitch2] -- [VLAN10],[VLAN11],[VLAN12],[VLAN13],[VLAN14],[VLAN15]"
[none] -- [vSwitch2] -- [VLAN10],[VLAN11],[VLAN12],[VLAN13],[VLAN14],[VLAN15]
■[VM Network]と[VLAN10]をアサインして、OVAをデプロイする。
ユーザ名、パスワードのデフォルトは「vyos/vyos」。
■ポートグループ[VLAN10]を[eth1]に割り当て、[172.31.10.0/24]としてsshサービスを起動する。
ポートグループ[VM Network]を[eth0]に割り当て、[172.31.31.0/24]としてデフォルトGWを定義。
ポートグループ[VLAN11]を[eth2]に割り当て、ルーティングなし、DNSを引かせず、隔離する。
Next-Hopのルーティング、DNS、NTPの設定と共に、上位FWでその通信を許可する。
※この環境では[vSwitch0]と[vSwitch2]の間をルーティングできても、
ソースIPを上位FWに定義していないので、(VyOS自身以外の)外部通信は出来ない。
プロキシサーバに[eth2(L2-Only)]か[eth1(L3-Route)]に接続して代理でWeb通信させるか、
NAT構成のインターフェイスを作成する必要があるが、今回はセグメントの分離が目的なので行わない。
set interfaces ethernet eth1 address 172.31.10.254/24
set interfaces ethernet eth1 description L3-Route
set service ssh port 22
commit
set interfaces ethernet eth0 address 172.31.31.10/24
set interfaces ethernet eth0 description Wan
set protocols static route 0.0.0.0/0 next-hop 172.31.31.252 distance '1'
set service dns forwarding name-server 172.31.31.251
set service dns forwarding cache-size 0
set service dns forwarding listen-on eth1
set system time-zone Asia/Tokyo
set system ntp server 172.31.31.252
delete system ntp server 0.pool.ntp.org
delete system ntp server 1.pool.ntp.org
delete system ntp server 2.pool.ntp.org
commit
save
$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route
S>* 0.0.0.0/0 [1/0] via 172.31.31.252, eth0
C>* 127.0.0.0/8 is directly connected, lo
C>* 172.31.10.0/24 is directly connected, eth1
C>* 172.31.31.0/24 is directly connected, eth0
$ show dns forwarding nameservers
-----------------------------------------------
Nameservers configured for DNS forwarding
-----------------------------------------------
172.31.31.251 available via 'statically configured'
$ show ntp
remote local st poll reach delay offset disp
=======================================================================
*172.31.31.252 172.31.31.10 2 64 1 0.00038 0.064986 0.45050
set interfaces ethernet eth2 address 172.31.11.254/24
set interfaces ethernet eth2 description L2-Only
commit
save
$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route
S>* 0.0.0.0/0 [1/0] via 172.31.31.252, eth0
C>* 127.0.0.0/8 is directly connected, lo
C>* 172.31.10.0/24 is directly connected, eth1
C>* 172.31.11.0/24 is directly connected, eth2
C>* 172.31.31.0/24 is directly connected, eth0
■「sudo vi」コマンドでVyOS自身のプロキシ通信設定を行う。
「packages.vyos.net」にはパッケージがないので「community」を無効にするべきだが、
プロキシを通ることを確認するだけなので、特に変更しない。
$ cat /etc/debian_version
6.0.10
$ dpkg -l | awk '/apt-utils|image/'
ii apt-utils 0.8.10.3+squeeze7 APT utility programs
ii linux-image-3.13.11-1-amd64-vyos 3.13.11-1+vyos1+helium11 Linux 3.13.11 image on amd64
VyOSにdebianのパッケージを入れる
https://exempla.jp/archives/33
set system package repository squeeze components 'main contrib non-free'
set system package repository squeeze distribution 'squeeze'
set system package repository squeeze url 'http://archive.debian.org/debian'
commit
save
$ cat /etc/apt/apt.conf
Acquire::http::Proxy "http://172.31.31.60:8080/";
APT::Periodic::Update-Package-Lists 1;
$ sudo apt-get update
Get:1 http://archive.debian.org squeeze Release.gpg [1655 B]
Ign http://packages.vyos.net helium Release.gpg
Ign http://archive.debian.org/debian/ squeeze/contrib Translation-en
Ign http://packages.vyos.net/vyos/ helium/main Translation-en
Ign http://archive.debian.org/debian/ squeeze/main Translation-en
Ign http://packages.vyos.net helium Release
Ign http://archive.debian.org/debian/ squeeze/non-free Translation-en
Err http://packages.vyos.net helium/main amd64 Packages
404 Not Found
Get:2 http://archive.debian.org squeeze Release [96.0 kB]
Get:3 http://archive.debian.org squeeze/main amd64 Packages [8370 kB]
Get:4 http://archive.debian.org squeeze/contrib amd64 Packages [61.7 kB]
Get:5 http://archive.debian.org squeeze/non-free amd64 Packages [121 kB]
Fetched 8651 kB in 12min 49s (11.2 kB/s)
W: Failed to fetch http://packages.vyos.net/vyos/dists/helium/main/binary-amd64/Packages.gz 404 Not Found
E: Some index files failed to download, they have been ignored, or old ones used instead.
$ cat /etc/profile.d/proxy.sh
export http_proxy=http://172.31.31.60:8080/
export http_proxys=http://172.31.31.60:8080/
export ftp_proxy=http://172.31.31.60:8080/
export HTTP_PROXY=http://172.31.31.60:8080/
export HTTPS_PROXY=http://172.31.31.60:8080/
export FTP_PROXY=http://172.31.31.60:8080/
export no_proxy=""
$ curl http://google.co.jp
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.co.jp/">here</A>.
</BODY></HTML>
■vSphereホストに「net51-r8169」と「net51-drivers」を入れ替える方法のおまけ
全くの脇道に逸れるが、VMKernelと3桁VLANのポートグループは半二重10Mbpsしか使えていない。
セグメント分離に失敗した場合の保険として搭載していたけれど、
実際全く使っていないので気にしてなかった。
LANケーブルを替えても、「net51-r8169」から「net51-drivers」に入れ替えても、
対向と合わせてautoから10MFullに固定しても状況は変わらなかったので放置する。
0000:00:19.0 Ethernet controller Network controller: Intel Corporation 82567LM-3 Gigabit Network Connection [vmnic0]
0000:10:00.0 Ethernet controller Network controller: Realtek Realtek 8169 Gigabit Ethernet [vmnic1]
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:XX:XX.0 e1000e Up 1000Mbps Full 00:XX:XX:XX:XX:XX 1500 Intel Corporation 82567LM-3 Gigabit Network Connection
vmnic1 0000:XX:XX.0 r8169 Up 10Mbps Half 34:XX:XX:XX:XX:XX 1500 Realtek Realtek 8169 Gigabit Ethernet
net51-r8169 6.011.00-2vft.510.0.0.799733 VFrontDe CommunitySupported 2015-11-28
Realtek NIC on vSphere 6
http://www.vdicloud.nl/2015/02/07/realtek-nic-on-vsphere-6/
VMWare hypervisor無償版にssh経由でNICドライバを追加する。
http://labunix.hateblo.jp/entry/20151113/1447340548
Disabled
Enabled
Removal Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed:
VIBs Removed: VFrontDe_bootbank_net51-r8169_6.011.00-2vft.510.0.0.799733
VIBs Skipped:
Enabled
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: VFrontDe_bootbank_net51-drivers_1.0.0-1vft.510.0.0.799733
VIBs Removed:
VIBs Skipped:
Enabled
net51-drivers 1.0.0-1vft.510.0.0.799733 VFrontDe CommunitySupported 2017-10-23
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:XX:XX.0 e1000e Up 1000Mbps Full 00:XX:XX:XX:XX:XX 1500 Intel Corporation 82567LM-3 Gigabit Network Connection
vmnic1 0000:XX:XX.0 r8169 Up 10Mbps Half 34:XX:XX:XX:XX:XX 1500 Realtek Realtek 8169 Gigabit Ethernet
Disabled
0000:10:00.0 Ethernet controller Network controller: Realtek Realtek 8169 Gigabit Ethernet [vmnic1]
0000:10:00.0 10ec:8169 10ec:8169 5/ 5/0x30 A V r8169 vmnic1
0000:10:00.0 Network controller: Realtek Realtek 8169 Gigabit Ethernet [vmnic1]
00: ec 10 69 81 17 01 b8 02 10 00 00 02 08 40 00 00
10: 01 11 00 00 00 00 60 f0 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 ec 10 69 81
30: 00 00 00 00 dc 00 00 00 00 00 00 00 05 01 20 40
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 c2 f7
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
grep -v "ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff"
0000:10:00.0 Network controller: Realtek Realtek 8169 Gigabit Ethernet [vmnic1]
00: ec 10 69 81 17 01 b8 02 10 00 00 02 08 40 00 00
10: 01 11 00 00 00 00 60 f0 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 ec 10 69 81
30: 00 00 00 00 dc 00 00 00 00 00 00 00 05 01 20 40
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 c2 f7
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
VMB: 323: name: /net51_dr.t00
2017-10-23T05:20:37.611Z cpu0:65536)VisorFSTar: 1954: net51_dr.t00 for 0x8c600 bytes
2017-10-23T05:20:47.416Z cpu1:65976)Loading module r8169 ...
2017-10-23T05:20:47.417Z cpu1:65976)Elf: 2043: module r8169 has license GPL
2017-10-23T05:20:47.417Z cpu1:65976)skb_mem_info mempool for module r8169 created - max size 23068672
2017-10-23T05:20:47.417Z cpu1:65976)module heap vmklnx_r8169: creation succeeded. id = 0x430407151000
2017-10-23T05:20:47.418Z cpu1:65976)PCI: driver r8169 is looking for devices
2017-10-23T05:20:47.418Z cpu1:65976)<6>r8169 Gigabit Ethernet driver 6.011.00-NAPI loaded
2017-10-23T05:20:47.418Z cpu1:65976)<6>r8169: This product is covered by one or more of the following patents: US5,307,459, US5,434,872, US5,732,094, US6,570,884, US6,115,776, and US6,327,625.
2017-10-23T05:20:47.420Z cpu0:65976)PCI: driver r8169 claimed device 0000:10:00.0
2017-10-23T05:20:47.420Z cpu0:65976)PCI: driver r8169 claimed 1 device
2017-10-23T05:20:47.421Z cpu1:65585)<6>r8169: vmnic1: link down
2017-10-23T05:20:47.422Z cpu0:65976)Mod: 4968: Initialization of r8169 succeeded with module ID 4128.
2017-10-23T05:20:47.422Z cpu0:65976)r8169 loaded successfully.
2017-10-23T05:20:49.553Z cpu0:65686)<6>r8169: vmnic1: link up
2017-10-23T05:23:29.159Z cpu0:66013)<6>r8169: vmnic1: link down
2017-10-23T05:24:04.658Z cpu0:65686)<6>r8169: vmnic1: link up
2017-10-23T05:30:17.126Z cpu1:67523 opID=539fd7dc)<6>r8169: vmnic1: link down
2017-10-23T05:30:19.284Z cpu1:65686)<6>r8169: vmnic1: link up
2017-10-23T05:30:33.315Z cpu1:67544 opID=e2b8f5ab)<6>r8169: vmnic1: link down
2017-10-23T05:30:35.474Z cpu1:65686)<6>r8169: vmnic1: link up
2017-10-23T05:30:40.866Z cpu1:67526 opID=6f2760b5)<6>r8169: vmnic1: link up
2017-10-23T05:30:40.866Z cpu1:67526 opID=6f2760b5)<6>r8169: vmnic1: link up
2017-10-23T05:30:49.514Z cpu0:67540 opID=4e9356c)<6>r8169: vmnic1: link down
2017-10-23T05:30:51.663Z cpu0:66026)<6>r8169: vmnic1: link up
