服务器评测
今天看到朋友推荐的一篇IBM的文章 http://www.linuxidc.com/Linux/2017-06/144770.htm
文章内容很好，如果我只是收藏起来，那就实在是太浪费，还是动手练习一次，好好补一下我的网络。还是用UnitedStack的UOS，创建一个虚拟机来完成全部的试验。
文章就一个小笔误，把ns2写成ns3。外面很多转载的文章，都没注明出处，搞的以为是原创。如果错误都一样，就有点不好意思。
和IBM的文档不一样的地方是：我使用的是Ubuntu 14.04，减少很多没必要的麻烦。
创建一个Ubuntu 14.04的最小配置虚拟机，ssh到虚拟机上。
查看Ubuntu版本
# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 14.04.1 LTS Release: 14.04 Codename: trusty
OVS
Ubuntu 14.04的OVS版本，已经是2.02，所以默认安装就可以。不过不同的发行版，ovs的名字会有点不同。
apt-cache search openvswitch
开始安装
apt-get install openvswitch-switch
查看OVS运行情况
# ps -ea | grep ovs 3007 ? 00:00:00 ovsdb-server 3017 ? 00:00:00 ovs-vswitchd
查看OVS版本
# ovs-appctl --version ovs-appctl (Open vSwitch) 2.0.2 Compiled Aug 15 2014 14:31:01
查看 OVS 支持的 OpenFlow 协议的版本
# ovs-ofctl --version ovs-ofctl (Open vSwitch) 2.0.2 Compiled Aug 15 2014 14:31:02 OpenFlow versions 0x1:0x4
OpenFlow 命令
创建一个OVS交换机
ovs-vsctl add-br ovs-switch
创建一个端口 p0，设置端口 p0 的 OpenFlow 端口编号为 100
ovs-vsctl add-port ovs-switch p0 -- set Interface p0 ofport_request=100
设置网络接口设备类型为”internal”,
ovs-vsctl set Interface p0 type=internal
查看设置结果
# ethtool -i p0 driver: openvswitch version: firmware-version: bus-info: supports-statistics: no supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no
创建一个name space：ns0，把p0端口接入到ns0里，并且配置ip地址 192.168.1.100/24
ip netns add ns0 ip link set p0 netns ns0 ip netns exec ns0 ip addr add 192.168.1.100/24 dev p0 ip netns exec ns0 ifconfig p0 promisc up
查看创建结果
# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
Bridge ovs-switch
Port "p0"
Interface "p0"
type: internal
Port ovs-switch
Interface ovs-switch
type: internal
ovs_version: "2.0.2"
重复步骤，创建p1和p2端口
ovs-vsctl add-port ovs-switch p1 -- set Interface p1 ofport_request=101 ovs-vsctl set Interface p1 type=internal ip netns add ns1 ip link set p1 netns ns1 ip netns exec ns1 ip addr add 192.168.1.101/24 dev p1 ip netns exec ns1 ifconfig p1 promisc up
查看创建结果
# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
Bridge ovs-switch
Port "p1"
Interface "p1"
type: internal
Port "p0"
Interface "p0"
type: internal
Port ovs-switch
Interface ovs-switch
type: internal
ovs_version: "2.0.2"
创建p2
ovs-vsctl add-port ovs-switch p2 -- set Interface p2 ofport_request=102 ovs-vsctl set Interface p2 type=internal ip netns add ns2 ip link set p2 netns ns2 ip netns exec ns2 ip addr add 192.168.1.102/24 dev p2 ip netns exec ns2 ifconfig p2 promisc up
查看
# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
Bridge ovs-switch
Port "p1"
Interface "p1"
type: internal
Port "p2"
Interface "p2"
type: internal
Port "p0"
Interface "p0"
type: internal
Port ovs-switch
Interface ovs-switch
type: internal
ovs_version: "2.0.2"
查看创建的交换机信息，获得dpid，端口openflow端口编号
# ovs-ofctl show ovs-switch OFPT_FEATURES_REPLY (xid=0x2): dpid:0000d23b94ce4146 n_tables:254, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_ DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 100(p0): addr:00:00:00:00:00:00 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max 101(p1): addr:00:00:00:00:00:00 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max 102(p2): addr:00:00:00:00:00:00 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max LOCAL(ovs-switch): addr:2a:be:0c:72:40:45 config: PORT_DOWN state: LINK_DOWN speed: 0 Mbps now, 0 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
获取openflow端口编号
# ovs-vsctl get Interface p0 ofport 100 # ovs-vsctl get Interface p1 ofport 101 # ovs-vsctl get Interface p2 ofport 102
查看 datapath 的信息
# ovs-dpctl show
system@ovs-system:
lookups: hit:34 missed:21 lost:0
flows: 0
port 0: ovs-system (internal)
port 1: ovs-switch (internal)
port 2: p0 (internal)
port 3: p1 (internal)
port 4: p2 (internal)
查看mac地址
ip netns exec ns0 ping 192.168.1.100 ip netns exec ns0 ping 192.168.1.101 ip netns exec ns0 ping 192.168.1.102
然后运行
# ovs-appctl fdb/show ovs-switch
port VLAN MAC Age
102 0 22:8e:52:36:92:25 17
100 0 d6:0f:7e:ed:11:e4 4
101 0 f2:0d:06:ff:79:d7 4
查看交换机所有table
ovs-ofctl dump-tables ovs-switch
查看交换机中的所有流表项
ovs−ofctl dump−flows ovs-switch
删除编号为 100 的端口上的所有流表项
ovs-ofctl del-flows ovs-switch "in_port=100"
查看交换机端口信息
ovs-ofctl show ovs-switch
修改数据包
屏蔽所有进入 OVS 的以太网广播数据包
$ ovs-ofctl add-flow ovs-switch "table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop"
屏蔽 STP 协议的广播数据包
$ ovs-ofctl add-flow ovs-switch "table=0, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop"
修改数据包，添加新的 OpenFlow 条目，修改从端口 p0 收到的数据包的源地址为 9.181.137.1
ovs-ofctl add-flow ovs-switch "priority=1 idle_timeout=0,\ in_port=100,actions=mod_nw_src:9.181.137.1,normal"
从端口 p0（192.168.1.100）发送测试数据到端口 p1（192.168.1.101），就是没啥响应
# ip netns exec ns0 ping 192.168.1.101 PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data.
再打开一个ssh终端，登录进去，运行tcpdump，需要等待几分钟，才能看到响应
~# ip netns exec ns1 tcpdump -i p1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p1, link-type EN10MB (Ethernet), capture size 65535 bytes
06:21:23.802308 IP 9.181.137.1 > 192.168.1.101: ICMP echo request, id 4533, seq 19, length 64
06:21:24.802358 IP 9.181.137.1 > 192.168.1.101: ICMP echo request, id 4533, seq 20, length 64
重定向数据包
添加新的 OpenFlow 条目，重定向所有的 ICMP 数据包到端口 p2
ovs-ofctl add-flow ovs-switch idle_timeout=0,dl_type=0x0800,nw_proto=1,actions=output:102
从端口 p0 （192.168.1.100）发送数据到端口 p1（192.168.1.101）
ip netns exec ns0 ping 192.168.1.101
这个时候你从p2里，可以看到
# ip netns exec ns2 tcpdump -i p2 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p2, link-type EN10MB (Ethernet), capture size 65535 bytes
06:25:38.252471 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 35, length 64
06:25:39.260438 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 36, length 64
06:25:40.268419 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 37, length 64
修改vlan tag
修改端口 p1 的 VLAN tag 为 101，使端口 p1 成为一个隶属于 VLAN 101 的端口
ovs-vsctl set Port p1 tag=101
现在由于端口 p0 和 p1 属于不同的 VLAN，它们之间无法进行数据交换。我们使用 ovs-appctl ofproto/trace 生成一个从端口 p0 发送到端口 p1 的数据包，这个数据包不包含任何 VLAN tag，并观察 OVS 的处理过程
ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,\
dl_dst=f2:0d:06:ff:79:d7 -generate
注意：上面��一个mac地址，是p0的，第二个mac地址是p1的，你需要替换，上面有获取mac地址的方法。
# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,\ > dl_dst=f2:0d:06:ff:79:d7 -generate Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000 Rule: table=0 cookie=0 priority=1,in_port=100 OpenFlow actions=mod_nw_src:9.181.137.1,NORMAL no learned MAC for destination, flooding Final flow: unchanged Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=d6:0f:7e:ed:11:e4, dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_src=0.0.0.0,nw_proto=0,nw_frag=no Datapath actions: 1,4
在第一行输出中，“Flow:”之后的字段描述了输入的流的信息。由于我们没有指定太多信息，所以多数字段 （例如 dl_type 和 vlan_tci）被 OVS 设置为空值。
在第二行的输出中，“Rule:” 之后的字段描述了匹配成功的流表项。
在第三行的输出中，“OpenFlow actions”之后的字段描述了实际执行的操作。
最后一段以”Final flow”开始的字段是整个处理过程的总结，“Datapath actions: 4,1”代表数据包被发送到 datapath 的 4 和 1 号端口。
创建一条新的 Flow
对于从端口 p0 进入交换机的数据包，如果它不包含任何 VLAN tag，则自动为它添加 VLAN tag 101
ovs-ofctl add-flow ovs-switch "priority=3,in_port=100,dl_vlan=0xffff,\ actions=mod_vlan_vid:101,normal"
再次尝试从端口 p0 发送一个不包含任何 VLAN tag 的数据包，发现数据包进入端口 p0 之后, 会被加上 VLAN tag101, 同时转发到端口 p1 上
# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,\ > dl_dst=f2:0d:06:ff:79:d7 -generate Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000 Rule: table=0 cookie=0 priority=1,in_port=100 OpenFlow actions=mod_nw_src:9.181.137.1,NORMAL no learned MAC for destination, flooding Final flow: unchanged Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_src=0.0.0.0,nw_proto=0,nw_frag=no Datapath actions: 1,4 root@ovs:~# ovs-ofctl add-flow ovs-switch "priority=3,in_port=100,dl_vlan=0xffff,\ > actions=mod_vlan_vid:101,normal" root@ovs:~# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,\ > dl_dst=f2:0d:06:ff:79:d7 -generate Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000 Rule: table=0 cookie=0 priority=3,in_port=100,vlan_tci=0x0000 OpenFlow actions=mod_vlan_vid:101,NORMAL no learned MAC for destination, flooding Final flow: metadata=0,in_port=100,dl_vlan=101,dl_vlan_pcp=0,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000 Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_proto=0,nw_frag=no Datapath actions: push_vlan(vid=101,pcp=0),1,pop_vlan,3,push_vlan(vid=101,pcp=0),4
反过来从端口 p1 发送数据包，由于 p1 现在是带有 VLAN tag 101 的 Access 类型的端口，所以数据包进入端口 p1 之后，会被 OVS 添加 VLAN tag 101 并发送到端口 p0
# ovs-appctl ofproto/trace ovs-switch in_port=101,dl_src=f2:0d:06:ff:79:d7,\ > dl_dst=d6:0f:7e:ed:11:e4 -generate Flow: metadata=0,in_port=101,vlan_tci=0x0000,dl_src=f2:0d:06:ff:79:d7,dl_dst=d6:0f:7e:ed:11:e4,dl_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=NORMAL no learned MAC for destination, flooding Final flow: unchanged Relevant fields: skb_priority=0,in_port=101,vlan_tci=0x0000,dl_src=f2:0d:06:ff:79:d7,dl_dst=d6:0f:7e:ed:11:e4,dl_type=0x0000,nw_proto=0,nw_frag=no Datapath actions: push_vlan(vid=101,pcp=0),1,2,4
Floodlight
新创建一个ubuntu 14.04的虚拟机。
apt-get update apt-get install git apt-get install ant apt-get install openjdk-7-jdk
源码安装
git clone git://github.com/floodlight/floodlight.git
cd floodlight/
ant
java -jar target/floodlight.jar
这个时候floodlight就启动起来，最后一条命令，就是启动floodlight。
登录OVS节点
设置ovs的控制器为floodlight，10.250.3.10，就是floodlight虚拟机的IP。
ovs-vsctl set-controller ovs-switch tcp:10.250.3.10:6633
设置 OVS 的连接模式为 secure 模式
ovs-vsctl set Bridge ovs-switch fail-mode=secure
查看
# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
Bridge ovs-switch
Controller "tcp:10.250.3.10:6633"
is_connected: true
fail_mode: secure
Port "p1"
tag: 101
Interface "p1"
type: internal
Port "p2"
Interface "p2"
type: internal
Port "p0"
Interface "p0"
type: internal
Port ovs-switch
Interface ovs-switch
type: internal
ovs_version: "2.0.2"
通过访问 Floodlight 提供的 Web 管理界面 http://<Host Address>:8080/ui/index.html，我们可以查看 Floodlight 控制器的状态以及所有连接到 Floodlight 的交换机列表
通过 Floodlight 的 RESTAPI，添加两条新的规则让端口 p0 和 p1 可以相互通讯。注意：替换命令行中的 switch 的 ID 为交换机的 datapath ID
注意curl命令，尽量别用 / 换行
curl -d '{"switch": "00:00:d2:3b:94:ce:41:46", "name":"my-flow1", "cookie":"0","priority":"32768","ingress-port":"100","active":"true", "actions":"output=flood"}' http://10.250.3.10:8080/wm/staticflowentrypusher/json curl -d '{"switch": "00:00:d2:3b:94:ce:41:46", "name":"my-flow2", "cookie":"0","priority":"32768","ingress-port":"101","active":"true", "actions":"output=flood"}' http://10.250.3.10:8080/wm/staticflowentrypusher/json
验证是否能从端口 p0 发送数据包到 p1
# ip netns exec ns0 ping -c4 192.168.1.101 PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data. 64 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=0.625 ms 64 bytes from 192.168.1.101: icmp_seq=2 ttl=64 time=0.088 ms 64 bytes from 192.168.1.101: icmp_seq=3 ttl=64 time=0.082 ms 64 bytes from 192.168.1.101: icmp_seq=4 ttl=64 time=0.048 ms
在 OVS 端也可以看到，流表规则已经被 OVS 同步到本地。
# ovs-ofctl dump-flows ovs-switch
NXST_FLOW reply (xid=0x4):
cookie=0xa00000626d6af5, duration=111.468s, table=0, n_packets=7, n_bytes=630, idle_age=2, in_port=100 actions=FLOOD
cookie=0xa00000626d6af6, duration=83.717s, table=0, n_packets=7, n_bytes=630, idle_age=1, in_port=101 actions=FLOOD
通过 Floodlight 的 RestAPI，查看交换机上的流表规则
curl http://10.250.3.10:8080/wm/staticflowentrypusher/list/00:00:d2:3b:94:ce:41:46/json | python -mjson.tool
采用python的输出，好看很多的
# curl http://10.250.3.10:8080/wm/staticflowentrypusher/list/00:00:d2:3b:94:ce:41:46/json | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1435 0 1435 0 0 109k 0 --:--:-- --:--:-- --:--:-- 116k { "00:00:d2:3b:94:ce:41:46": { "my-flow1": { "actions": [ { "length": 8, "lengthU": 8, "maxLength": 32767, "port": -5, "type": "OUTPUT" } ], "bufferId": -1, "command": 0, "cookie": 45035997925042933, "flags": 0, "hardTimeout": 0, "idleTimeout": 0, "length": 80, "lengthU": 80, "match": { "dataLayerDestination": "00:00:00:00:00:00", "dataLayerSource": "00:00:00:00:00:00", "dataLayerType": "0x0000", "dataLayerVirtualLan": -1, "dataLayerVirtualLanPriorityCodePoint": 0, "inputPort": 100, "networkDestination": "0.0.0.0", "networkDestinationMaskLen": 0, "networkProtocol": 0, "networkSource": "0.0.0.0", "networkSourceMaskLen": 0, "networkTypeOfService": 0, "transportDestination": 0, "transportSource": 0, "wildcards": 4194302 }, "outPort": -1, "priority": -32768, "type": "FLOW_MOD", "version": 1, "xid": 0 }, "my-flow2": { "actions": [ { "length": 8, "lengthU": 8, "maxLength": 32767, "port": -5, "type": "OUTPUT" } ], "bufferId": -1, "command": 0, "cookie": 45035997925042934, "flags": 0, "hardTimeout": 0, "idleTimeout": 0, "length": 80, "lengthU": 80, "match": { "dataLayerDestination": "00:00:00:00:00:00", "dataLayerSource": "00:00:00:00:00:00", "dataLayerType": "0x0000", "dataLayerVirtualLan": -1, "dataLayerVirtualLanPriorityCodePoint": 0, "inputPort": 101, "networkDestination": "0.0.0.0", "networkDestinationMaskLen": 0, "networkProtocol": 0, "networkSource": "0.0.0.0", "networkSourceMaskLen": 0, "networkTypeOfService": 0, "transportDestination": 0, "transportSource": 0, "wildcards": 4194302 }, "outPort": -1, "priority": -32768, "type": "FLOW_MOD", "version": 1, "xid": 0 } } }
通过 Floodlight 的 RestAPI，删除交换机上的流表规则
curl http://10.250.3.10:8080/wm/staticflowentrypusher/clear/00:00:d2:3b:94:ce:41:46/json
本文永久更新链接地址：http://www.linuxidc.com/Linux/2017-06/144772.htm
