背景
记录iptables DNAT下,数据包转换方式
参考
链接: linux内核协议栈 netfilter 之 ip 层 netfilter 的 NAT 模块 hook 及 target 代码剖析
链接: iptables提供的NAT的分析与应用
链接: nf_conntrack连接跟踪机制
链接: 【博客524】iptables nat的实现根基:conntrack
链接: iptables详解(图文)
链接: Linux协议栈-netfilter(1)-框架
概念
netfilter 表和链
详见: Linux协议栈-netfilter(1)-框架 和 【博客524】iptables nat的实现根基:conntrack
de-DNAT和de-SNAT
de-DNAT(反向目的地址转换)以及de-SNAT(反向源地址转换),de-DNAT和de-SNAT发生的位置与DNAT和SNAT正好相反。也就是说DNAT发生在PREROUTING阶段,而de-DNAT发生在POSTROUTING阶段,都是是将地址进行对换的过程,前者改变的目的地址,而后者其实改变的是源地址(该源地址是DNAT操作前的目的地址)(以上摘自:iptables提供的NAT的分析与应用)
测试
环境
三台虚拟机:
1. centos7-10 作为Client 配置IP 172.16.0.200/24
2. centos7-18 作为Server 配置IP 192.168.10.100/24,启动nginx服务,监听192.168.10.100:80
3. centos7-22 作为Route 配置Out IP 172.16.0.1 和 Inner IP 192.168.10.1;启动路由转发功能,配置iptables DNAT 规则。
步骤
配置Server
- 宿主机Centos7-18的接口enp0s5添加IP 192.168.10.100/24
// 添加IP 192.168.10.100/24
[root@centos7-18 nginx]# ip addr add 192.168.10.100/24 dev enp0s5 label enp0s5:0
[root@centos7-18 nginx]# ip addr show enp0s5:0
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:1c:42:60:87:b2 brd ff:ff:ff:ff:ff:ff
inet 10.211.55.18/24 brd 10.211.55.255 scope global enp0s5
valid_lft forever preferred_lft forever
inet 192.168.10.100/24 scope global enp0s5:0
valid_lft forever preferred_lft forever
inet6 fdb2:2c26:f4e4:0:21c:42ff:fe60:87b2/64 scope global mngtmpaddr dynamic
valid_lft 2591581sec preferred_lft 604381sec
inet6 fe80::21c:42ff:fe60:87b2/64 scope link
valid_lft forever preferred_lft forever
- 启动nginx服务
- 修改nginx.conf配置文件,指定监听192.168.10.100:80
- 修改nginx.conf配置文件,指定监听192.168.10.100:80
// 修改/etc/nginx/nginx.conf配置,启动服务,查看监听192.168.10.100:80
[root@centos7-18 nginx]# systemctl start nginx
[root@centos7-18 nginx]# netstat -ntlp | grep nginx
tcp 0 0 192.168.10.100:80 0.0.0.0:* LISTEN 22687/nginx: master
tcp6 0 0 :::80 :::* LISTEN 22687/nginx: master
[root@centos7-18 nginx]#
- 在nginx服务root目录下增加serverinfo.html文件
[root@centos7-18 nginx]# echo 'Real server is 192.168.10.100:80!'>> /usr/share/nginx/html/serverinfo.html
- 添加静态路由,访问172.16.0.0/24 指向 192.168.10.1
[root@centos7-18 nginx]# ip route add 172.16.0.0/24 via 192.168.10.1 dev enp0s5:0
[root@centos7-18 nginx]# ip route
default via 10.211.55.1 dev enp0s5
10.211.55.0/24 dev enp0s5 proto kernel scope link src 10.211.55.18
169.254.0.0/16 dev enp0s5 scope link metric 1002
172.16.0.0/24 via 192.168.10.1 dev enp0s5
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.10.0/24 dev enp0s5 proto kernel scope link src 192.168.10.100
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
配置Client
- 宿主机Centos7-10的接口enp0s5添加IP 172.16.0.200/24
// An highlighted block
[root@centos7-10 ~]# ip add add 172.16.0.200/24 dev enp0s5 label enp0s5:0
[root@centos7-10 ~]# ip addr show enp0s5:0
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:1c:42:ae:b6:41 brd ff:ff:ff:ff:ff:ff
inet 10.211.55.10/24 brd 10.211.55.255 scope global noprefixroute dynamic enp0s5
valid_lft 1413sec preferred_lft 1413sec
inet 172.16.0.100/24 scope global enp0s5:0
valid_lft forever preferred_lft forever
inet6 fdb2:2c26:f4e4:0:cd1f:12f3:4076:6d89/64 scope global noprefixroute dynamic
valid_lft 2591724sec preferred_lft 604524sec
inet6 fe80::7e0c:1902:e1ca:4324/64 scope link noprefixroute
valid_lft forever preferred_lft forever
配置Route
- 宿主机Centos7-22的接口enp0s5添加Out IP 172.16.0.1/24 和 Inner IP 192.168.10.1/24
// 配置Out IP 172.16.0.1/24 和 Inner IP 192.168.10.1/24
[root@centos7-22 ~]# ip addr add 192.168.10.1/24 dev enp0s5 label enp0s5:inner
[root@centos7-22 ~]# ip addr add 172.16.0.1/24 dev enp0s5 label enp0s5:out
[root@centos7-22 ~]# ifconfig -a
enp0s5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.211.55.22 netmask 255.255.255.0 broadcast 10.211.55.255
inet6 fe80::7e0c:1902:e1ca:4324 prefixlen 64 scopeid 0x20<link>
inet6 fe80::567a:248b:5e94:5d19 prefixlen 64 scopeid 0x20<link>
inet6 fdb2:2c26:f4e4:0:233e:38df:2cbd:cec1 prefixlen 64 scopeid 0x0<global>
ether 00:1c:42:44:79:a5 txqueuelen 1000 (Ethernet)
RX packets 1864685 bytes 1399088235 (1.3 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1512576 bytes 1121134362 (1.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s5:inner: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.10.1 netmask 255.255.255.0 broadcast 0.0.0.0
ether 00:1c:42:44:79:a5 txqueuelen 1000 (Ethernet)
enp0s5:out: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.0.1 netmask 255.255.255.0 broadcast 0.0.0.0
ether 00:1c:42:44:79:a5 txqueuelen 1000 (Ethernet)
- 开启路由功能(ip_forward)
[root@centos7-22 ~]# sysctl -w net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1
[root@centos7-22 ~]#
- iptables 添加 nat表PREROUTING链下的DNAT规则
- 将访问172.16.0.1:80 的数据包目地IP修改为192.168.10.100
[root@centos7-22 ~]# iptables -t nat -I PREROUTING --dst 172.16.0.1 -p tcp --dport 80 -j DNAT --to-destination 192.168.10.100
// 查看添加结果
[root@centos7-22 ~]# iptables -t nat -L -nv
Chain PREROUTING (policy ACCEPT 447 packets, 42445 bytes)
pkts bytes target prot opt in out source destination
13 780 DNAT tcp -- * * 0.0.0.0/0 172.16.0.1 tcp dpt:80 to:192.168.10.100
25426 1539K DOCKER-INGRESS all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
27108 1641K DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 341 packets, 20764 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 499 packets, 33693 bytes)
pkts bytes target prot opt in out source destination
24 1218 DOCKER-INGRESS all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
10 672 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 512 packets, 34617 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * docker_gwbridge 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match src-type LOCAL
6 400 MASQUERADE all -- * !docker_gwbridge 172.18.0.0/16 0.0.0.0/0
0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker_gwbridge * 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-INGRESS (2 references)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:9090 to:172.18.0.2:9090
25450 1540K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
测试
- Client curl http://172.16.0.1/serverinfo.html
// centos7-10 作为客户端 curl http://172.16.0.1/serverinfo.html
[root@centos7-10 ~]# curl http://172.16.0.1/serverinfo.html
Real server is 192.168.10.100:80!
- Route 查看/proc/net/nf_conntrack
- origin: src=172.16.0.200 dst=172.16.0.1 sport=12390 dport=80
- reply: src=192.168.10.100 dst=172.16.0.200 sport=80 dport=12390
// An highlighted block
[root@centos7-22 ~]# cat /proc/net/nf_conntrack | grep 'port=80'
ipv4 2 tcp 6 115 TIME_WAIT src=172.16.0.200 dst=172.16.0.1 sport=12390 dport=80 src=192.168.10.100 dst=172.16.0.200 sport=80 dport=12390 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=2
- Route使用tcpdump 抓包查看到以下流程
- 23:06:32.279831 IP 172.16.0.200.12646 > 172.16.0.1.80
- 23:06:32.279902 IP 172.16.0.200.12646 > 192.168.10.100.80 (DNAT)
- 23:06:32.280218 IP 192.168.10.100.80 > 172.16.0.200.12646
- 23:06:32.280247 IP 172.16.0.1.80 > 172.16.0.200.12646(de-DNAT)
[root@centos7-22 ~]# tcpdump -nnn -i enp0s5 port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s5, link-type EN10MB (Ethernet), capture size 262144 bytes
23:06:32.279831 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [S], seq 3662704632, win 29200, options [mss 1460,sackOK,TS val 267072047 ecr 0,nop,wscale 7], length 0
23:06:32.279902 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [S], seq 3662704632, win 29200, options [mss 1460,sackOK,TS val 267072047 ecr 0,nop,wscale 7], length 0
23:06:32.280218 IP 192.168.10.100.80 > 172.16.0.200.12646: Flags [S.], seq 907403295, ack 3662704633, win 28960, options [mss 1460,sackOK,TS val 269979035 ecr 267072047,nop,wscale 7], length 0
23:06:32.280247 IP 172.16.0.1.80 > 172.16.0.200.12646: Flags [S.], seq 907403295, ack 3662704633, win 28960, options [mss 1460,sackOK,TS val 269979035 ecr 267072047,nop,wscale 7], length 0
23:06:32.280551 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.280567 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.280638 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [P.], seq 1:90, ack 1, win 229, options [nop,nop,TS val 267072048 ecr 269979035], length 89: HTTP: GET /serverinfo.html HTTP/1.1
23:06:32.280650 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [P.], seq 1:90, ack 1, win 229, options [nop,nop,TS val 267072048 ecr 269979035], length 89: HTTP: GET /serverinfo.html HTTP/1.1
23:06:32.280732 IP 192.168.10.100.80 > 172.16.0.200.12646: Flags [.], ack 90, win 227, options [nop,nop,TS val 269979035 ecr 267072048], length 0
23:06:32.280743 IP 172.16.0.1.80 > 172.16.0.200.12646: Flags [.], ack 90, win 227, options [nop,nop,TS val 269979035 ecr 267072048], length 0
23:06:32.280994 IP 192.168.10.100.80 > 172.16.0.200.12646: Flags [P.], seq 1:271, ack 90, win 227, options [nop,nop,TS val 269979035 ecr 267072048], length 270: HTTP: HTTP/1.1 200 OK
23:06:32.281008 IP 172.16.0.1.80 > 172.16.0.200.12646: Flags [P.], seq 1:271, ack 90, win 227, options [nop,nop,TS val 269979035 ecr 267072048], length 270: HTTP: HTTP/1.1 200 OK
23:06:32.281178 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [.], ack 271, win 237, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.281195 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [.], ack 271, win 237, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.281352 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [F.], seq 90, ack 271, win 237, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.281369 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [F.], seq 90, ack 271, win 237, options [nop,nop,TS val 267072048 ecr 269979035], length 0
23:06:32.281610 IP 192.168.10.100.80 > 172.16.0.200.12646: Flags [F.], seq 271, ack 91, win 227, options [nop,nop,TS val 269979036 ecr 267072048], length 0
23:06:32.281626 IP 172.16.0.1.80 > 172.16.0.200.12646: Flags [F.], seq 271, ack 91, win 227, options [nop,nop,TS val 269979036 ecr 267072048], length 0
23:06:32.281786 IP 172.16.0.200.12646 > 172.16.0.1.80: Flags [.], ack 272, win 237, options [nop,nop,TS val 267072049 ecr 269979036], length 0
23:06:32.281802 IP 172.16.0.200.12646 > 192.168.10.100.80: Flags [.], ack 272, win 237, options [nop,nop,TS val 267072049 ecr 269979036], length 0
总结
DNAT是在请求进入PREROUTING时发生的,修改数据包的目地IP,然后查找路由;de-DNAT是在响应进入POSTROUTING时发生的,根据之前建立的nf_conntrack,修改响应数据包的源IP,然后发出。
同样SNAT是在请求进入POSTROUTING时发生的,修改数据包的源IP,然后发出;de-SNAT响应进入PREROUTING时发生的,根据之前建立的nf_conntrack,修改响应数据包的目地IP,然后查找路由;