管理集群工具之LVS
集群概念
将很多机器组织在一起,作为一个整体对外提供服务
集群在扩展性、性能方面都可以做到很灵活
集群分类
负载均衡集群:Load Balance
高可用集群:High Availability
高性能计算:High Performance Computing
LVS
LVS
:Linux Virtual Server
,Linux
虚拟服务器
实现负载均衡集群
作者:章文嵩。国防科技大学读博士期间编写
LVS
的工作模式
NAT
:网络地址转换
DR
:路由模式
TUN
:隧道模式
术语
调度器:LVS
服务器
真实服务器:提供服务的服务器
VIP
:虚拟地址,提供给用户访问的地址
DIR
:指定地址,LVS
服务器上与真实服务器通信的地址
RIP
:真实地址,真实服务器的地址
常见的调度算法,共10个,常用的有4个
轮巡rr
:真实服务器轮流提供服务(一共有两台服务器,第一个给A,第二个给B,第三个再给A…)
加权轮巡wrr
:根据服务器性能设置权重,权重大的服务器得到分配的请求就越多
最少连接lc
:根据真实服务器的连接数分配请求,优先分配连接少的
加权最少连接wlc
:类似wrr
,根据权重分配请求
ipvsadm
使用说明
-A
:添加虚拟服务器
-E
:编辑虚拟服务器
-D
:删除虚拟服务器
-t
:添加tcp
服务器
-u
:添加udp
服务器
-s
:指定调度算法。如轮巡rr
、加权轮巡wrr
、最小连接lc
、加权最小连接wlc
-a
:添加虚拟服务器后,向虚拟服务器中加入真实服务器
-r
:指定真实服务器
-w
:设置权重
-m
:指定工作模式为NAT
-g
:指定工作模式为DR
配置LVS NAT模式
NAT
模式:通过lvs
服务器来接收用户的请求,然后通过调度算法分配请求到真实服务器,然后真实服务器返回数据给lvs
服务器,再有lvs
服务器返回给用户
client
:eth0
->192.168.88.10,gateway
->192.168.88.5
lvs
:eth0
->192.168.88.5;eth1
->192.168.99.5
web1
:eth0
->192.168.99.100,gateway
->192.168.99.5
web2
:eth0
->192.168.99.200,gateway
->192.168.99.5
环境准备
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .88.10/24 ipv4.gateway 192.168 .88.5 connection.autoconnect yes
nmcli connection up eth0
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .88.5/24 connection.autoconnect yes
nmcli connection up eth0
nmcli connection modify eth1 ipv4.method manual ipv4.addresses 192.168 .99.5/24 connection.autoconnect yes
nmcli connection up eth1
sysctl -a
sysctl -a | grep ip_forward
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .99.100/24 ipv4.gateway 192.168 .99.5 connection.autoconnect yes
nmcli connection up eth0
yum install -y httpd
systemctl enable httpd --now
echo "web1" > /var/www/html/index.html
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .99.200/24 ipv4.gateway 192.168 .99.5 connection.autoconnect yes
nmcli connection up eth0
yum install -y httpd
systemctl enable httpd --now
echo "web1" > /var/www/html/index.html
相关配置
yum install -y ipvsadm
ipvsadm -A -t 192.168 .88.5:80 -s rr
ipvsadm -a -t 192.168 .88.5:80 -r 192.168 .99.100 -m
ipvsadm -a -t 192.168 .88.5:80 -r 192.168 .99.200 -m
ipvsadm -Ln
ipvsadm -D -t 192.168 .88.5:80 -s wrr
ipvsadm -a -t 192.168 .88.5:80 -r 192.168 .99.100 -w 1 -m
ipvsadm -a -t 192.168 .88.5:80 -r 192.168 .99.200 -w 2 -m
配置LVS DR模式
DR
模式相比于NAT
模式,它不会因为lvs
服务器的带宽而影响业务
DR
模式:通过lvs
服务器来接收请求,然后分配请求到真实服务器,然后通过真实服务器的虚拟网卡直接返回给用户
client
:eth0
->192.168.88.10
lvs
:eth0
->192.168.88.5;eth0:0
->192.168.88.15
web1
:eth0
->192.168.88.100;lo:0
->192.168.88.15/32
web2
:eth0
->192.168.88.200;lo:0
->192.168.88.15/32
环境准备
nmcli connection delete eth1
vim ifcfg-eth0
TYPE = Ethernet
BOOTPROTO = none
NAME = eth0
DEVICE = eth0
ONBOOT = yes
IPADDR = 192.168 .88.5
PREFIX = 24
GATEWAY = 192.168 .88.254
ifdown eth0; ifup eth0
cd /etc/sysconfig/network-scripts/
cp ifcfg-eth0 ifcfg-eth0:0
vim ifcfg-eth0:0
TYPE = Ethernet
BOOTPROTO = none
NAME = eth0:0
DEVICE = eth0:0
ONBOOT = yes
IPADDR = 192.168 .88.15
PREFIX = 24
ifdown eth0:0; ifup eth0:0
ifconfig
cd /etc/sysconfig/network-scripts/
vim ifcfg-eth0
TYPE = Ethernet
BOOTPROTO = none
NAME = eth0
DEVICE = eth0
ONBOOT = yes
IPADDR = 192.168 .88.200
PREFIX = 24
GATEWAY = 192.168 .88.254
ifdown eth0; ifup eth0
cp ifcfg-eth0 ifcfg-lo:0
TYPE = Ethernet
BOOTPROTO = none
NAME = lo:0
DEVICE = lo:0
ONBOOT = yes
IPADDR = 192.168 .88.15
PREFIX = 32
ifdown lo:0; ifup lo:0
ifconfig
vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
sysctl -p
cd /etc/sysconfig/network-scripts/
vim ifcfg-eth0
TYPE = Ethernet
BOOTPROTO = none
NAME = eth0
DEVICE = eth0
ONBOOT = yes
IPADDR = 192.168 .88.100
PREFIX = 24
GATEWAY = 192.168 .88.254
ifdown eth0; ifup eth0
cp ifcfg-eth0 ifcfg-lo:0
TYPE = Ethernet
BOOTPROTO = none
NAME = lo:0
DEVICE = lo:0
ONBOOT = yes
IPADDR = 192.168 .88.15
PREFIX = 32
ifdown lo:0; ifup lo:0
ifconfig
vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
sysctl -p
相关配置
ipvsadm -D -t 192.168 .88.5:80
ipvsadm -A -t 192.168 .88.15:80 -s lc
ipvsadm -a -t 192.168 .88.15:80 -r 192.168 .88.100 -w 100 -g
ipvsadm -a -t 192.168 .88.15:80 -r 192.168 .88.200 -w 200 -g
ipvsadm -E -t 192.168 .88.15:80 -s wlc
ipvsadm -Ln
配置Keepalived+LVS DR模式
keepalived
用于实现高可用集群
它的工作原理就是VRRP
(虚拟冗余路由协议)
可以实现lvs
服务器自动切换,并且可以根据web
服务器的工作情况自动添加和删除
client
:eth0
->192.168.88.10
lvs
集群,使用keepalived
服务,配置虚拟ip
为192.168.88.15/24
lvs1
:eth0
->192.168.88.5,Master
lvs2
:eth0
->192.168.88.6,Backup
web
集群,都配置lo:0
为192.168.88.15/32
web1
:eth0
->192.168.88.100
web2
:eth0
->192.168.88.200
环境准备
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .88.5/24 ipv4.gateway 192.168 .88.254 autoconnect yes
nmcli connection up eth0
yum install -y keepalived ipvsadm
nmcli connection modify eth0 ipv4.method manual ipv4.addresses 192.168 .88.6/24 ipv4.gateway 192.168 .88.254 autoconnect yes
nmcli connection up eth0
yum install -y keepalived ipvsadm
web1
和web2
跟上次实验操作一致,不需要额外操作
相关配置
vim /etc/keepalived/keepalived.conf
1 ! Configuration File for keepalived
2
3 global_defs {
4 notification_email {
5 acassen@firewall.loc
6 failover@firewall.loc
7 sysadmin@firewall.loc
8 }
9 notification_email_from Alexandre.Cassen@firewall.loc
10 smtp_server 192.168 .200.1
11 smtp_connect_timeout 30
12 router_id lvs1
13 vrrp_iptables
14 vrrp_skip_check_adv_addr
15 vrrp_strict
16 vrrp_garp_interval 0
17 vrrp_gna_interval 0
18 }
19
20 vrrp_instance VI_1 {
21 state MASTER
22 interface eth0
23 virtual_router_id 51
24 priority 100
25 advert_int 1
26 authentication {
27 auth_type PASS
28 auth_pass aabb
29 }
30 virtual_ipaddress {
31 192.168 .88.15
32 }
33 }
34
35 virtual_server 192.168 .88.15 80 {
36 delay_loop 6
37 lb_algo rr
38 lb_kind DR
39 persistence_timeout 50
40 protocol TCP
41
42 real_server 192.168 .88.100 80 {
43 weight 1
44 TCP_CHECK {
45 connect_timeout 3
46 nb_get_retry 3
47 delay_before_retry 3
48 }
49 }
50
51 real_server 192.168 .88.200 80 {
52 weight 2
53 TCP_CHECK {
54 connect_timeout 3
55 nb_get_retry 3
56 delay_before_retry 3
57 }
58 }
59 }
systemctl start keepalived.service
ip a s eth0 | grep 192.168 .88.15
ipvsadm -Ln
1 ! Configuration File for keepalived
2
3 global_defs {
4 notification_email {
5 acassen@firewall.loc
6 failover@firewall.loc
7 sysadmin@firewall.loc
8 }
9 notification_email_from Alexandre.Cassen@firewall.loc
10 smtp_server 192.168 .200.1
11 smtp_connect_timeout 30
12 router_id lvs2
13 vrrp_iptables
14 vrrp_skip_check_adv_addr
15 vrrp_strict
16 vrrp_garp_interval 0
17 vrrp_gna_interval 0
18 }
19
20 vrrp_instance VI_1 {
21 state BACKUP
22 interface eth0
23 virtual_router_id 51
24 priority 100
25 advert_int 1
26 authentication {
27 auth_type PASS
28 auth_pass aabb
29 }
30 virtual_ipaddress {
31 192.168 .88.15
32 }
33 }
34
35 virtual_server 192.168 .88.15 80 {
36 delay_loop 6
37 lb_algo rr
38 lb_kind DR
39 persistence_timeout 50
40 protocol TCP
41
42 real_server 192.168 .88.100 80 {
43 weight 1
44 TCP_CHECK {
45 connect_timeout 3
46 nb_get_retry 3
47 delay_before_retry 3
48 }
49 }
50
51 real_server 192.168 .88.200 80 {
52 weight 2
53 TCP_CHECK {
54 connect_timeout 3
55 nb_get_retry 3
56 delay_before_retry 3
57 }
58 }
59 }
systemctl start keepalived.service
ipvsadm -Ln
效果演示
正常访问
for i in { 1 .. 6 } ; do curl http://192.168.88.15; done
将lvs1
关机,然后查看lvs2
的ip,再次访问测试
ip a s | grep 192.168 .88.15
for i in { 1 .. 6 } ; do curl http://192.168.88.15; done
将web2
的httpd
服务器关闭,然后查看lvs2
的ipvsadm
配置,再次访问测试
ipvsadm -Ln
for i in { 1 .. 6 } ; do curl http://192.168.88.15; done
将lvs1
启动,并启动它的keepalived
服务,查看VIP
会不会自动切换过来,再次访问测试
ip a s | grep 192.168 .88.15
for i in { 1 .. 6 } ; do curl http://192.168.88.15; done