32.0Keepalived高可用
程序员文章站
2024-03-21 09:42:46
...
KeepAlived实现
keepalived基于lvs，功能包含lvs，解决了lvs不具备健康性检查，单点失败问题；lvs是4层调度，基于内核级，并发访问量400万左右
LB Cluster:
四层：lvs, nginx(stream)，haproxy(mode tcp) l
七层：http: nginx(http), haproxy(mode http), httpd...
lvs号称400万并发，内核级；haproxy 1万，功能多；nginx当代理2到3万并发HA Cluster 配置准备： #高可用集群 high availability
(1) 各节点时间必须同步
ntp, chrony
(2) 确保iptables及selinux不会成为阻碍
(3) 各节点之间可通过主机名互相通信（对KA并非必须）
建议使用/etc/hosts文件实现
(4) 各节点之间的root用户可以基于**认证的ssh服务完成互相通信（对KA并非必须）
keepalived的nat模式下如果rs服务器和keepalived连接，需要指向keepalived的网关，需要两个浮动地址来完成请求报文和响应报文(基于ip的原因)；而dr模式下，只需要请求报文时的浮动地址，响应报文无需浮动地址（基于mac地址的原因），keepalived的高可用集群需要多播，即D类地址，multicast用来沟通监控或者交换浮动ip地址
keepalived： #集成了lvs的功能，使其具备高可用性，健康性检查
vrrp协议：Virtual Router Redundancy Protocol
术语：
虚拟路由器：Virtual Router
虚拟路由器标识：VRID(0-255)，唯一标识虚拟路由器
物理路由器：
master：主设备
backup：备用设备
priority：优先级
VIP：Virtual IP
VMAC：Virutal MAC (00-00-5e-00-01-VRID)
KeepAlived
通告：心跳，优先级等；周期性
工作方式：抢占式，非抢占式
安全工作：
认证：
无认证
简单字符认证：预共享**
MD5
工作模式：
主/备：单虚拟路径器
主/主：主/备（虚拟路径器1），备/主（虚拟路径器2） #互为主备，如果一个宕机，则另一个需要承担两个的工作量
keepalived:
vrrp协议的软件实现，原生设计目的为了高可用ipvs服务
功能：
vrrp协议完成地址流动
为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
为ipvs集群的各RS做健康状态检测
基于脚本调用接口通过执行脚本完成脚本中定义的功能，进而影响集群事务， 以此支持nginx、haproxy等服务
组件：
核心组件：
vrrp stack
ipvs wrapper ipvs规则
checkers
控制组件：配置文件分析器
IO复用器
内存管理组件
实验
[root@cos7 ~ ]#hostnamectl set-hostname ka1
[root@ka1 ~ ]#cd /etc/keepalived/
[root@ka1 keepalived ]#vim keepalived.conf
[root@ka1 keepalived ]#ssh-genkey
[root@ka1 keepalived ]#ssh-copy-id 192.168.31.17
[root@ka2 ~ ]#ssh-******
[root@ka2 ~ ]#ssh-copy-id 192.168.31.7
[root@ka1 keepalived ]#vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ka1
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.31.17 ka2
[root@ka2 etc ]#vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ka2
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.31.7 ka1
[root@ka1 etc ]#vim /etc/keepalived/keepalived.conf
"keepalived.conf" 157L, 3598C 1,1 Top
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost #通知邮件
}
notification_email_from ka@localhost #发送邮件的地址
smtp_server 127.0.0.1 #邮件服务器
smtp_connect_timeout 30 #邮件超时时长
router_id ka1 #ka2为ka2 #router id名称
vrrp_mcast_group4 230.10.10.10 #多播地址
}
vrrp_instance VI_1 { #vrrp实例1
state MASTER #主
interface eth0 #绑定网卡eth0
virtual_router_id 50 #id号
priority 100 #ka2为80 #优先级100
advert_int 1 #检查可用性间隔1s
nopreempt #非抢占式选举
authentication {
auth_type PASS #认证
auth_pass 123456
}
virtual_ipaddress {
192.168.31.100/24 #浮动ip地址
}
notify_master "/etc/keepalived/notify.sh master" #脚本
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
[root@ka2 etc ]#systemctl start keepalived.service
[root@ka2 etc ]#systemctl status keepalived.service
Aug 31 20:54:11 ka2 Keepalived_vrrp[3147]: VRRP_Instance(VI_1) Transition to MASTER STATE
[root@ka2 etc ]#ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:50:56:22:38:c9 brd ff:ff:ff:ff:ff:ff
inet 192.168.31.17/24 brd 192.168.31.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet 192.168.31.100/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::4619:664b:f1e4:d5b3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@client ~ ]#ping 192.168.31.100
[root@cos27 ~ ]#tcpdump -i eth0 -nn dst 230.10.10.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:57:14.870785 IP 192.168.31.17 > 230.10.10.10: VRRPv2, Advertisement, vrid 50, prio 80, authtype simple, intvl 1s, length 20
20:57:15.873160 IP 192.168.31.17 > 230.10.10.10: VRRPv2, Advertisement, vrid 50, prio 80, authtype simple, intvl 1s, length 20
若启动ka1，则100优先级高于ka2的80,192.168.31.100的ip地址将浮动到ka1的eth0网卡上日志
[aaa@qq.com ~ ]#rpm -ql keepalived
[aaa@qq.com ~ ]#vim /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs -I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#
#KEEPALIVED_OPTIONS="-D"
KEEPALIVED_OPTIONS="-D -S 3"
[aaa@qq.com ~ ]#vim /etc/rsyslog.conf
# Save boot messages also to boot.log
local7.* /var/log/boot.log
local3.* /var/log/keepalived.log
[aaa@qq.com ~ ]#systemctl restart rsyslog
[aaa@qq.com ~ ]#systemctl restart keepalived
[aaa@qq.com ~ ]#tail -f /var/log/keepalived.log通知邮件脚本
[aaa@qq.com keepalived ]#vim notify.sh
#!/bin/bash
#
contact='aaa@qq.com'
notify() {
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
[aaa@qq.com keepalived ]#systemctl reload keepalived
#把ka1的网络连接断开，用另一个网卡连接xshell，如下
[aaa@qq.com ~ ]#mail
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/root": 1 message 1 new
>N 1 root Sun Sep 2 14:17 18/665 "ka1 to be fault, vip floating"
>& 1
>Message 1:
>From aaa@qq.com Sun Sep 2 14:17:15 2018
>Return-Path: <aaa@qq.com>
>X-Original-To: aaa@qq.com
>Delivered-To: aaa@qq.com
>Date: Sun, 02 Sep 2018 14:17:15 +0800
>To: aaa@qq.com
>Subject: ka1 to be fault, vip floating
>User-Agent: Heirloom mailx 12.5 7/5/10
>Content-Type: text/plain; charset=us-ascii
>From: aaa@qq.com (root)
>Status: Rkeepalived双主配置
[root@ka1 keepalived ]#cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from ka@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1
vrrp_mcast_group4 230.10.10.10
}
vrrp_instance VI_1 {
state MASTER #ka2为BACKUP
interface eth0
virtual_router_id 50
priority 100 #ka2为80
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.31.100/24
}
}
vrrp_instance VI_2 {
state BACKUP #ka2为MASTER
interface eth0
virtual_router_id 60
priority 80 #ka2位100
advert_int 1
authentication {
auth_type PASS
auth_pass 654321
}
virtual_ipaddress {
192.168.31.200/24
}
}
[root@ka1 keepalived ]#systemctl reload keepalived
[root@ka2 keepalived ]#systemctl reload keepalived实验：利用keepalived实现双主的IPVS 的DR模型高可用性
图keepalived-dr 30.2lvs-dr
ka1
[root@ka1 keepalived]#cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from ka@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1
vrrp_mcast_group4 230.10.10.10
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.100/24
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 60
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 654321
}
virtual_ipaddress {
10.0.0.200/24
}
}
virtual_server 10.0.0.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.31.27 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
virtual_server 10.0.0.200 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.31.37 80 {
weight 2
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
------------------------------------------------
ka2
[root@ka2 keepalived]#cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from ka@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka2
vrrp_mcast_group4 230.10.10.10
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.100/24
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 60
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 654321
}
virtual_ipaddress {
10.0.0.200/24
}
}
virtual_server 10.0.0.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.31.27 80 {
weight 2
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
virtual_server 10.0.0.200 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.31.37 80 {
weight 10
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
rs服务器的公网ip要配置不响应，不宣传拥有ip[root@ka1 keepalived]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 wrr
-> 192.168.31.27:80 Route 1 0 0
TCP 10.0.0.200:80 wrr
-> 192.168.31.37:80 Route 2 0 0
#rs服务器的htptd服务都停止后，如下
[root@ka2 keepalived ]#echo sorry_server 129.0.0.1 80 > /var/www/html/index.html
[root@client ~ ]#curl 10.0.0.100
sorry_server 127.0.0.1 80