|
|
相信很多朋友对企业级的负载均衡高可用实例非常感兴趣，此篇文章根据成熟的线上环境而写，旨在帮助大家迅速架构一个企业级的负载均衡高可用的web环境。
此系统架构仅映射内网VIP的80及443端口于外网的Juniper防火墙下，其它端口均关闭，内网所有机器均关闭iptables及ipfw防火墙；外网DNS指向即通过Juniper映射出来的外网地址。
第一部分：Nginx+Keepalived的说明及环境说明
喜欢看我博客或文章的小伙都知道，我一直主力推崇Nginx+Keepalived作web的负载均衡高可用架构，并积极将其用于项目方案中；Nginx负载均衡作服务器遇到的故障一般有①服务器网线松动等网络故障；②服务器硬件故障从而crash；③nginx服务死掉；遇到前二者情况，keeaplived是能起到HA的作用的；然而遇到③种情况就没有办法了，但可以通过shell监控解决这问题，从而实现真正意义上的负载均衡高可用。此篇的最新更新时间为2010年6月25号，下面将其安装步骤详细说明下：
环境：
centos5.3(64位)、nginx-0.7.51、keepalived-1.1.15
主nginx负载均衡器：192.168.0.154
辅nginx负载均衡器：192.168.9.155
vip：192.168.0.188
第二部分：分别安装Nginx负载均衡器及相关配置脚本
先安装Nginx负载均衡器，nginx负载的配置就用一般的模板来配置了
#添加运行nginx的用户和组www groupadd www useradd -g www www wget ftp://ftp.csx.cam.ac.uk/pub/soft ... cre/pcre-7.8.tar.gz tar zxvf pcre-7.8.tar.gz cd pcre-7.8/ ./configure make && make install wget http://sysoev.ru/nginx/nginx-0.7.51.tar.gz tar zxvf nginx-0.7.51.tar.gz cd nginx-0.7.51/ ./configure --user=www --group=www --prefix=/usr/local/webserver/nginx --with-http_stub_status_module --with-http_ssl_module make && make install
配置nginx负载均衡器的配置文件vim /usr/local/nginx/conf/nginx.conf,此篇文章仅仅只是我的某项目的配置文档，纯80转发；如果对nginx配置有https要求的可参考张宴的相关文章。
user www www; worker_processes 8; error_log /usr/local/webserver/nginx/logs/nginx_error.log crit;pid /usr/local/webserver/nginx/logs/nginx.pid; worker_rlimit_nofile 65535; events { use epoll; worker_connections 65535; } http { include mime.types; default_type application/octet-stream; server_names_hash_bucket_size 128; client_header_buffer_size 32k; large_client_header_buffers 4 32k; client_max_body_size 8m; sendfile on; tcp_nopush on; keepalive_timeout 60; tcp_nodelay on; fastcgi_connect_timeout 300; fastcgi_send_timeout 300; fastcgi_read_timeout 300; fastcgi_buffer_size 64k; fastcgi_buffers 4 64k; fastcgi_busy_buffers_size 128k; fastcgi_temp_file_write_size 128k; gzip on; gzip_min_length 1k; gzip_buffers 4 16k; gzip_http_version 1.0; gzip_comp_level 2; gzip_types text/plain application/x-javascript text/css application/xml; gzip_vary on; server { listen 80; server_name www.cn7788.com; index index.php index.htm; location ~ .*\.(php|php5)?$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fcgi.conf; } location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ { expires 30d; } location ~ .*\.(js|css)?$ { expires 1h; } location /nginx/ { stub_status on; access_log off; log_format weblogs '$remote_addr - $remote_user [$time_local] $request ' '"$status" $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /data/logs/weblogs.log weblogs; } } }
小节：
第一部分和第二部分讲的是如何通过安装Nginx来达到负载均衡后端web的目的，Nginx能实现自动切换后端有故障的web服务器，但Nginx与Nginx之间的高HA是如何实现的呢？
第三部分：安装Keepalived，让其分别作web及Nginx的HA
安装keepalived，并将其做成服务模式，方便以后调试。
wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz #tar zxvf keepalived-1.1.15.tar.gz #cd keepalived-1.1.15 #./configure --prefix=/usr/local/keepalived #make #make install #cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ #cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ #cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ #mkdir /etc/keepalived #cd /etc/keepalived/ vim keepalived.conf ! Configuration File for keepalived global_defs { notification_email { yuhongchun027@163.com } notification_email_from keepalived@chtopnet.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 mcast_src_ip 192.168.0.155 <==主nginx的IP地址 priority 100 advert_int 1 authentication { auth_type PASS auth_pass chtopnet } virtual_ipaddress { 192.168.0.188 <==vip地址 } } #service keepalived start
我们来看一下日志：
[root@ltos ~]# tail /var/log/messages Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.188 on eth0. Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.154 on eth0. Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering HINFO record with values 'I686'/'LINUX'. Oct 6 03:25:23 ltos avahi-daemon[2306]: Withdrawing address record for fe80::20c:29ff:feb9:eeab on eth0. Oct 6 03:25:23 ltos avahi-daemon[2306]: Withdrawing address record for 192.168.0.154 on eth0. Oct 6 03:25:23 ltos avahi-daemon[2306]: Host name conflict, retrying with <ltos-31>
很显然vrrp已经启动，我们还可以通过命令来检查
[root@ltos html]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:ba:9b:e7 brd ff:ff:ff:ff:ff:ff inet 192.168.0.154/24 brd 192.168.0.255 scope global eth0 inet 192.168.0.188/32 scope global eth0 inet6 fe80::20c:29ff:feba:9be7/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
说明vip已经启动，这样主服务器就配置好了，辅机的配置大致一样，除了配置文件有少部分的变化，下面贴出辅机的配置文件：
! Configuration File for keepalived global_defs { notification_email { yuhongchun027@163.com } notification_email_from keepalived@chtopnet.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 mcast_src_ip 192.168.0.154 <==辅nginx的IP的地址 priority 100 advert_int 1 authentication { auth_type PASS auth_pass chtopnet } virtual_ipaddress { 192.168.0.188 } }
第四部分：针对Keepalived的不足，用Nginx_pid.sh来监控nginx进程，实现真正意义上的负载均衡高可用。
针对Nginx+Keepalived，编写nginx监控脚本nginx_pid.sh，此脚本思路其实也很简单，即放置在后台一直监控nginx进程；如进程消失，尝试重启nginx，如是失败则立即停掉本机的keepalived服务，让另一台负载均衡器接手，此脚本直接从生产环境下载：
vim /root/nginx_pid.sh #!/bin/bash while : do nginxpid=`ps -C nginx --no-header | wc -l` if [ $nginxpid -eq 0 ];then /usr/local/nginx/sbin/nginx sleep 5 if [ $nginxpid -eq 0 ];then /etc/init.d/keepalived stop fi fi sleep 5 done
然后置于后台运行 sh /root/nginx_pid.sh &，这种写法是错误的，这样你用root用户logout后，此进程会消失；正确写法为nohu /bin/bash /root/nginx_pid.sh &,附带下注释:如果你正在运行一个进程，而且你觉得在退出帐户时该进程还不会结束，那么可以使用nohup命令。该命令可以在你退出root帐户之后继续运行相应的进程。nohup就是不挂起的意思( no hang up)，哈哈，差点老马失蹄了。 |
|
/1 
关于我们
发表于 2010-8-23 23:42:37
QQ好友和群
QQ空间
腾讯微博
腾讯朋友
微信
收藏
置顶服务
换色服务
楼主