用nginx做負(fù)載均衡，作為架構(gòu)的最前端或中間層，隨著日益增長(zhǎng)的訪問(wèn)量，需要給負(fù)載均衡做高可用架構(gòu)，利用keepalived解決單點(diǎn)風(fēng)險(xiǎn)，一旦 nginx宕機(jī)能快速切換到備份服務(wù)器。
VMware DHCP Service和VMware NAT Service兩個(gè)服務(wù)節(jié)點(diǎn)部署
| 節(jié)點(diǎn) | 地址 | 服務(wù) |
|---|---|---|
| centos7_1 | 192.168.211.130 | Keepalived+Nginx |
| centos7_2 | 192.168.211.131 | Keepalived+Nginx |
| centos7_3 | 192.168.211.132 | Redis服務(wù)器 |
| web1（物理機(jī)） | 192.168.211.128 | FastApi+Celery |
| web2（物理機(jī)） | 192.168.211.129 | FastApi+Celery |
web1啟動(dòng)python http服務(wù)器
vim index.html <html> <body> <h1>Web Svr 1</h1> </body> </html> nohup python -m SimpleHTTPServer 8080 > running.log 2>&1 &
web2啟動(dòng)python http服務(wù)器
vim index.html <html> <body> <h1>Web Svr 2</h1> </body> </html> nohup python -m SimpleHTTPServer 8080 > running.log 2>&1 &
關(guān)閉防火墻
firewall-cmd --state systemctl stop firewalld.service systemctl disable firewalld.service
現(xiàn)在瀏覽器訪問(wèn)就正常了，頁(yè)面顯示W(wǎng)eb Svr 1 和 2
首先配置阿里云的源
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
安裝依賴包
yum -y install gcc yum install -y pcre pcre-devel yum install -y zlib zlib-devel yum install -y openssl openssl-devel
下載nginx，并解壓
wget http://nginx.org/download/nginx-1.8.0.tar.gz tar -zxvf nginx-1.8.0.tar.gz
安裝nginx
cd nginx-1.8.0 ./configure --user=nobody --group=nobody --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_gzip_static_module --with-http_realip_module --with-http_sub_module --with-http_ssl_module make make install cd /usr/local/nginx/sbin/ # 檢查配置文件 ./nginx -t # 啟動(dòng)nginx ./nginx
開(kāi)放nginx訪問(wèn)
firewall-cmd --zone=public --add-port=80/tcp --permanent systemctl restart firewalld.service
此時(shí)訪問(wèn)130和131都可以看到nginx的首頁(yè)。
需要在init.d文件夾中創(chuàng)建nginx啟動(dòng)文件。 這樣每次服務(wù)器重新啟動(dòng)init進(jìn)程都會(huì)自動(dòng)啟動(dòng)Nginx。
cd /etc/init.d/
vim nginx
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemin
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# pidfile: /var/run/nginx.pid
# user: nginx
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/local/nginx/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"
lockfile=/var/run/nginx.lock
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
校驗(yàn)配置文件依次輸入下列命令
chkconfig --add nginx chkconfig --level 345 nginx on
給這個(gè)文件添加執(zhí)行權(quán)限
chmod +x nginx ls functions netconsole network nginx README
啟動(dòng)Nginx服務(wù)
service nginx start service nginx status service nginx reload
Nginx反向代理、負(fù)載均衡（centos_1）
修改nginx.conf配置文件，去除注釋的代碼
cd /usr/local/nginx/conf/ mv nginx.conf nginx.conf.bak egrep -v '^#' nginx.conf.bak egrep -v '^#|^[ ]*#' nginx.conf.bak egrep -v '^#|^[ ]*#|^$' nginx.conf.bak egrep -v '^#|^[ ]*#|^$' nginx.conf.bak >> nginx.conf cat nginx.conf
輸出如下
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
重新加載nginx配置
# 測(cè)試配置文件是否正常 ../sbin/nginx -t # 重新加載nginx配置 ../sbin/nginx -s reload
配置nginx反向代理、負(fù)載均衡
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# websvr 服務(wù)器集群（也可以叫負(fù)載均衡池）
upstream websvr {
server 192.168.211.128:8001 weight=1;
server 192.168.211.129:8001 weight=2;
}
server {
listen 80;
# 用來(lái)指定ip地址或者域名，多個(gè)配置之間用空格分隔
server_name 192.168.211.130;
location / {
# 將所有請(qǐng)求交給websvr集群去處理
proxy_pass http://websvr;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
現(xiàn)在重啟nginx
sbin/nginx -s reload
websvr名稱可自定義，可以指明這些服務(wù)器的含義。也就是只需要添加upstream websvr和proxy_pass就可以實(shí)現(xiàn)負(fù)載均衡。
現(xiàn)在訪問(wèn)130，頁(yè)面上就會(huì)出現(xiàn)Web Svr 1和Web Svr 2切換，會(huì)根據(jù)權(quán)重選擇服務(wù)器，weight值越大，權(quán)重越高，也就是重復(fù)刷新該頁(yè)面，平均Web Svr 2出現(xiàn)2次，Web Svr 1出現(xiàn)1次。
到目前為止，仍然不能實(shí)現(xiàn)高可用，雖然web服務(wù)可以這樣做，單點(diǎn)故障可以通過(guò)這種方式處理，但是如果nginx服務(wù)故障了，整個(gè)系統(tǒng)基本就無(wú)法訪問(wèn)了，那么就需要使用多臺(tái)Nginx來(lái)保障。
多個(gè)Nginx協(xié)同工作，Nginx高可用【雙機(jī)主從模式】
在131服務(wù)器（centos_2）上新增一臺(tái)nginx服務(wù)，和之前的配置一樣，只需要修改 nginx.conf 即可
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream websvr {
server 192.168.211.128:8001 weight=1;
server 192.168.211.129:8001 weight=2;
}
server {
listen 80;
server_name 192.168.211.131;
location / {
proxy_pass http://websvr;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
# 重新加載nginx
sbin/nginx -s reload
現(xiàn)在訪問(wèn) http://192.168.211.130/ 也可以得到和 http://192.168.211.131/ 類似的結(jié)果。
這兩臺(tái)Nginx服務(wù)器的IP是不同的，那怎么做才能將這兩臺(tái)nginx服務(wù)器一起工作呢？這就需要用到keepalived了。
安裝軟件，兩臺(tái)centos同時(shí)安裝
yum install keepalived pcre-devel -y
配置keepalived
兩臺(tái)均備份
cp /etc/keepalived/keepalived.conf keepalived.conf.bak
centos_1配置Keepalived-MASTER
[root@localhost keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
script_user root
enable_script_security
}
vrrp_script chk_nginx {
# 指定監(jiān)控腳本，檢測(cè)nginx服務(wù)是否正常運(yùn)行
script "/etc/keepalived/chk_nginx.sh"
# 指定監(jiān)控時(shí)間，每10s執(zhí)行一次
interval 10
# 腳本結(jié)果導(dǎo)致的優(yōu)先級(jí)變更，檢測(cè)失?。_本返回非0）則優(yōu)先級(jí) -5
# weight -5
# # 檢測(cè)連續(xù)2次失敗才算確定是真失敗。會(huì)用weight減少優(yōu)先級(jí)（1-255之間）
# fall 2
# 檢測(cè)1次成功就算成功。但不修改優(yōu)先級(jí)
# rise 1
}
vrrp_instance VI_1 {
# 指定keepalived的角色，主機(jī)設(shè)置為MASTER，備用機(jī)設(shè)置為BACKUP
state BACKUP
# 指定HA監(jiān)測(cè)網(wǎng)絡(luò)的接口。centos7使用 ip addr 獲取
interface ens33
# 主備的virtual_router_id必須一樣，可以設(shè)置為IP后一組：must be between 1 & 255
virtual_router_id 51
# 優(yōu)先級(jí)值，在同一個(gè)vrrp_instance下， MASTRE 一定要高于 BAUCKUP，MASTER恢復(fù)后，BACKUP自動(dòng)交接
priority 90
# VRRP 廣播周期秒數(shù)，如果沒(méi)檢測(cè)到該廣播，就被認(rèn)為服務(wù)掛了，將切換主備
advert_int 1
# 設(shè)置驗(yàn)證類型和密碼。主從必須一樣
authentication {
# 設(shè)置vrrp驗(yàn)證類型，主要有PASS和AH兩種
auth_type PASS
# 加密的密碼，兩臺(tái)服務(wù)器一定要一樣，才能正常通信
auth_pass 1111
}
track_script {
# 執(zhí)行監(jiān)控的服務(wù)，引用VRRP腳本，即在 vrrp_script 部分指定的名字。定期運(yùn)行它們來(lái)改變優(yōu)先級(jí)
chk_nginx
}
virtual_ipaddress {
# VRRP HA 虛擬地址 如果有多個(gè)VIP，繼續(xù)換行填寫(xiě)
192.168.211.140
}
}
把配置文件發(fā)送到131節(jié)點(diǎn)
scp /etc/keepalived/keppalived.conf 192.168.211.131:/etc/keepalived/keepalived.conf
對(duì)于131節(jié)點(diǎn)只需要修改
state BACKUP priority 90
主keepalived配置監(jiān)控腳本chk_nginx.sh
創(chuàng)建一個(gè)腳本，用于在keepalived中執(zhí)行
vi /etc/keepalived/chk_nginx.sh
#!/bin/bash
# 查看是否有 nginx進(jìn)程 把值賦給變量counter
counter=`ps -C nginx --no-header |wc -l`
# 如果沒(méi)有進(jìn)程值得為 0
if [ $counter -eq 0 ];then
# 嘗試啟動(dòng)nginx
echo "Keepalived Info: Try to start nginx" >> /var/log/messages
/etc/nginx/sbin/nginx
sleep 3
if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then
# 輸出日至道系統(tǒng)消息
echo "Keepalived Info: Unable to start nginx" >> /var/log/messages
# 如果還沒(méi)沒(méi)啟動(dòng)，則結(jié)束 keepalived 進(jìn)程
# killall keepalived
# 或者停止
/etc/init.d/keepalived stop
exit 1
else
echo "Keepalived Info: Nginx service has been restored" >> /var/log/messages
exit 0
fi
else
# 狀態(tài)正常
echo "Keepalived Info: Nginx detection is normal" >> /var/log/messages;
exit 0
fi
接下來(lái)授予執(zhí)行權(quán)限，并測(cè)試
chmod +x chk_nginx.sh ./chk_nginx.sh
兩邊重啟keepalived
systemctl restart keepalived systemctl status keepalived
此時(shí)訪問(wèn).140也是可以正常顯示的，也就是綁定的IP成功了。執(zhí)行前可以通過(guò)下面命令實(shí)時(shí)查看 messages 中的輸出日志
tail -f /var/log/messages # 如果nginx關(guān)閉 Keepalived Info: Try to start nginx Keepalived Info: Nginx service has been restored # nginx正常打開(kāi) Keepalived Info: Nginx detection is normal
當(dāng)nginx檢測(cè)正常，就會(huì)返回0；檢測(cè)沒(méi)有了，返回1，但是keepalived似乎不是檢測(cè)這個(gè)返回值來(lái)實(shí)現(xiàn)轉(zhuǎn)移，而是檢測(cè)keepalived服務(wù)是否存在，來(lái)釋放本地VIP后，最終轉(zhuǎn)移虛擬IP，到另一臺(tái)服務(wù)器。
參考文章
https://www.jianshu.com/p/7e8e61d34960
https://www.cnblogs.com/zhangxingeng/p/10721083.html
到此這篇關(guān)于Vmware部署Nginx+KeepAlived集群雙主架構(gòu)的文章就介紹到這了,更多相關(guān)Nginx+KeepAlived集群內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
標(biāo)簽：銅川 周口 朝陽(yáng) 泰州 那曲 慶陽(yáng) 松原 蕪湖
巨人網(wǎng)絡(luò)通訊聲明：本文標(biāo)題《Vmware部署Nginx+KeepAlived集群雙主架構(gòu)的問(wèn)題及解決方法》，本文關(guān)鍵詞 Vmware,部署,Nginx+KeepAlived,；如發(fā)現(xiàn)本文內(nèi)容存在版權(quán)問(wèn)題，煩請(qǐng)?zhí)峁┫嚓P(guān)信息告之我們，我們將及時(shí)溝通與處理。本站內(nèi)容系統(tǒng)采集于網(wǎng)絡(luò)，涉及言論、版權(quán)與本站無(wú)關(guān)。
