redis-cluster部署及數(shù)據(jù)遷移-創(chuàng)新互聯(lián)

工作原理

節(jié)選自redis官方文檔：http://www.redis.cn/topics/cluster-tutorial.html

成都創(chuàng)新互聯(lián)長期為上千余家客戶提供的網(wǎng)站建設(shè)服務(wù)，團(tuán)隊從業(yè)經(jīng)驗10年，關(guān)注不同地域、不同群體，并針對不同對象提供差異化的產(chǎn)品和服務(wù)；打造開放共贏平臺，與合作伙伴共同營造健康的互聯(lián)網(wǎng)生態(tài)環(huán)境。為新洲企業(yè)提供專業(yè)的網(wǎng)站建設(shè)、成都網(wǎng)站制作，新洲網(wǎng)站改版等技術(shù)服務(wù)。擁有10多年豐富建站經(jīng)驗和眾多成功案例,為您定制開發(fā)。

Redis集群介紹

Redis 集群是一個提供在多個Redis間節(jié)點(diǎn)間共享數(shù)據(jù)的程序集。
Redis集群并不支持處理多個keys的命令,因為這需要在不同的節(jié)點(diǎn)間移動數(shù)據(jù),從而達(dá)不到像Redis那樣的性能,在高負(fù)載的情況下可能會導(dǎo)致不可預(yù)料的錯誤.
Redis 集群通過分區(qū)來提供一定程度的可用性,在實際環(huán)境中當(dāng)某個節(jié)點(diǎn)宕機(jī)或者不可達(dá)的情況下繼續(xù)處理命令. Redis 集群的優(yōu)勢:

  • 自動分割數(shù)據(jù)到不同的節(jié)點(diǎn)上。
  • 整個集群的部分節(jié)點(diǎn)失敗或者不可達(dá)的情況下能夠繼續(xù)處理命令。

    Redis 集群的數(shù)據(jù)分片

    Redis 集群沒有使用一致性hash, 而是引入了 哈希槽的概念.
    Redis 集群有16384個哈希槽,每個key通過CRC16校驗后對16384取模來決定放置哪個槽.集群的每個節(jié)點(diǎn)負(fù)責(zé)一部分hash槽,舉個例子,比如當(dāng)前集群有3個節(jié)點(diǎn),那么:

  • 節(jié)點(diǎn) A 包含 0 到 5500號哈希槽.
  • 節(jié)點(diǎn) B 包含5501 到 11000 號哈希槽.
  • 節(jié)點(diǎn) C 包含11001 到 16384號哈希槽.

這種結(jié)構(gòu)很容易添加或者刪除節(jié)點(diǎn). 比如如果我想新添加個節(jié)點(diǎn)D, 我需要從節(jié)點(diǎn) A, B, C中得部分槽到D上. 如果我像移除節(jié)點(diǎn)A,需要將A中得槽移到B和C節(jié)點(diǎn)上,然后將沒有任何槽的A節(jié)點(diǎn)從集群中移除即可. 由于從一個節(jié)點(diǎn)將哈希槽移動到另一個節(jié)點(diǎn)并不會停止服務(wù),所以無論添加刪除或者改變某個節(jié)點(diǎn)的哈希槽的數(shù)量都不會造成集群不可用的狀態(tài).

Redis 集群的主從復(fù)制模型

為了使在部分節(jié)點(diǎn)失敗或者大部分節(jié)點(diǎn)無法通信的情況下集群仍然可用，所以集群使用了主從復(fù)制模型,每個節(jié)點(diǎn)都會有N-1個復(fù)制品.
在我們例子中具有A，B，C三個節(jié)點(diǎn)的集群,在沒有復(fù)制模型的情況下,如果節(jié)點(diǎn)B失敗了，那么整個集群就會以為缺少5501-11000這個范圍的槽而不可用.
然而如果在集群創(chuàng)建的時候（或者過一段時間）我們?yōu)槊總€節(jié)點(diǎn)添加一個從節(jié)點(diǎn)A1，B1，C1,那么整個集群便有三個master節(jié)點(diǎn)和三個slave節(jié)點(diǎn)組成，這樣在節(jié)點(diǎn)B失敗后，集群便會選舉B1為新的主節(jié)點(diǎn)繼續(xù)服務(wù)，整個集群便不會因為槽找不到而不可用了
不過當(dāng)B和B1 都失敗后，集群是不可用的.

redis安裝過程

1、下載redis安裝包

2、解壓

cd /app
tar zxf redis-3.2.11.tar.gz 
mv redis-3.2.11.tar.gz redis

3、編譯安裝

cd redis

make && make install

4、創(chuàng)建相關(guān)目錄,將相關(guān)命令從src目錄復(fù)制到bin目錄

mkdir -pv /app/redis/{bin,conf,data,logs}

5、使用utils中的install_server.sh安裝redis server

部署過程

Redis-cluster搭建

1、從測試打包已經(jīng)編譯redis文件夾，上傳至需要安裝的服務(wù)器

2、解壓至app目錄，目錄結(jié)構(gòu)如下，其中data,conf,logs為編譯完成后自行創(chuàng)建的目錄，分別用來存放數(shù)據(jù)文件，配置文件，日志文件，data目錄下自行創(chuàng)建以端口號命名的文件夾，分別用來存放各個端口號實例的數(shù)據(jù)文件

-rw-rw-r--  1 root root 92766 Sep 21 22:20 00-RELEASENOTES
drwxr-xr-x  2 root root  4096 Dec 20 16:40 bin
-rw-rw-r--  1 root root    53 Sep 21 22:20 BUGS
drwxr-xr-x  2 root root  4096 Jan  4 15:37 conf
-rw-rw-r--  1 root root  1805 Sep 21 22:20 CONTRIBUTING
-rw-rw-r--  1 root root  1487 Sep 21 22:20 COPYING
drwxr-xr-x 10 root root  4096 Jan  3 11:24 data
drwxrwxr-x  7 root root  4096 Dec 20 14:50 deps
-rw-rw-r--  1 root root    11 Sep 21 22:20 INSTALL
drwxr-xr-x  2 root root  4096 Jan  4 15:37 logs
-rw-rw-r--  1 root root   151 Sep 21 22:20 Makefile
-rw-rw-r--  1 root root  4223 Sep 21 22:20 MANIFESTO
-rw-rw-r--  1 root root  6834 Sep 21 22:20 README.md
-rw-rw-r--  1 root root 46695 Sep 21 22:20 redis.conf
-rwxrwxr-x  1 root root   271 Sep 21 22:20 runtest
-rwxrwxr-x  1 root root   280 Sep 21 22:20 runtest-cluster
-rwxrwxr-x  1 root root   281 Sep 21 22:20 runtest-sentinel
-rw-rw-r--  1 root root  7606 Sep 21 22:20 sentinel.conf
drwxrwxr-x  2 root root  4096 Dec 20 14:51 src
drwxrwxr-x 10 root root  4096 Sep 21 22:20 tests
drwxrwxr-x  7 root root  4096 Jan  4 14:36 utils

3、修改各個端口的配置文件，以node1:7000的配置文件為例，根據(jù)情況修改

bind node1
protected-mode yes
port 7000
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_7000.pid
loglevel notice
logfile /app/redis/logs/redis_7000.log
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /app/redis/data/7000
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

4、啟動對應(yīng)端口的服務(wù)（所有節(jié)點(diǎn)的主從都啟動）,以7000為例

/app/redis/bin/redis-server  /app/redis/conf/7000.conf

5、創(chuàng)建集群

 /app/redis/bin/redis-trib.rb create node1:7000 node2:7000 node2:7000

[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:5461-10922 (5461 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

6、從節(jié)點(diǎn)的添加需要在數(shù)據(jù)遷移之后

數(shù)據(jù)遷移

根據(jù)實際情況，參考：https://www.18188.org/articles/2016/04/23/1461374145366.html 進(jìn)行數(shù)據(jù)遷移。

1、將所有的slot都轉(zhuǎn)移到一臺主實例

[root@node1 bin]# ./redis-trib.rb reshard --from b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f   --to 3b485951a72e16133464585dc2920c10ce75967b  --slots 5462 --yes node1:7000

[root@node1 bin]# ./redis-trib.rb reshard --from 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc   --to 3b485951a72e16133464585dc2920c10ce75967b  --slots 5462 --yes node1:7000

2、將原有的阿里云redis備份rdb文件下載放到對應(yīng)目錄

cp /app/hins2533955_data_20180105195042.rdb /app/redis/data/7000/dump.rdb

3、重啟將備份文件讀入,重啟完成連接進(jìn)入使用dbsize查看數(shù)據(jù)是否導(dǎo)入

/app/redis/bin/redis-cli -h node1 -p 7000 shutdown
/app/redis/bin/redis-server /app/redis/conf/7000.conf

4、重新分配slot

[root@node1 bin]# ./redis-trib.rb reshard --from 3b485951a72e16133464585dc2920c10ce75967b   --to b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f  --slots 5462 --yes node2:7000

[root@node1 bin]# ./redis-trib.rb reshard --from 3b485951a72e16133464585dc2920c10ce75967b   --to 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc  --slots 5462 --yes node3:7000

5、增加從節(jié)點(diǎn),各節(jié)點(diǎn)對應(yīng)關(guān)系：

1.node1:7000(主)---》node4:7000(從)
2.node2:7000(主)---》node5:7000(從)
3.node3:7000(主)---》 node6:7000(從)
[root@node1 bin]# ./redis-trib.rb check node1.7000
Invalid IP or Port (given as node1.7000) - use IP:Port format
[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id 3b485951a72e16133464585dc2920c10ce75967b  node4:7000 node1:7000
>>> Adding node node4:7000 to cluster node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node4:7000 to make it join the cluster.
Waiting for the cluster to join...
>>> Configure node as replica of node1:7000.
[OK] New node added correctly.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f  node5:7000 node2:7000
>>> Adding node node5:7000 to cluster node2:7000
>>> Performing Cluster Check (using node node2:7000)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node5:7000 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of node2:7000.
[OK] New node added correctly.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc  node6:7000 node3:7000
>>> Adding node node6:7000 to cluster node3:7000
>>> Performing Cluster Check (using node node3:7000)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
S: c09449b9dce5af4ac485ccedb664188229d75430 node5:7000
   slots: (0 slots) slave
   replicates b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   1 additional replica(s)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node6:7000 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of node3:7000.
[OK] New node added correctly.

[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
S: 7501954b635724d66285afbb95841f4b49df5645 node6:7000
   slots: (0 slots) slave
   replicates 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc
S: c09449b9dce5af4ac485ccedb664188229d75430 node5:7000
   slots: (0 slots) slave
   replicates b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   1 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

另外有需要云服務(wù)器可以了解下創(chuàng)新互聯(lián)scvps.cn，海內(nèi)外云服務(wù)器15元起步，三天無理由+7*72小時售后在線，公司持有idc許可證，提供“云服務(wù)器、裸金屬服務(wù)器、高防服務(wù)器、香港服務(wù)器、美國服務(wù)器、虛擬主機(jī)、免備案服務(wù)器”等云主機(jī)租用服務(wù)以及企業(yè)上云的綜合解決方案，具有“安全穩(wěn)定、簡單易用、服務(wù)可用性高、性價比高”等特點(diǎn)與優(yōu)勢，專為企業(yè)上云打造定制，能夠滿足用戶豐富、多元化的應(yīng)用場景需求。

文章標(biāo)題：redis-cluster部署及數(shù)據(jù)遷移-創(chuàng)新互聯(lián)
本文路徑：http://www.muchs.cn/article6/dcddog.html

成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián)，為您提供App設(shè)計、微信小程序、小程序開發(fā)、定制網(wǎng)站、虛擬主機(jī)、網(wǎng)站收錄

廣告

聲明：本網(wǎng)站發(fā)布的內(nèi)容（圖片、視頻和文字）以用戶投稿、用戶轉(zhuǎn)載內(nèi)容為主，如果涉及侵權(quán)請盡快告知，我們將會在第一時間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場，如需處理請聯(lián)系客服。電話：028-86922220；郵箱：631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載，或轉(zhuǎn)載時需注明來源： 創(chuàng)新互聯(lián)

網(wǎng)站托管運(yùn)營