本文介绍GBase 8a的分布策略distribution的不同方案pattern的结果样例,方便用户创建高可用程度更高的集群。所有的策略,都是为了在节点故障时,更合理有效的利用副本。
目录导航
环境
我们准备了9个IP,分成3组(rack),代表3个机柜,看每种策略的主备策略的结果。
gcChangeInfo.xml
<?xml version="1.0" encoding="utf-8"?>
<servers>
<rack>
<node ip="10.0.2.103"/>
<node ip="10.0.2.104"/>
<node ip="10.0.2.105"/>
</rack>
<rack>
<node ip="10.0.2.106"/>
<node ip="10.0.2.107"/>
<node ip="10.0.2.108"/>
</rack>
<rack>
<node ip="10.0.2.109"/>
<node ip="10.0.2.110"/>
<node ip="10.0.2.111"/>
</rack>
</servers>
parttern 1负载均衡默认
目标:
- 任意一台服务器离线
- 任意一个rack整体离线
- 任意横向同层(同上下位置)的机器离线
其负载要均衡在不同的机器上;
第一个分片的副本1:放下一个rack的下一个位置,副本2,则放在上一个rack的上一个位置。
第二个分片的副本1:放下一个rack的下下一个位置,副本2,则放在上一个rack的上上一个位置。
第三个分片的副本1:放下一个rack的下下下一个位置,副本2,则放在上一个rack的上上上一个位置。
以此类推
在考虑副本前提下,一个主分片的副本,必须落在第2台机器上,第二个主分片,必须放在第三台机器上。所以主分片数量必须小于rack内的节点数量。
同时建议不同rack内机器数量相同,否则都是10个节点的rack,有一个rack是3个节点。当10节点rack故障时,3节点的rack负载必然比别的rack的高。如果不同rack无法整齐,不如用pattern 2或者自定义。
注意:parrten 1时,建议不要使用单rack,无意义。
1个副本的1+1模式
副本位置是下一个rack的下一个位置,比如
- rack1的节点1,副本1放到rack2的节点2
- rack1的节点2,副本1放到rack2的节点3
- 同理
- rack2的节点3,副本1放到rack3的节点4
如果节点超过了末尾,则转到开头,比如rack之后3个节点,那么
- rack1的节点3,副本1放到rack2的节点1(回到开头)
实际样例
[gbase@gbase_rh7_003 gcinstall]$ gcadmin distribution gcChangeInfo.xml p 1 d 1 pattern 1
gcadmin generate distribution ...
NOTE: node [10.0.2.103] is coordinator node, it shall be data node too
gcadmin generate distribution successful
[gbase@gbase_rh7_003 gcinstall]$ gcadmin showdistribution
Distribution ID: 1 | State: new | Total segment num: 9
Primary Segment Node IP Segment ID Duplicate Segment node IP
========================================================================================================================
| 10.0.2.103 | 1 | 10.0.2.107 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.104 | 2 | 10.0.2.108 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.105 | 3 | 10.0.2.106 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.106 | 4 | 10.0.2.110 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.107 | 5 | 10.0.2.111 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.108 | 6 | 10.0.2.109 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.109 | 7 | 10.0.2.104 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.110 | 8 | 10.0.2.105 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.111 | 9 | 10.0.2.103 |
========================================================================================================================
2个副本的1+2模式
副本2,则放在上一个rack的上一个位置,如果小于第一个,则返回末尾一个。
[gbase@gbase_rh7_003 gcinstall]$ gcadmin distribution gcChangeInfo.xml p 1 d 2 pattern 1
gcadmin generate distribution ...
gcadmin generate distribution successful
[gbase@gbase_rh7_003 gcinstall]$ gcadmin showdistribution
Distribution ID: 2 | State: new | Total segment num: 9
Primary Segment Node IP Segment ID Duplicate Segment node IP
========================================================================================================================
| 10.0.2.103 | 1 | 10.0.2.107 |
| | | 10.0.2.111 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.104 | 2 | 10.0.2.108 |
| | | 10.0.2.109 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.105 | 3 | 10.0.2.106 |
| | | 10.0.2.110 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.106 | 4 | 10.0.2.110 |
| | | 10.0.2.105 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.107 | 5 | 10.0.2.111 |
| | | 10.0.2.103 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.108 | 6 | 10.0.2.109 |
| | | 10.0.2.104 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.109 | 7 | 10.0.2.104 |
| | | 10.0.2.108 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.110 | 8 | 10.0.2.105 |
| | | 10.0.2.106 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.111 | 9 | 10.0.2.103 |
| | | 10.0.2.107 |
========================================================================================================================
如果只有2个rack,采用了2副本
副本1不变,下一个rack的下一个位置,副本2则变成了本rack的下下一个位置,也就是位置+2。如下是2个rack的样例。
gcChangeInfo_rack2.xml
[gbase@gbase_rh7_003 gcinstall]$ cat gcChangeInfo_rack2.xml
<?xml version="1.0" encoding="utf-8"?>
<servers>
<rack>
<node ip="10.0.2.103"/>
<node ip="10.0.2.104"/>
<node ip="10.0.2.105"/>
<node ip="10.0.2.106"/>
</rack>
<rack>
<node ip="10.0.2.107"/>
<node ip="10.0.2.108"/>
<node ip="10.0.2.109"/>
<node ip="10.0.2.110"/>
</rack>
结果样例
[gbase@gbase_rh7_003 gcinstall]$ gcadmin distribution gcChangeInfo_rack2.xml p 1 d 2 pattern 1
gcadmin generate distribution ...
gcadmin generate distribution successful
[gbase@gbase_rh7_003 gcinstall]$ gcadmin showdistribution
Distribution ID: 4 | State: new | Total segment num: 8
Primary Segment Node IP Segment ID Duplicate Segment node IP
========================================================================================================================
| 10.0.2.103 | 1 | 10.0.2.108 |
| | | 10.0.2.105 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.104 | 2 | 10.0.2.109 |
| | | 10.0.2.106 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.105 | 3 | 10.0.2.110 |
| | | 10.0.2.103 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.106 | 4 | 10.0.2.107 |
| | | 10.0.2.104 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.107 | 5 | 10.0.2.104 |
| | | 10.0.2.109 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.108 | 6 | 10.0.2.105 |
| | | 10.0.2.110 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.109 | 7 | 10.0.2.106 |
| | | 10.0.2.107 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.110 | 8 | 10.0.2.103 |
| | | 10.0.2.108 |
pattern 2 高可用模式
该模式忽视rack,按照出现先后顺序,副本1在下一个节点,副本2在上一个节点。如果超过末尾,则返回第一个,如果低于第一个,则返回末尾。
无论多少个主分片,副本1都在下一个节点,副本2都在上一个节点。
如果有多个分片,副本策略同第一个相同,也就是无论多少个分片,其第一个副本都在下一个节点,第二个副本在下下一个。
1主分片+2副本输出样例
[gbase@gbase_rh7_003 gcinstall]$ gcadmin distribution gcChangeInfo.xml p 1 d 2 pattern 2
gcadmin generate distribution ...
NOTE: node [10.0.2.103] is coordinator node, it shall be data node too
gcadmin generate distribution successful
[gbase@gbase_rh7_003 gcinstall]$ gcadmin showdistribution
Distribution ID: 3 | State: new | Total segment num: 9
Primary Segment Node IP Segment ID Duplicate Segment node IP
========================================================================================================================
| 10.0.2.103 | 1 | 10.0.2.104 |
| | | 10.0.2.111 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.104 | 2 | 10.0.2.105 |
| | | 10.0.2.103 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.105 | 3 | 10.0.2.106 |
| | | 10.0.2.104 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.106 | 4 | 10.0.2.107 |
| | | 10.0.2.105 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.107 | 5 | 10.0.2.108 |
| | | 10.0.2.106 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.108 | 6 | 10.0.2.109 |
| | | 10.0.2.107 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.109 | 7 | 10.0.2.110 |
| | | 10.0.2.108 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.110 | 8 | 10.0.2.111 |
| | | 10.0.2.109 |
------------------------------------------------------------------------------------------------------------------------
| 10.0.2.111 | 9 | 10.0.2.103 |
| | | 10.0.2.110 |
========================================================================================================================
多主分片 2分片+2副本
[gbase@localhost gcinstall_9524410p5]$ cat gcChangeInfo.xml
<?xml version="1.0" encoding="utf-8"?>
<servers>
<rack>
<node ip="10.0.2.102"/>
<node ip="10.0.2.202"/>
<node ip="10.0.2.203"/>
</rack>
</servers>
[gbase@localhost gcinstall_9524410p5]$ gcadmin showdistribution
Distribution ID: 2 | State: new | Total segment num: 6
Primary Segment Node IP Segment ID Duplicate Segment node IP
====================================================================================
| 10.0.2.102 | 1 | 10.0.2.202 |
| | | 10.0.2.203 |
------------------------------------------------------------------------------------
| 10.0.2.202 | 2 | 10.0.2.203 |
| | | 10.0.2.102 |
------------------------------------------------------------------------------------
| 10.0.2.203 | 3 | 10.0.2.102 |
| | | 10.0.2.202 |
------------------------------------------------------------------------------------
| 10.0.2.102 | 4 | 10.0.2.202 |
| | | 10.0.2.203 |
------------------------------------------------------------------------------------
| 10.0.2.202 | 5 | 10.0.2.203 |
| | | 10.0.2.102 |
------------------------------------------------------------------------------------
| 10.0.2.203 | 6 | 10.0.2.102 |
| | | 10.0.2.202 |
====================================================================================
总结
为了最大限度的高可用,建议不小于3个机柜,避免某个机柜整体出问题,导致集群整体不可用的情况出现。