南大通用GBase 8a集群监控所用磁盘空间使用情况

GBase 8a提供了监控集群安装的所在磁盘使用情况的元数据表,包括集群层的CLUSTER_DISK_USAGE_INFO和节点层的DISK_USAGE_INFO。

提醒

该功能计算的磁盘使用情况是精确的,其消耗的磁盘资源,随着文件的增多而增多,在几十万个表的级别,可能会消耗几分钟甚至更长时间的磁盘资源(看磁盘随机读取性能)。

所以实际环境,表非常多的情况下,建议自行统计df 目录的方式,获得所在磁盘的使用情况即可,不要精确到每个目录。

[gbase@gbase_rh7_001 ~]$ df /opt/gbase
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sda1       29340676 12357904  16982772  43% /
[gbase@gbase_rh7_001 ~]$
[gbase@gbase_rh7_001 ~]$ df /opt/gbase/gnode/userdata
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sda1       29340676 12357856  16982820  43% /
[gbase@gbase_rh7_001 ~]$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sda1       29340676 12357856  16982820  43% /
devtmpfs         3990588        0   3990588   0% /dev
tmpfs            4005244     4188   4001056   1% /dev/shm
tmpfs            4005244   222016   3783228   6% /run
tmpfs            4005244        0   4005244   0% /sys/fs/cgroup
tmpfs             801052       16    801036   1% /run/user/42
tmpfs             801052        0    801052   0% /run/user/0
[gbase@gbase_rh7_001 ~]$

DISK_USAGE_INFO、CLUSTER_DISK_USAGE_INFO

所在库为performance_schema,表结构如下

字段类型说明
HOSTvarchar(64)主机名
DIR_TYPEvarchar(16)目录分类,目前分成
datadir : 数据目录,对应userdata
logdir : 日志目录,对应log
gbase_cache_data : 对应tmpdata
PATHvarchar(512)具体的目录,相同类型,调度节点和计算节点目录是略有差别的。
DIR_SIZEbigint(16)目录占用的空间,实际使用情况,基本等于du的精确结果。
该统计会消耗磁盘资源,且文件越多,耗时越长。
FILESYTEMvarchar(128)所在磁盘的文件系统类型,比如xfs
SIZEbigint(16)所在磁盘的空间,单位字节
USEDbigint(16)所在磁盘的已用空间,单位字节
AVAILbigint(16)所在磁盘的空闲空间,单位字节
PCTvarchar(16)所在磁盘的空间使用比例
gbase> desc Cluster_disk_usage_info;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| HOST      | varchar(64)  | NO   |     |         |       |
| DIR_TYPE  | varchar(16)  | NO   |     |         |       |
| PATH      | varchar(512) | NO   |     |         |       |
| DIR_SIZE  | bigint(16)   | NO   |     | 0       |       |
| FILESYTEM | varchar(128) | NO   |     |         |       |
| SIZE      | bigint(16)   | NO   |     | 0       |       |
| USED      | bigint(16)   | NO   |     | 0       |       |
| AVAIL     | bigint(16)   | NO   |     | 0       |       |
| PCT       | varchar(16)  | NO   |     |         |       |
+-----------+--------------+------+-----+---------+-------+
9 rows in set (Elapsed: 00:00:00.01)

gbase> desc disk_usage_info;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| HOST      | varchar(64)  | NO   |     |         |       |
| DIR_TYPE  | varchar(16)  | NO   |     |         |       |
| PATH      | varchar(512) | NO   |     |         |       |
| DIR_SIZE  | bigint(16)   | NO   |     | 0       |       |
| FILESYTEM | varchar(128) | NO   |     |         |       |
| SIZE      | bigint(16)   | NO   |     | 0       |       |
| USED      | bigint(16)   | NO   |     | 0       |       |
| AVAIL     | bigint(16)   | NO   |     | 0       |       |
| PCT       | varchar(16)  | NO   |     |         |       |
+-----------+--------------+------+-----+---------+-------+
9 rows in set (Elapsed: 00:00:00.00)

输出样例

gbase> select * from  disk_usage_info;
+---------------+------------------+--------------------------------------------+----------+-----------+-------------+-------------+-------------+-----+
| HOST          | DIR_TYPE         | PATH                                       | DIR_SIZE | FILESYTEM | SIZE        | USED        | AVAIL       | PCT |
+---------------+------------------+--------------------------------------------+----------+-----------+-------------+-------------+-------------+-----+
| gbase_rh7_001 | datadir          | /opt/gbase/gcluster/userdata/gcluster/     |  7081984 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
| gbase_rh7_001 | gbase_cache_data | /opt/gbase/gcluster/tmpdata/cache_gcluster |        0 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
| gbase_rh7_001 | logdir           | /opt/gbase/gcluster/log/gcluster/          | 15941632 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
+---------------+------------------+--------------------------------------------+----------+-----------+-------------+-------------+-------------+-----+
3 rows in set (Elapsed: 00:00:00.02)

gbase> select * from  Cluster_disk_usage_info;
+---------------+------------------+--------------------------------------------+-----------+-----------+-------------+-------------+-------------+-----+
| HOST          | DIR_TYPE         | PATH                                       | DIR_SIZE  | FILESYTEM | SIZE        | USED        | AVAIL       | PCT |
+---------------+------------------+--------------------------------------------+-----------+-----------+-------------+-------------+-------------+-----+
| gbase_rh7_001 | datadir          | /opt/gbase/gnode/userdata/gbase/           | 170815488 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
| gbase_rh7_001 | gbase_cache_data | /opt/gbase/gnode/tmpdata/cache_gbase       |         0 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
| gbase_rh7_001 | logdir           | /opt/gbase/gnode/log/gbase/                |    266240 | rootfs    | 30044852224 | 12711493632 | 17333358592 | 42% |
| gbase_rh7_001 | datadir          | /opt/gbase/gcluster/userdata/gcluster/     |   7081984 | rootfs    | 30044852224 | 12711518208 | 17333334016 | 42% |
| gbase_rh7_001 | gbase_cache_data | /opt/gbase/gcluster/tmpdata/cache_gcluster |         0 | rootfs    | 30044852224 | 12711518208 | 17333334016 | 42% |
| gbase_rh7_001 | logdir           | /opt/gbase/gcluster/log/gcluster/          |  15941632 | rootfs    | 30044852224 | 12711518208 | 17333334016 | 42% |
+---------------+------------------+--------------------------------------------+-----------+-----------+-------------+-------------+-------------+-----+
6 rows in set (Elapsed: 00:00:00.02)