南大通用GCDW技术栈-kerberos的搭建和凭证生成

发表于 作者 laozizhu

GCDW除了支持S3外，新版本还支持hdfs作为存储。本文介绍hadoop安全模式下kerberos的配置和凭证的生成。

参考

GCDW技术栈-hadoop+kerberos配置

概念

KDC（Key Distribution Centor）

验证各个模块，是统一认证服务

Principal

账号

Keytab

包含一个或多个账号和密码的文件

Relam

域，简单可以理解成域名，或者组。

Authentication Sever（AS）

用于初始化认证，并生成Ticket Granting Ticket（TGT）

Ticket Granting Server（TGS）

在TGT的基础上生成Service Ticket

认证流程

客户端会先访问两次KDC，然后再访问目标Service，如：HTTP服务

安装

yum -y install krb5-server krb5-libs krb5-workstation

[root@hadoop141 hadoop]# yum list installed | grep kr
krb5-libs.x86_64                            1.15.1-55.el7_9            @updates
krb5-server.x86_64                          1.15.1-55.el7_9            @updates
krb5-workstation.x86_64                     1.15.1-55.el7_9

配置

Kerberos的配置 /etc/krb5.conf

包含KDC的位置，Kerberos的admin的realms 等。需要所有使用的Kerberos的机器上的配置文件都同步。

默认在/etc目录下，可以通过环境变量 KRB5_CONFIG 指定。该环境变量可以指定多个冒号分隔的文件名，所有存在的都将被读取。

Normally, you should install your krb5.conf file in the directory /etc. You can override the default location by setting the environment variable KRB5_CONFIG. Multiple colon-separated filenames may be specified in KRB5_CONFIG; all files which are present will be read.

配置样例

[root@hadoop141 opt]# cat /etc/krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 renewable = true
# rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
 default_realm = GCDW
# default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 GCDW = {
  kdc = hadoop141
  admin_server = hadoop141
 }

[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM

主要变动内容，是realms部分。默认是EXAMPLE.COM, 我这里改成了GCDW。

rdns

这个默认是true. 在部分文章里，说如果出现主机名多个Ip时，建议关掉。 这里测试发现，在1.15.1-55版本，如果设置成false, 会出现GCDW无法正确连接hadooop的问题（Access denied to remote resource, HTTP/1.1 401 Authentication required get kerberos token failed !）。

default_realm

libdefaults部分的默认域default_realm设置成自己的GCDW

kdc、admin_server

KDC和admin_server我这里全部用的本机(hadoop141),也就是和hadoop部署在了一起了。

KDC的配置文件 /var/kerberos/krb5kdc/kdc.conf

需要改的也是realms部分，从EXAMPLA.COM改成自己的域GCDW。

该配置默认在系统的/var下面。如果要修改位置，可以用 KRB5_KDC_PROFILE 环境变量指定。

Normally, the kdc.conf file is found in the KDC state directory, LOCALSTATEDIR/krb5kdc. You can override the default location by setting the environment variable KRB5_KDC_PROFILE.
Please note that you need to restart the KDC daemon for any configuration changes to take effect.

也可以在krb5.conf的kdc部分通过profile指定，类似如下

在krb5.conf的kdc部分通过profile指定
[root@hadoop141 opt]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 GCDW = {
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }
[root@hadoop141 opt]#

KDC服务的权限管理文件 /var/kerberos/krb5kdc/kadm5.acl

同样的，改成自定义的GCDW。位置在前一个配置文件里设置的。

[root@hadoop141 opt]# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@GCDW    *
[root@hadoop141 opt]#

主机DNS /etc/hosts

我这里和hadoop复用了。

[root@hadoop141 opt]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.2.141  hadoop141

初始化KDC数据库

通过kdb5_util create创建KDC数据库，密码要输入2次。

[root@mdw ~]# kdb5_util create -r GCDW -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'GCDW',
master key name 'K/M@GCDW'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:
[root@mdw ~]#

生成的文件如下


[root@hadoop141 ~]# ll /var/kerberos/krb5kdc/
total 24
-rw------- 1 root root   15 Feb 28 09:09 kadm5.acl
-rw------- 1 root root  444 Feb 28 09:08 kdc.conf
-rw------- 1 root root 8192 Feb 28 09:14 principal              #Kerberos 数据库文件
-rw------- 1 root root 8192 Feb 28 09:09 principal.kadm5        #Kerberos 管理数据库
-rw------- 1 root root    0 Feb 28 09:09 principal.kadm5.lock   #Kerberos 管理数据库锁文件
-rw------- 1 root root    0 Feb 28 09:14 principal.ok           #Kerberos 数据库文件

服务启停

status/start/stop/restart标准的systemctl命令。

systemctl start krb5kdc.service
systemctl start kadmin.service

建议加入开机自启动

systemctl enable krb5kdc.service
systemctl enable kadmin.service

默认KDC日志

/var/log/krb5kdc.log
/var/log/kadmind.log

[root@vm102 ~]# ll /var/log/k*
-rw------- 1 root root   0 Mar 15 12:38 /var/log/kadmind.log
-rw-r----- 1 root root 720 Mar 15 12:38 /var/log/krb5kdc.log
[root@vm102 ~]#

KDC 服务器上添加超级管理员账户

通过kadmin.local的addprinc创建管理员账户

[root@mdw ~]# kadmin.local -q "addprinc root/admin"
Authenticating as principal root/admin@GBASE8A with password.
WARNING: no policy specified for root/admin@GBASE8A; defaulting to no policy
Enter password for principal "root/admin@GBASE8A":
Re-enter password for principal "root/admin@GBASE8A":
Principal "root/admin@GBASE8A" created.
[root@mdw ~]#

导出krb5.keytab

[root@hadoop141 ~]# kadmin.local -q "ktadd root/admin"
Authenticating as principal root/admin@GCDW with password.
Entry for principal root/admin with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type des3-cbc-sha1 added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type arcfour-hmac added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type camellia256-cts-cmac added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type camellia128-cts-cmac added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type des-hmac-sha1 added to keytab FILE:/etc/krb5.keytab.
Entry for principal root/admin with kvno 2, encryption type des-cbc-md5 added to keytab FILE:/etc/krb5.keytab.
[root@hadoop141 ~]# ll /etc/krb5.keytab
-rw------- 1 root root 490 Mar  4 09:41 /etc/krb5.keytab
[root@hadoop141 ~]# klist -kt /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
   2 03/04/2024 09:41:58 root/admin@GCDW
[root@hadoop141 ~]#

hadoop+kerberos的凭据

如下是hadoop141,hadoop142两节点hadoop集群所需kerberos凭证的创建过程。包含了2类用户hdfs和HTTP。

创建hdfs的principal主体

kadmin.local -q "listprincs"
kadmin.local -q "addprinc -randkey hdfs/hadoop141@GCDW"
kadmin.local -q "addprinc -randkey hdfs/hadoop142@GCDW"
kadmin.local -q "addprinc -randkey HTTP/hadoop141@GCDW"
kadmin.local -q "addprinc -randkey HTTP/hadoop142@GCDW"
kadmin.local -q "listprincs"

运行记录

[root@hadoop141 ~]# kadmin.local -q "addprinc -randkey hdfs/hadoop141@GCDW"
Authenticating as principal root/admin@GCDW with password.
WARNING: no policy specified for hdfs/hadoop141@GCDW; defaulting to no policy
Principal "hdfs/hadoop141@GCDW" created.
[root@hadoop141 ~]# kadmin.local -q "addprinc -randkey hdfs/hadoop142@GCDW"
Authenticating as principal root/admin@GCDW with password.
WARNING: no policy specified for hdfs/hadoop142@GCDW; defaulting to no policy
Principal "hdfs/hadoop142@GCDW" created.
[root@hadoop141 ~]# kadmin.local -q "listprincs"
Authenticating as principal root/admin@GCDW with password.
K/M@GCDW
hdfs/hadoop141@GCDW
hdfs/hadoop142@GCDW
kadmin/admin@GCDW
kadmin/changepw@GCDW
kadmin/hadoop141@GCDW
kiprop/hadoop141@GCDW
krbtgt/GCDW@GCDW
root/admin@GCDW
[root@hadoop141 ~]#

服务端创建keytab文件用于服务通过kerberos认证时免密

kadmin.local -q "xst -k hdfs-unmerged.keytab hdfs/hadoop141@GCDW"
kadmin.local -q "xst -k hdfs-unmerged.keytab hdfs/hadoop142@GCDW"
kadmin.local -q "xst -k http.keytab HTTP/hadoop141@GCDW"
kadmin.local -q "xst -k http.keytab HTTP/hadoop142@GCDW"

[root@hadoop141 ~]# kadmin.local -q "xst -k hdfs-unmerged.keytab hdfs/hadoop141@GCDW"
Authenticating as principal root/admin@GCDW with password.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop141@GCDW with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:hdfs-unmerged.keytab.
[root@hadoop141 ~]# kadmin.local -q "xst -k hdfs-unmerged.keytab hdfs/hadoop142@GCDW"
Authenticating as principal root/admin@GCDW with password.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:hdfs-unmerged.keytab.
Entry for principal hdfs/hadoop142@GCDW with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:hdfs-unmerged.keytab.
[root@hadoop141 ~]# kadmin.local -q "xst -k http.keytab HTTP/hadoop141@GCDW"
Authenticating as principal root/admin@GCDW with password.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop141@GCDW with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:http.keytab.
[root@hadoop141 ~]# kadmin.local -q "xst -k http.keytab HTTP/hadoop142@GCDW"
Authenticating as principal root/admin@GCDW with password.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type camellia256-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type camellia128-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/hadoop142@GCDW with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:http.keytab.
[root@hadoop141 ~]#

用kutil工具将生成的keytab文件合并成一个hdfs.keytab

[root@hadoop141 ~]# ktutil
ktutil:  rkt hdfs-unmerged.keytab
ktutil:  rkt http.keytab
ktutil:  wkt hdfs.keytab
ktutil:  exit
[root@hadoop141 ~]# klist -kt hdfs.keytab
Keytab name: FILE:hdfs.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop141@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 hdfs/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop141@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
   2 03/04/2024 09:54:19 HTTP/hadoop142@GCDW
[root@hadoop141 ~]#
[root@hadoop141 ~]# kinit -k -t hdfs.keytab hdfs/hadoop141@GCDW
[root@hadoop141 ~]# kinit -k -t hdfs.keytab HTTP/hadoop141@GCDW
[root@hadoop141 ~]# klist
Ticket cache: KEYRING:persistent:0:krb_ccache_Gg1tLQo
Default principal: HTTP/hadoop141@GCDW

Valid starting       Expires              Service principal
03/04/2024 09:57:15  03/05/2024 09:57:15  krbtgt/GCDW@GCDW
[root@hadoop141 ~]#

分发

将hdfs.keytab分发到所有需要凭据的hadoop节点上，并配置hadoop对应的凭据和用户。

Post Views: 122

相关文章:

  1. 南大通用GCDW技术栈-hadoop+kerberos配置
  2. 南大通用GBase 的GCDW 云数仓是什么产品？
  3. 南大通用GCDW技术栈-GCDW物理机版本使用hdfs作为存储的安装
  4. Linux 开启IPV6协议的相关参数
  5. 基于南大通用 GBase 8a MPP Cluster* 的 英特尔® 精选解决方案
  6. redhat 7 / Centos 7 配置多个IP地址的方法
  7. Redhat 6 修改主机名hostname
  8. ssh 在安全加固场景下需要增加额外参数进行远程访问
  9. 虚拟化，超融合的关系和区别以及主流产品介绍
  10. redhat7 升级openssh 8.5 p1手顺