impala添加kerberos认证

impala添加kerberos认证
- 背景
- 环境
- 配置
- - kerberos操作
  - 集群配置
- 问题
- - impalad启动报错
  - catalog启动报错
- 注意
- 参考

impala添加kerberos认证

背景

公司测试集群需要配置impala+kerberos，但是测试集群很乱，很多人用，用户还有权限比较混乱，而且是ambari HDP的集群。

环境

需要kerbreos安装参考这里
大数据环境：HDP-3.1.4.0
hdfs yarn hive这些都需要有

配置

kerberos操作

1.创建kerberos Impala 服务主体，指定运行 Impala 守护程序的操作系统用户的名称、运行 impalad的每个节点的完全限定域名以及领域名称。例如：

$ kadmin
kadmin: addprinc -requires_preauth -randkey impala/impala_host.example.com@TEST.EXAMPLE.COM

实际执行

kadmin.local -q "addprinc -requires_preauth -randkey impala/slave2.am.com@AM.COM"

2.创建 HTTP 服务主体。例如：

kadmin: addprinc -randkey HTTP/impala_host.example.com@TEST.EXAMPLE.COM

实际执行

kadmin.local -q "addprinc -randkey HTTP/slave2.am.com@AM.COM"

注意：服务主体的HTTP组件必须为大写，如上例所示。
3.keytab使用两个主体创建文件。例如：

kadmin: xst -k impala.keytab impala/impala_host.example.com
kadmin: xst -k http.keytab HTTP/impala_host.example.com
kadmin: quit

实际执行

[root@slave2 ~]# kadmin -padmin/admin -wadmin -q"xst -k impala.keytab impala/slave2.am.com"                         Authenticating as principal admin/admin with password.
Entry for principal impala/slave2.am.com with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type des3-cbc-sha1 added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type arcfour-hmac added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type camellia256-cts-cmac added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type camellia128-cts-cmac added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type des-hmac-sha1 added to keytab WRFILE:impala.keytab.
Entry for principal impala/slave2.am.com with kvno 3, encryption type des-cbc-md5 added to keytab WRFILE:impala.keytab.
[root@slave2 ~]# kadmin -padmin/admin -wadmin -q"xst -k http.keytab HTTP/slave2.am.com"
Authenticating as principal admin/admin with password.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type des3-cbc-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type arcfour-hmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type camellia256-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type camellia128-cts-cmac added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type des-hmac-sha1 added to keytab WRFILE:http.keytab.
Entry for principal HTTP/slave2.am.com with kvno 4, encryption type des-cbc-md5 added to keytab WRFILE:http.keytab.

4.用于ktutil读取两个 keytab 文件的内容，然后将这些内容写入新文件。例如：

$ ktutil
ktutil: rkt impala.keytab
ktutil: rkt http.keytab
ktutil: wkt impala-http.keytab
ktutil: quit

实际执行一致

5.（可选）测试合并的 keytab 文件中的凭据是否有效，以及 “续订截止日期”是否在将来。例如：

$ klist -e -k -t impala-http.keytab

6.将impala-http.keytab文件复制到 Impala 配置目录。将权限更改为仅供文件所有者读取，并将文件所有者更改为impala用户。默认情况下，Impala 用户和组都命名为impala. 例如：

$ cp impala-http.keytab /etc/impala/conf
$ cd /etc/impala/conf
$ chmod 400 impala-http.keytab
$ chown impala:impala impala-http.keytab

实际执行

$ cp impala-http.keytab /etc/impala/conf
$ cd /etc/impala/conf
#400这个我没执行也没问题
$ chmod 400 impala-http.keytab
$ chown impala:impala impala-http.keytab

7.将 Kerberos 选项添加到 Impala 默认文件 /etc/default/impala。使用和变量为impalad和statestored守护进程添加选项。例如，您可以添加：

IMPALA_SERVER_ARGS IMPALA_STATE_STORE_ARGS
-kerberos_reinit_interval=60
-principal=impala_1/impala_host.example.com@TEST.EXAMPLE.COM
-keytab_file=/path/to/impala.keytab

实际完整配置文件/etc/default/impala

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}  -state_store_host=${IMPALA_STATE_STORE_HOST}  -kerberos_reinit_interval=60 -principal=impala/master.am.com@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT} -kerberos_reinit_interval=60 -principal=impala/master.am.com@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_SERVER_ARGS=" \-log_dir=${IMPALA_LOG_DIR} \-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \-state_store_port=${IMPALA_STATE_STORE_PORT} \-use_statestore=true \-state_store_host=${IMPALA_STATE_STORE_HOST} \-be_port=${IMPALA_BACKEND_PORT} \-kudu_master_hosts=10.1.251.124:7051 \-kerberos_reinit_interval=60 \-principal=impala/master.am.com@AM.COM \-keytab_file=/etc/impala/conf/impala-http.keytab"ENABLE_CORE_DUMPS=false# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf
# HBASE_CONF_DIR=/etc/impala/conf

有关更改 /etc/default/impala中指定的 Impala 默认值的更多信息，请参阅修改 Impala 启动选项。

注意：重新启动impalad和statestored以使这些配置更改生效。

8.重启impala服务

service impala-state-store restart
service impala-catalog restart
service impala-server restart

9启动impala-shell -k验证一下

[root@master ~]# impala-shell -k
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Connected to master.am.com:21000
Server version: impalad version 2.7.0-IMPALA_KUDU-cdh5 RELEASE (build 10d4ebec3c23961218e972e74e9d342ffc417af1)
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights reserved.
(Impala Shell v2.6.0-cdh5.8.0 (8d8652f) built on Tue Jul 12 15:43:17 PDT 2016)You can change the Impala daemon that you're connected to by using the CONNECT
command.To see how Impala will plan to run your query without actually executing
it, use the EXPLAIN command. You can change the level of detail in the EXPLAIN
output by setting the EXPLAIN_LEVEL query option.
***********************************************************************************
[master.am.com:21000] > create table jzy (id int);

集群配置

/etc/default/impala中可以使用_HOST代替当前主机，方便集群统一配置例如：

-principal=impala/_HOST@AM.COM -keytab_file=/etc/impala/conf/impala-httpx.keytab

实际配置

IMPALA_CATALOG_SERVICE_HOST=10.1.x.x
IMPALA_STATE_STORE_HOST=10.1.x.x
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impalaIMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}  -state_store_host=${IMPALA_STATE_STORE_HOST}  -kerberos_reinit_interval=60 -principal=impala/_HOST@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT} -kerberos_reinit_interval=60 -principal=impala/_HOST@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_SERVER_ARGS=" \-log_dir=${IMPALA_LOG_DIR} \-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \-state_store_port=${IMPALA_STATE_STORE_PORT} \-use_statestore=true \-state_store_host=${IMPALA_STATE_STORE_HOST} \-be_port=${IMPALA_BACKEND_PORT} \-kudu_master_hosts=10.1.251.124:7051 \-kerberos_reinit_interval=60 \-principal=impala/_HOST@AM.COM \-keytab_file=/etc/impala/conf/impala-http.keytab"ENABLE_CORE_DUMPS=false# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf

问题

impalad启动报错

报错：

Failed to obtain Kerberos ticket for principal: root/master.am.com@AM.COM. Shell cmd: 'kinit -k -t /etc/impala/conf/root-http.keytab root/master.am.com@AM.COM 2>&1' exited with error status: '1'. Stdout was: 'kinit: Permission denied while getting initial credentials
'
. Impalad exiting.
*** Check failure stack trace: ***@          0x1b4a2ad  (unknown)@          0x1b4cbd6  (unknown)@          0x1b49dcd  (unknown)@          0x1b4d67e  (unknown)@           0x82b37a  (unknown)@           0xb1e2f0  (unknown)@           0x7cfb23  (unknown)@     0x7f0aba100555  __libc_start_main@           0x80068d  (unknown)
Wrote minidump to /var/log/impala/minidumps/impalad/6cb16f39-930c-2056-76702486-2a8c8d51.dmp

解决

chown impala:impala /etc/impala/conf/impala-http.keytab

报错：

E0923 15:32:52.320331 3788567 logging.cc:121] stderr will be logged to this file.
F0923 15:32:52.332217 3788567 init.cc:197] Kerberos principal should be of the form: <service>/<hostname>@<realm> - got: impala@AM.COM
. Impalad exiting.
*** Check failure stack trace: ***@          0x1b4a2ad  (unknown)@          0x1b4cbd6  (unknown)@          0x1b49dcd  (unknown)@          0x1b4d67e  (unknown)@           0x82b37a  (unknown)@           0xb1e2f0  (unknown)@           0x7cfb23  (unknown)@     0x7f1c1294e555  __libc_start_main@           0x80068d  (unknown)

解决：创建kerberos Impala 服务主体和生成的keytab文件必须要有节点的完全限定域名以及领域名称例如impala/impala_host.example.com@TEST.EXAMPLE.COM
按要求执行配置中1到6步即可

catalog启动报错

E0923 15:48:13.755832 3809847 authentication.cc:160] SASL message (Kerberos (internal)): GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server impala/localhost@AM.COM not found in Kerberos database)
E0923 15:48:16.762388 3809847 authentication.cc:160] SASL message (Kerberos (internal)): GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Server impala/localhost@AM.COM not found in Kerberos database)
F0923 15:48:19.763552 3809847 catalogd-main.cc:80] RPC Error: No more data to read.
. Impalad exiting.
*** Check failure stack trace: ***@          0x1b4a2ad  (unknown)@          0x1b4cbd6  (unknown)@          0x1b49dcd  (unknown)@          0x1b4d67e  (unknown)@           0x801854  (unknown)@           0x7cfb16  (unknown)@     0x7f268aa67555  __libc_start_main@           0x80068d  (unknown)

解决修改/etc/default/impala
原来：

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}"

修改后：

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}  -state_store_host=${IMPALA_STATE_STORE_HOST}  -kerberos_reinit_interval=60 -principal=impala/master.am.com@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"

报错：

E0923 15:59:00.070447 3825300 TSaslTransport.java:296] SASL negotiation failure
Java exception follows:
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:430)
......

注意

1.创建kerberos Impala 服务主体和生成的keytab文件必须要有节点的完全限定域名以及领域名称例如impala/impala_host.example.com@TEST.EXAMPLE.COM
2./etc/default/impala配置中IMPALA_CATALOG_ARGS
需要有 -state_store_host=${IMPALA_STATE_STORE_HOST}否则启动catalog会报错

TSaslTransport.java:296] SASL negotiation failure
Java exception follows:
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

3.修改/etc/default/impala配置后
注意：重新启动impalad和statestored以使这些配置更改生效。

参考

impala官网
kerberos配置