Sentry手动安装、使用手册

1 Sentry简介

Apache Sentry 是Cloudera公司发布的一个Hadoop开源组件，截止目前还是Apache的孵化项目，它提供了细粒度级、基于角色的授权以及多租户的管理模式。Sentry当前可以和Hive/Hcatalog、Apache Solr 和Cloudera Impala集成，未来会扩展到其他的Hadoop组件，例如HDFS和HBase.

2 准备

2.1 环境说明

1 Sentry安装采用rpm包的方式.

2 Hadoop版本为hadoop-2.5.0-cdh5.3.3,Hive版本为hive-0.13.1-cdh5.3.3,Sentry版本为sentry-1.4.0-cdh5.3.3

3 Sentry下载地址:

http://archive-primary.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.3.3/RPMS/noarch/

2.1.1虚拟机里Linux系统版本

[root@localhost ranger-0.5.0-usersync]# cat /etc/issue | grep Linux

Red Hat Enterprise Linux Server release 6.5 (Santiago)

2.1.2 JDK版本

[root@localhost native]# java -version

java version "1.7.0_67"

Java(TM) SE RuntimeEnvironment (build 1.7.0_67-b01)

Java HotSpot(TM) 64-BitServer VM (build 24.65-b04, mixed mode)

2.1.3mysql版本

[root@localhost native]# mysql -uroot -proot-e"select version()";

Warning: Using a password onthe command line interface can be insecure.

1 Mysql 驱动为mysql-connector-java-5.1.31-bin.jar

2 改jar被重命名后放置在/usr/share/java/内被其它Ranger插件共享

3 安装

3.1安装Mysql

rpm -ivh MySQL-server-5.6.14-1.el6.x86_64.rpm时报如下错误:

file/usr/share/mysql/charsets/macroman.xml from install of MySQL-server-5.6.14-1.el6.x86_64conflicts with file from package mysql-libs-5.1.71-1.el6.x86_64 file/usr/share/mysql/charsets/swe7.xml from install ofMySQL-server-5.6.14-1.el6.x86_64 conflicts with file from packagemysql-libs-5.1.71-1.el6.x86_64

rpm -q mysql-libs-5.1.71-1.el6.x86_64

rpm -e --nodeps mysql-libs-5.1.71-1.el6.x86_64

rpm -ivh MySQL-server-5.6.14-1.el6.x86_64.rpm

ARANDOM PASSWORD HAS BEEN SET FOR THE MySQL root USER !

Youwill find that password in '/root/.mysql_secret'.(生成mysql root密码)

Youmust change that password on your first connect,

noother statement but 'SET PASSWORD' will be accepted.

rpm -ivh MySQL-client-5.6.14-1.el6.x86_64.rpm

service mysql start

#这里密码来自/root/.mysql_secret

mysql -uroot -p9RNrbk9O

#首次执行会提示修改mysql root密码

SET PASSWORD=PASSWORD('root');

#创建Hive数据库为Hive的元数据库

create database hive;

GRANT all ON hive.* TO root@'%' IDENTIFIED BY 'root';

#创建sentry数据库为sentry元数据库

create database sentry;

CREATE USER sentry IDENTIFIED BY 'sentry';

GRANT all ON sentry.* TO sentry@'%' IDENTIFIED BY'sentry';

flush privileges;

3.2 安装Hive

1)解压Hive压缩包,并配置环境变量

cd /root

tar –zxvf hive-0.13.1-cdh5.3.3.tar.gz

vi ~/.bash_profile

exportHIVE_HOME=/root/hive-0.13.1-cdh5.3.3

#追加HIVE执行文件路径到PATH内

exportPATH=$PATH:$HIVE_HOME/bin

2) 拷贝Mysql驱动到Hive的lib目录下:

cp /root/mysql-connector-java-5.1.31-bin.jar/root/hive-0.13.1-cdh5.3.3/lib/mysql-connector-java-5.1.31-bin.jar

3)配置Hive的conf.详细的hive-site.xml配置内容见下:

<?xmlversion="1.0"?>
<?xml-stylesheettype="text/xsl" href="configuration.xsl"?>
<configuration>
<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value><description>JDBC connect string for aJDBC metastore</description>
</property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for aJDBC metastore</description>
</property><property><name>javax.jdo.PersistenceManagerFactoryClass</name><value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value><description>class implementing the jdopersistence</description>
</property><property><name>javax.jdo.option.DetachAllOnCommit</name><value>true</value><description>detaches all objects fromsession so that they can be used after transaction iscommitted</description>
</property><property><name>javax.jdo.option.NonTransactionalRead</name><value>true</value><description>reads outside oftransactions</description>
</property><property><name>javax.jdo.option.ConnectionUserName</name><value>root</value><description>username to use againstmetastore database</description>
</property><property><name>javax.jdo.option.ConnectionPassword</name><value>root</value><description>password to use againstmetastore database</description>
</property><property><name>javax.jdo.option.Multithreaded</name><value>true</value><description>Set this to true ifmultiple threads access metastore through JDO concurrently.</description>
</property></configuration>

3.3 安装Sentry

rpm -ivh --nodeps sentry-hdfs-plugin-1.4.0+cdh5.3.3+137-1.cdh5.3.3.p0.8.el6.noarch.rpm

rpm -ivh --nodepssentry-store-1.4.0+cdh5.3.3+137-1.cdh5.3.3.p0.8.el6.noarch.rpm

2)替换Sentry内的Hadoop、Hive、Impala、Hbase、Zookeeper、Parquet、Avro等jar包

rm -rf /usr/lib/sentry/lib/hive*.jar

rm -rf /usr/lib/sentry/lib/hadoop*.jar

rm -rf /usr/lib/sentry/lib/zookeeper*.jar

rm -rf /usr/lib/sentry/lib/avro*.jar

rm -rf /usr/lib/sentry/lib/server/hive-beeline.jar

cp ~/SentryLibs/* /usr/lib/sentry/lib/

注:这里的Sentrylibs里的jar包是通过/usr/lib/sentry/lib里的文件名分别在Hadoop、Hive、Hbase等里的安装目录里一一找到的.

mv /root/SentryLibs/hive-beeline.jar/usr/lib/sentry/lib/server/

#拷贝mysql驱动到sentry的lib目录内

cp/root/hive-0.13.1-cdh5.3.3/lib/mysql-connector-java-5.1.31-bin.jar/usr/lib/sentry/lib/

#如果装了Impala,此步可忽略

rpm -ivh bigtop-utils-0.7.0+cdh5.3.3+0-1.cdh5.3.3.p0.8.el6.noarch.rpm

3) 配置Sentry,详细配置见sentry-site.xml内容如下:

<?xmlversion="1.0" encoding="UTF-8"?>
<configuration><property><name>sentry.service.security.mode</name><value>none</value></property><property><name>sentry.service.admin.group</name><value>impala,hive</value></property><property><name>sentry.service.allow.connect</name><value>impala,hive</value></property><property><name>sentry.verify.schema.version</name><value>true</value></property><property><name>sentry.service.server.rpc-address</name><value>data1</value></property><property><name>sentry.service.server.rpc-port</name><value>8038</value></property><property><name>sentry.store.jdbc.url</name><value>jdbc:mysql://localhost:3306/sentry</value></property><property><name>sentry.store.jdbc.driver</name><value>com.mysql.jdbc.Driver</value></property><property><name>sentry.store.jdbc.user</name><value>sentry</value></property><property><name>sentry.store.jdbc.password</name><value>sentry</value></property><property><name>sentry.hive.server</name><value>data1</value></property><property><name>sentry.store.group.mapping</name><value>org.apache.sentry.provider.common.HadoopGroupMappingService</value></property>
</configuration>

4)初始化Sentry元数据

sentry --command schema-tool --conffile/etc/sentry/conf/sentry-site.xml --dbType mysql --initSchema

..........................

No rows affected (0.094 seconds)

No rows affected (0.015 seconds)

No rows affected (0.075 seconds)

1 row affected (0.007 seconds)

Closing: 0: jdbc:mysql://localhost:3306/sentry

Initialization script completed

Sentry schemaTool completed

4 配置

Ranger在solr里存储日志,RangerAdmin UI依赖solr组件完成审计日志的查询,所以需要先安装和配置好Solr

注:目前(HDFS-Plugin)的测试日志审计时没选择Solr方式,但还是先配置好Standalone模式的solr.

4.2 Impala集成Sentry配置

1) Impala的conf里新增sentry配置,/etc/impala/conf/sentry-site.xml内容如下:

<?xmlversion="1.0" encoding="UTF-8"?>
<configuration>
<property><name>sentry.service.client.server.rpc-port</name><value>8038</value>
</property>
<property><name>sentry.service.client.server.rpc-address</name><value>data1</value>
</property>
<property><name>sentry.service.client.server.rpc-connection-timeout</name><value>200000</value>
</property>
<property><name>sentry.service.security.mode</name><value>none</value>
</property>
</configuration>

2) 修改impala的参数

vi/etc/default/impala

IMPALA_CATALOG_ARGS 参数里追加 -sentry_config=/etc/impala/conf/sentry-site.xml

IMPALA_SERVER_ARGS 参数里追加 -server_name=server1\

-sentry_config=/etc/impala/conf/sentry-site.xml

5 使用

5.1 Hive集成Sentry权限验证

1) 启动sentry服务,hive的metastore和hiveserver2服务

sentry --command service--conffile /etc/sentry/conf/sentry-site.xml

nohup hive --service metastore-hiveconf hive.root.logger=INFO,console > myout1.file 2>&1 &

nohup hiveserver2 -hiveconfhive.root.logger=INFO,console > myout2.file 2>&1 &

#hiveserver2里新增hive和test用户,通过beeline方式访问.

groupadd hive; useradd hive -ghive;passwd hive (type hive123)

groupadd test; useradd test -gtest;passwd test (type test123)

beeline -u "jdbc:hive2://data1:10000"-n hive -p test

2)验证整体思路及场景概述

思路:

1 HIVE用户属于管理员组,服务整个server的权限

2 Test用户创建两个数据库test_only(all权限),test_select_only(仅有select权限)

具体验证:

1 通过show databases和use database验证访问权限

2 通过test用户的use test_only能drop table,create table验证all权限

3 通过test用户的use test_select_only不能drop table验证仅有select权限

#场景一 root用户连入,尝试执行DDL操作.(无权操作)

[root@data1 conf]#beeline-u "jdbc:hive2://data1:10000" -n root

scan complete in 17ms

Connecting to jdbc:hive2://data1:10000

Connected to: Apache Hive (version 0.13.1-cdh5.3.3)

Driver: Hive JDBC (version 0.13.1-cdh5.3.3)

Transaction isolation: TRANSACTION_REPEATABLE_READ

Beeline version 0.13.1-cdh5.3.3 by Apache Hive

0:jdbc:hive2://data1:10000>create database sensitive;

Error: Error while compiling statement: FAILED:SemanticException No valid privileges

Required privileges for this query: Server=server1->action=*;(state=42000,code=40000)

0: jdbc:hive2://data1:10000> create role admin_role;

Error: Error while processing statement: FAILED: Execution Error, return code 1from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.SentryAccessDeniedException: Access denied to root (state=08S01,code=1)

0:jdbc:hive2://data1:10000> !q

Closing: 0: jdbc:hive2://data1:10000

#场景二 hive用户连入,对用户hive、test进行授权.

[root@data1 conf]#beeline-u "jdbc:hive2://data1:10000" -n hive

scan complete in 5ms

Connecting to jdbc:hive2://data1:10000

Connected to: Apache Hive (version 0.13.1-cdh5.3.3)

Driver: Hive JDBC (version 0.13.1-cdh5.3.3)

Transaction isolation: TRANSACTION_REPEATABLE_READ

Beeline version 0.13.1-cdh5.3.3 by Apache Hive

0:jdbc:hive2://data1:10000>create role admin_role;

No rows affected (1.383 seconds)

GRANT ALL ONSERVER server1 TO ROLE admin_role;

GRANT ROLEadmin_role TO GROUP hive;

create roletest_role;

GRANT ALL ONDATABASE test_only TO ROLE test_role;

GRANT ROLEtest_role TO GROUP test;

GRANT SELECT ONDATABASE test_select_only TO ROLE test_role;

#场景三 test用户连入,对数据库admin_only、test_select_only进行权限验证.

beeline -u"jdbc:hive2://data1:10000" -n test

0:jdbc:hive2://data1:10000> showdatabases;

+----------------+--+

| database_name |

+----------------+--+

| admin_only |

| default |

| test_only |

+----------------+--+

3 rows selected (0.721 seconds)

0:jdbc:hive2://data1:10000>use admin_only;

Error: Error while compiling statement: FAILED:SemanticException No valid privileges

Required privileges for this query:Server=server1->Db=admin_only->Table=*->action=insert;Server=server1->Db=admin_only->Table=*->action=select;(state=42000,code=40000)

0:jdbc:hive2://data1:10000>use test_select_only;

No rows affected (0.313 seconds)

0:jdbc:hive2://data1:10000>show tables;

+--------------+--+

| tab_name |

+--------------+--+

| select_only |

+--------------+--+

1 row selected (0.337 seconds)

0:jdbc:hive2://data1:10000>drop table select_only;

Error: Error while processing statement: FAILED: Execution Error, return code 1from org.apache.hadoop.hive.ql.exec.DDLTask.MetaException(message:hive.metastore.execute.setugi can't be false in nonsecure mode) (state=08S01,code=1)

#这里需要在HIVE服务端添加如下参数

<property>

<name>hive.metastore.execute.setugi</name>

<value>true</value>

</property>

4 rows selected (2.9 seconds)

0:jdbc:hive2://data1:10000>use test_select_only;

Norows affected (0.46 seconds)

0:jdbc:hive2://data1:10000>show tables;

+--------------+--+

| tab_name |

+--------------+--+

| select_only |

+--------------+--+

1 row selected (0.507 seconds)

0:jdbc:hive2://data1:10000>drop table select_only;

Error: Error while compilingstatement: FAILED: SemanticException No valid privileges

Required privileges for this query:Server=server1->Db=test_select_only->Table=select_only->action=*;(state=42000,code=40000)

#场景四 test用户连入,对数据库test_only进行验证,拥有所有权限:可以查看、删除表

0: jdbc:hive2://data1:10000>usetest_only;

No rows affected (0.819 seconds)

0:jdbc:hive2://data1:10000> showtables;

+--------------+--+

| tab_name |

+--------------+--+

| test_itself |

+--------------+--+

1 row selected (0.426 seconds)

0:jdbc:hive2://data1:10000>drop table test_itself;

No rows affected (6.336 seconds)

0: jdbc:hive2://data1:10000> create table test_newone ( ont string);

Norows affected (1.377 seconds)

0:jdbc:hive2://data1:10000>show tables;

+--------------+--+

| tab_name |

+--------------+--+

| test_newone |

+--------------+--+

5.2 Impala集成Sentry权限验证

1) 进入impala-shell,执行权限验证

create role supervisor;

grant all on server to rolesupervisor;

grant role supervisor to groupimpala;

GRANT ALL ON SERVER server1 TOROLE any_operation;

GRANT ROLE any_operation TOGROUP hive;

grant ALL ON database EDA TOtest_role;

grant select ON databasepriselect TO test_role;

2) 切换至test用户测试impala权限:

6 总结

仔细跟踪log,具体问题具体分析.

Apache Sentry手动安装、使用手册相关推荐

Apache 如何手动安装为服务并启动运行？
Apache 在windows下作为服务运行的命令比较简单,在进入cmd模式窗口后依次输入下面的命令即可: 1.安装作为服务:httpd -k install 2.启动服务:net start apa ...
linux apache 手动安装教程,linux下手动安装apache
下面是官方文档的要求,必须安装APR.APR-Util.PCRE,gcc-c++等包,文档URL地址http://httpd.apache.org/docs/2.4/install.html. 所以, ...
sentry mysql_安装配置apache sentry服务
环境系统环境:Centos6.7 Hadoop版本:CDH5.10 jdk版本:jdk7 注:本文并未集成kerberos组件安装Sentry Server 选择安装hive的节点进行安装测试: ...
apache证书不受信任_苹果iOS手动安装和信任企业级应用
当您首次打开手动安装的企业级应用时,会看到一个通知,提示该应用的开发者在设备上不受信任.您可以关闭此信息,但您无法打开该应用. 关闭此信息后,您可以针对此应用开发者建立信任.轻点"设置&qu ...
Ubuntu下利用安装包手动安装Apache和Php
前言首先说一下,Ubuntu中常用的安装软件的方法有两种: 1. apt-get方法使用apt-get install来安装应用程序算是最常见的一种安装方法,apt-get的基本软件安装命令是: ...
testlink mysql配置_TestLink安装配置手册
TestLink安装配置手册范围本文仅作为安装和配置TestLink的参考文档. 本文的第一部分为TestLink的安装步骤,第二部分是TestLink的配置方法. 本文和其他相关文档可在中寻求帮 ...
Apache+php+tomcat+mysql整合手册
Apache+Php+tomcat+mysql整合手册 Apache+Php+tomcat+mysql整合手册一. 需要的安装包 httpd-2.0.52.tar.gz jakarta-tomcat ...
Centos7手动安装OpenStack Mitaka版本--KeyStone安装
按照官方文档手动安装的话,基本上是复制粘贴的过程,小心点的话基本上能安装成功!如果报错我基本上干掉重来,我使用的是VM,有快照的... 创建Keystne数据库 [root@openstack-3 ~ ...
Apache+PHP+MySQL安装与配置
Windows: 为了以后重装系统方便,建议不要安装在系统盘,这里安装在D盘.可以是根目录,亦可以是其它目录,但最好目录名中不要有空格,这样可以避免某些错误的出现. 配置Apache和PHP 打开Ap ...

Apache Sentry手动安装、使用手册

1 Sentry简介

2 准备

2.1 环境说明

3 安装

3.1安装Mysql

3.2 安装Hive

3.3 安装Sentry

3.4 安装问题

4 配置

4.1 Hive集成Sentry配置

4.2 Impala集成Sentry配置

5 使用

5.1 Hive集成Sentry权限验证

5.2 Impala集成Sentry权限验证

6 总结

Apache Sentry手动安装、使用手册相关推荐

最新文章

热门文章