HBase学习之路 (六)过滤器
过滤器(Filter)
基础API中的查询操作在面对大量数据的时候是非常苍白的,这里Hbase提供了高级的查询方法:Filter。Filter可以根据簇、列、版本等更多的条件来对数据进行过滤,基于Hbase本身提供的三维有序(主键有序、列有序、版本有序),这些Filter可以高效的完成查询过滤的任务。带有Filter条件的RPC查询请求会把Filter分发到各个RegionServer,是一个服务器端(Server-side)的过滤器,这样也可以降低网络传输的压力。
要完成一个过滤的操作,至少需要两个参数。一个是抽象的操作符,Hbase提供了枚举类型的变量来表示这些抽象的操作符:LESS/LESS_OR_EQUAL/EQUAL/NOT_EUQAL等;另外一个就是具体的比较器(Comparator),代表具体的比较逻辑,如果可以提高字节级的比较、字符串级的比较等。有了这两个参数,我们就可以清晰的定义筛选的条件,过滤数据。
抽象操作符(比较运算符)
LESS <
LESS_OR_EQUAL <=
EQUAL =
NOT_EQUAL <>
GREATER_OR_EQUAL >=
GREATER >
NO_OP 排除所有
比较器(指定比较机制)
BinaryComparator 按字节索引顺序比较指定字节数组,采用 Bytes.compareTo(byte[])
BinaryPrefixComparator 跟前面相同,只是比较左端的数据是否相同
NullComparator 判断给定的是否为空
BitComparator 按位比较
RegexStringComparator 提供一个正则的比较器,仅支持 EQUAL 和非 EQUAL
SubstringComparator 判断提供的子串是否出现在 value 中
HBase过滤器的分类
比较过滤器
1、行键过滤器 RowFilter
Filter rowFilter = new RowFilter(CompareOp.GREATER, new BinaryComparator("95007".getBytes())); scan.setFilter(rowFilter);
1 public class HbaseFilterTest { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 Filter rowFilter = new RowFilter(CompareOp.GREATER, new BinaryComparator("95007".getBytes())); 20 scan.setFilter(rowFilter); 21 ResultScanner resultScanner = table.getScanner(scan); 22 for(Result result : resultScanner) { 23 List<Cell> cells = result.listCells(); 24 for(Cell cell : cells) { 25 System.out.println(cell); 26 } 27 } 28 29 30 }
View Code
运行结果部分截图
2、列簇过滤器 FamilyFilter
Filter familyFilter = new FamilyFilter(CompareOp.EQUAL, new BinaryComparator("info".getBytes())); scan.setFilter(familyFilter);
1 public class HbaseFilterTest { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 Filter familyFilter = new FamilyFilter(CompareOp.EQUAL, new BinaryComparator("info".getBytes())); 20 scan.setFilter(familyFilter); 21 ResultScanner resultScanner = table.getScanner(scan); 22 for(Result result : resultScanner) { 23 List<Cell> cells = result.listCells(); 24 for(Cell cell : cells) { 25 System.out.println(cell); 26 } 27 } 28 29 30 } 31 32 33 }
View Code
3、列过滤器 QualifierFilter
Filter qualifierFilter = new QualifierFilter(CompareOp.EQUAL, new BinaryComparator("name".getBytes())); scan.setFilter(qualifierFilter);
1 public class HbaseFilterTest { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 Filter qualifierFilter = new QualifierFilter(CompareOp.EQUAL, new BinaryComparator("name".getBytes())); 20 scan.setFilter(qualifierFilter); 21 ResultScanner resultScanner = table.getScanner(scan); 22 for(Result result : resultScanner) { 23 List<Cell> cells = result.listCells(); 24 for(Cell cell : cells) { 25 System.out.println(cell); 26 } 27 } 28 29 30 } 31 32 33 }
View Code
4、值过滤器 ValueFilter
Filter valueFilter = new ValueFilter(CompareOp.EQUAL, new SubstringComparator("男")); scan.setFilter(valueFilter);
1 public class HbaseFilterTest { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 Filter valueFilter = new ValueFilter(CompareOp.EQUAL, new SubstringComparator("男")); 20 scan.setFilter(valueFilter); 21 ResultScanner resultScanner = table.getScanner(scan); 22 for(Result result : resultScanner) { 23 List<Cell> cells = result.listCells(); 24 for(Cell cell : cells) { 25 System.out.println(cell); 26 } 27 } 28 29 30 } 31 32 33 }
View Code
5、时间戳过滤器 TimestampsFilter
List<Long> list = new ArrayList<>(); list.add(1522469029503l); TimestampsFilter timestampsFilter = new TimestampsFilter(list); scan.setFilter(timestampsFilter);
1 public class HbaseFilterTest { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 List<Long> list = new ArrayList<>(); 20 list.add(1522469029503l); 21 TimestampsFilter timestampsFilter = new TimestampsFilter(list); 22 scan.setFilter(timestampsFilter); 23 ResultScanner resultScanner = table.getScanner(scan); 24 for(Result result : resultScanner) { 25 List<Cell> cells = result.listCells(); 26 for(Cell cell : cells) { 27 System.out.println(Bytes.toString(cell.getRow()) + "\t" + Bytes.toString(cell.getFamily()) + "\t" + Bytes.toString(cell.getQualifier()) 28 + "\t" + Bytes.toString(cell.getValue()) + "\t" + cell.getTimestamp()); 29 } 30 } 31 32 33 } 34 35 36 }
View Code
专用过滤器
1、单列值过滤器 SingleColumnValueFilter ----会返回满足条件的整行
SingleColumnValueFilter singleColumnValueFilter = new SingleColumnValueFilter("info".getBytes(), //列簇"name".getBytes(), //列CompareOp.EQUAL, new SubstringComparator("刘晨")); //如果不设置为 true,则那些不包含指定 column 的行也会返回 singleColumnValueFilter.setFilterIfMissing(true); scan.setFilter(singleColumnValueFilter);
1 public class HbaseFilterTest2 { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 SingleColumnValueFilter singleColumnValueFilter = new SingleColumnValueFilter( 20 "info".getBytes(), 21 "name".getBytes(), 22 CompareOp.EQUAL, 23 new SubstringComparator("刘晨")); 24 singleColumnValueFilter.setFilterIfMissing(true); 25 26 scan.setFilter(singleColumnValueFilter); 27 ResultScanner resultScanner = table.getScanner(scan); 28 for(Result result : resultScanner) { 29 List<Cell> cells = result.listCells(); 30 for(Cell cell : cells) { 31 System.out.println(Bytes.toString(cell.getRow()) + "\t" + Bytes.toString(cell.getFamily()) + "\t" + Bytes.toString(cell.getQualifier()) 32 + "\t" + Bytes.toString(cell.getValue()) + "\t" + cell.getTimestamp()); 33 } 34 } 35 36 37 } 38 39 40 }
View Code
2、单列值排除器 SingleColumnValueExcludeFilter
SingleColumnValueExcludeFilter singleColumnValueExcludeFilter = new SingleColumnValueExcludeFilter("info".getBytes(), "name".getBytes(), CompareOp.EQUAL, new SubstringComparator("刘晨")); singleColumnValueExcludeFilter.setFilterIfMissing(true);scan.setFilter(singleColumnValueExcludeFilter);
1 public class HbaseFilterTest2 { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 SingleColumnValueExcludeFilter singleColumnValueExcludeFilter = new SingleColumnValueExcludeFilter( 20 "info".getBytes(), 21 "name".getBytes(), 22 CompareOp.EQUAL, 23 new SubstringComparator("刘晨")); 24 singleColumnValueExcludeFilter.setFilterIfMissing(true); 25 26 scan.setFilter(singleColumnValueExcludeFilter); 27 ResultScanner resultScanner = table.getScanner(scan); 28 for(Result result : resultScanner) { 29 List<Cell> cells = result.listCells(); 30 for(Cell cell : cells) { 31 System.out.println(Bytes.toString(cell.getRow()) + "\t" + Bytes.toString(cell.getFamily()) + "\t" + Bytes.toString(cell.getQualifier()) 32 + "\t" + Bytes.toString(cell.getValue()) + "\t" + cell.getTimestamp()); 33 } 34 } 35 36 37 } 38 39 40 }
View Code
3、前缀过滤器 PrefixFilter----针对行键
PrefixFilter prefixFilter = new PrefixFilter("9501".getBytes());scan.setFilter(prefixFilter);
1 public class HbaseFilterTest2 { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 PrefixFilter prefixFilter = new PrefixFilter("9501".getBytes()); 20 21 scan.setFilter(prefixFilter); 22 ResultScanner resultScanner = table.getScanner(scan); 23 for(Result result : resultScanner) { 24 List<Cell> cells = result.listCells(); 25 for(Cell cell : cells) { 26 System.out.println(Bytes.toString(cell.getRow()) + "\t" + Bytes.toString(cell.getFamily()) + "\t" + Bytes.toString(cell.getQualifier()) 27 + "\t" + Bytes.toString(cell.getValue()) + "\t" + cell.getTimestamp()); 28 } 29 } 30 31 32 } 33 34 35 }
View Code
4、列前缀过滤器 ColumnPrefixFilter
ColumnPrefixFilter columnPrefixFilter = new ColumnPrefixFilter("name".getBytes());scan.setFilter(columnPrefixFilter);
1 public class HbaseFilterTest2 { 2 3 private static final String ZK_CONNECT_KEY = "hbase.zookeeper.quorum"; 4 private static final String ZK_CONNECT_VALUE = "hadoop1:2181,hadoop2:2181,hadoop3:2181"; 5 6 private static Connection conn = null; 7 private static Admin admin = null; 8 9 public static void main(String[] args) throws Exception { 10 11 Configuration conf = HBaseConfiguration.create(); 12 conf.set(ZK_CONNECT_KEY, ZK_CONNECT_VALUE); 13 conn = ConnectionFactory.createConnection(conf); 14 admin = conn.getAdmin(); 15 Table table = conn.getTable(TableName.valueOf("student")); 16 17 Scan scan = new Scan(); 18 19 ColumnPrefixFilter columnPrefixFilter = new ColumnPrefixFilter("name".getBytes()); 20 21 scan.setFilter(columnPrefixFilter); 22 ResultScanner resultScanner = table.getScanner(scan); 23 for(Result result : resultScanner) { 24 List<Cell> cells = result.listCells(); 25 for(Cell cell : cells) { 26 System.out.println(Bytes.toString(cell.getRow()) + "\t" + Bytes.toString(cell.getFamily()) + "\t" + Bytes.toString(cell.getQualifier()) 27 + "\t" + Bytes.toString(cell.getValue()) + "\t" + cell.getTimestamp()); 28 } 29 } 30 31 32 } 33 34 35 }
View Code
5、分页过滤器 PageFilter
HBase学习之路 (六)过滤器相关推荐
- HBase学习之路(七):理解计数器的基本概念及计数器的使用
内容简介 一.计数器的基本概念 二.在Shell中创建并操作计数器 三.单计数器 四.多计数器 五.总结 一.计数器的基本概念 我在前两篇文章中详细介绍了HBase的过滤器,这是HBase的高级特性之 ...
- HBase学习之路(二):Java客户端的CRUD操作详讲
内容简介 一.概述 二.操作前的准备 三.put操作 1.单行put方法 2.使用Put列表 四.get操作 1.单行get方法 2.使用Get列表 五.delete操作 1.单行delete方法 2 ...
- HBase学习之路 (九)HBase phoenix的使用
HBase phoenix的下载 下载地址http://mirror.bit.edu.cn/apache/phoenix/ 选择对应的hbase版本进行下载,测试使用的是hbase-1.2.6版本
- android学习之路(六)---- 图片加载库的优化、封装
封装Image-Loader 一.背景 universal-image-loader是一项伟大的开源项目,作者在其中运用到的软件工程解决办法让人印象深刻,在本篇文章的开篇,首先向uni ...
- ios7学习之路六(隐藏状态栏 )
方法一(代码设置): 现在ios7已经更改为透明,并且不占用屏幕高度.其中隐藏及显示的方法如下: 在uiviewcontroller的子类下,调用: if ([self respondsToSelec ...
- hbase scan超时设置_HBase学习之路 (六)过滤器
点击上方蓝字 关注我们 HBase学习之路 (六)过滤器 过滤器(Filter) 基础API中的查询操作在面对大量数据的时候是非常苍白的,这里Hbase提供了高级的查询方法:Filter.Filte ...
- HBase学习(7)-HBase原理
原文来自:扎心了,老铁的<HBase学习之路 (七)HBase 原理>
- HBase学习(5)-MapReduce操作HBase
原文来自:扎心了,老铁的<HBase学习之路 (五)MapReduce操作Hbase>
- C/C++学习之路_六: 指针
C/C++学习之路_第六章: 指针 目录 概述 指针基础知识 指针和数组 多级指针 指针和函数 指针和字符串 指针小结 1. 概述 1. 内存 内存含义: 存储器:计算机的组成中,用来存储程序和数据, ...
最新文章
- 《数据库原理与应用(第3版)》——习题
- 来自nativeGetEnabledTags的意外值:0
- 大厂笔试 java基础
- 此版本的应用程序不支持其项目类型 (.etp)_适用于Microsoft Dynamics 365商业中心的VPS和VJS版本1.8...
- python 模块与包_Python 模块与包
- LibreOJ #103. 子串查找
- 看设备指示灯识别网络故障
- Python爬虫自学之第(零)篇——爬虫思路和request模块使用
- 用友云开发者中心助你上云系列之在线调试
- docker配置 nacos_Nacos - 阿里开源配置中心
- 【GPU编程】The Cg Runtime:OpenGL中调用Cg程序
- 前端基础之CSS属性
- matlab层次分析法代码_基于主成分分析法和层次分析法的工程项目经理胜任力评价研究...
- Unity 发射子弹的两种方式
- 【昇润蓝牙】蓝牙4.0BLE模组(型号CC2541) 两个蓝牙之间互相通信的使用日志
- 2.Windows 界面技术发展现状
- linux 卸载 flash,Linux系统下Flash Player崩溃的一个解决办法分享
- 今日科技联播:全球区块链专利申请数超5000件,Coinplug第一阿里紧随其后
- u盘第一扇区 分区表_硬盘主引导扇区、分区表和分区引导扇区(MBR、DPT、DBR、BPB)详解...
- python使用openCV图像加载(转化为灰度图像)、平滑图像处理就是将每个像素的值变换为其相邻元素的平均值、可视化平滑处理之后的图像(Blurring Images)
热门文章
- RBee ZigBee无线透明传输通信模块 兼容arduino平台 类似XBee
- Kmeans聚类算法分析(转帖)
- 分享Silverlight/WPF/Windows Phone一周学习导读(3月14日-3月19日)
- HTML5之Canvas标签简要学习
- 几种filter的比较
- 迈出物联网的第一步,玩儿一下Arduino
- 系统linux/redhat6.5 zabbix 2.47监控nginx1.8.0 (下)
- Autoit 在word中绘图
- RotateAnimation详解
- Android应用程序窗口(Activity)的测量(Measure)、布局(Layout)和绘制(Draw)过程分析(上)...