java多线程    Java入门    vsftp    ftp    linux配置    centos    FRP教程    HBase    Html5缓存    webp    zabbix    分布式    neo4j图数据库    

Hbase scan 查询命令大全，前缀，模糊，正则

发表于 2020-04-30 16:05:18 by 月小升

Hbase scan 查询例子数据
stu 学生
列族 base 存储学生姓名，身高基本信息
列族 score 存储成绩
c1_s1 c1 班级 s1 学生编号 

create 'stu','base','score'
put 'stu','c1_s1','base:name','jack'
put 'stu','c1_s2','base:name','jack2'
put 'stu','c1_s3','base:name','jack3'
put 'stu','c1_s4','base:name','jack4'
put 'stu','c2_s1','base:name','tom1'
put 'stu','c2_s2','base:name','tom2'
put 'stu','c2_s2','base:weight','70kg'
put 'stu','c2_s3','base:name','tom3'
put 'stu','c2_s3','base:weight','85kg'
put 'stu','c2_s3','base:height','1.70m'

小菜1：如何将查询的结果，输入文件

echo "scan 'stu',{LIMIT=>1}" | ./hbase shell > a.txt

小菜2:只返回2行

echo "scan 'stu',{LIMIT=>2}"

1. Hbase scan扫描全表，指定返回特定的列 

hbase(main):028:0> scan 'stu',{COLUMNS => ['base:weight','base:height']}
ROW                                  COLUMN+CELL
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
2 row(s)
Took 0.0113 seconds

2. Hbase TIMERANGE 扫描指定时间内数据，前闭后开
注意：包含等于前面时间的数据，不含等于后面时间的数据

hbase(main):028:0> scan 'stu',{TIMERANGE=>[1588153968060,1588153968207]}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
2 row(s)
Took 0.0108 seconds

3. Hbase 利用STARTROW STOPROW 扫描rowkey的范围
注意：包含等于前面key的数据，不含等于后面key的数据

hbase(main):028:0> scan 'stu',{STARTROW=>'c1_s1',STOPROW=>'c1_s3'}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
2 row(s)
Took 0.0092 seconds

4. HBase 翻转结果和时间组合排序 REVERSED

全表扫描翻转结果

scan 'stu', {REVERSED => TRUE}

和时间组合翻转

hbase(main):009:0> scan 'stu',{TIMERANGE=>[1588153968060,1588153968207],REVERSED => TRUE}
ROW                                  COLUMN+CELL
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack

5. Hbase 返回指标 ALL_METRICS or METRICS

hbase(main):011:0> scan 'stu',{ALL_METRICS => true}ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
7 row(s)

METRIC                               VALUE
 BYTES_IN_REMOTE_RESULTS             0
 BYTES_IN_RESULTS                    420
 MILLIS_BETWEEN_NEXTS                66
 NOT_SERVING_REGION_EXCEPTION        0
 REGIONS_SCANNED                     1
 REMOTE_RPC_CALLS                    0
 REMOTE_RPC_RETRIES                  0
 ROWS_FILTERED                       0
 ROWS_SCANNED                        7
 RPC_CALLS                           1
 RPC_RETRIES                         0
scan 'stu',{METRICS => ['ROWS_SCANNED','RPC_CALLS']}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
7 row(s)

METRIC                               VALUE
 ROWS_SCANNED                        7
 RPC_CALLS                           1

Took 0.0476 seconds

6.Hbase 查询以指定开头的rowkey数据。

hbase(main):014:0> scan 'stu',{ROWPREFIXFILTER => 'c1'}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
4 row(s)
hbase(main):016:0> scan 'stu',{FILTER => "PrefixFilter('c1')"}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
4 row(s)
Took 0.0181 seconds

7.按列查找 QualifierFilter

按列查找，可以指定某一确定的列或列的范围。binary是确定的参数，substring是参数中含有的值。

scan 'stu',{FILTER => "(QualifierFilter (<,'binary:name')) AND (QualifierFilter (=,'substring:jack'))"}

8.以指定列的前缀查找数据。ColumnPrefixFilter 

hbase(main):012:0> scan 'stu',{FILTER=>"ColumnPrefixFilter('nam') AND (ValueFilter(=,'substring:1') OR ValueFilter(=,'substring:3'))"}
ROW                                  COLUMN+CELL
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
3 row(s)
Took 0.0075 seconds

这个算法，实现了查找固定列，是否包含字符1和3，返回这些列。

## 查询一个列，并查询这个列含有1和3

hbase(main):018:0> scan 'stu',{COLUMNS => ['base:name'] , FILTER=>" (ValueFilter(=,'substring:1') OR ValueFilter(=,'substring:3'))" }
ROW                            COLUMN+CELL
 c1_s3                         column=base:name, timestamp=1588153968207, value=jack3
 c2_s1                         column=base:name, timestamp=1588153968324, value=tom1
 c2_s3                         column=base:name, timestamp=1588153968409, value=tom3
3 row(s)

9. 按值查找，可以指定确定的值或者值的范围。ValueFilter

hbase(main):018:0> scan 'stu',{FILTER=>"ValueFilter(=,'binary:jack')"}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 1 row(s)

10.按时间戳 TimestampsFilter 

hbase(main):022:0> scan 'stu',{FILTER => "TimestampsFilter(1588153968060,1588153968207)"}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
2 row(s)
Took 0.0151 seconds

时间等于1588153968060 和 1588153968207 的记录

11. RAW指导扫描器返回所有单元格(包括删除标记和未收集的已删除单元格)。此选项不能与请求特定列相结合。默认情况下禁用。

hbase(main):024:0> scan 'stu',{RAW => true,VERSIONS => 2}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
7 row(s)
Took 0.0346 seconds

我们删除一条

delete 'stu','c1_s4','base:name'

hbase(main):027:0> scan 'stu',{RAW => true,VERSIONS => 2}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c1_s4                               column=base:name, timestamp=1588153968258, type=Delete
 c1_s4                               column=base:name, timestamp=1588153968258, value=jack4
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
7 row(s)
Took 0.0189 seconds

显示type=Delete

12.FirstKeyOnlyFilter
一个rowkey可以有多个version,同一个rowkey的同一个column也会有多个的值, 只拿出key中的第一个column的第一个version
KeyOnlyFilter: 只要key,不要value

hbase(main):038:0> scan 'stu',FILTER => "FirstKeyOnlyFilter() AND ValueFilter(=,'binary:jack2') AND KeyOnlyFilter()"
ROW                                  COLUMN+CELL
 c1_s2                               column=base:name, timestamp=1588153968114, value=
1 row(s)
Took 0.0083 seconds

13. 限制返回只要两列

hbase(main):040:0> scan 'stu', {LIMIT => 2}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
2 row(s)
Took 0.0077 seconds

14.引入Java类包

列分页过滤器：基于列进行分页，需要设置偏移量与返回数量。分页ColumnPaginationFilter

语法 ColumnPaginationFilter.new(limit, offset)

hbase(main):002:0> import org.apache.hadoop.hbase.filter.ColumnPaginationFilter
=> [Java::OrgApacheHadoopHbaseFilter::ColumnPaginationFilter]

hbase(main):040:0> scan 'stu', {FILTER =>ColumnPaginationFilter.new(3, 1)}
ROW                                  COLUMN+CELL
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
2 row(s)
Took 0.0154 seconds

15. 查找rowkey里面包含s2

hbase(main):013:0> import org.apache.hadoop.hbase.filter.CompareFilter
=> [Java::OrgApacheHadoopHbaseFilter::CompareFilter]
hbase(main):014:0> import org.apache.hadoop.hbase.filter.CompareFilter
=> [Java::OrgApacheHadoopHbaseFilter::CompareFilter]
hbase(main):015:0> import org.apache.hadoop.hbase.filter.SubstringComparator
=> [Java::OrgApacheHadoopHbaseFilter::SubstringComparator]
hbase(main):016:0> import org.apache.hadoop.hbase.filter.RowFilter
=> [Java::OrgApacheHadoopHbaseFilter::RowFilter]

hbase(main):017:0> scan 'stu',{FILTER => RowFilter.new(CompareFilter::CompareOp.valueOf('EQUAL'),SubstringComparator.new('s2'))}
ROW                                  COLUMN+CELL
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
2 row(s)
Took 0.0427 seconds

16. 正则表达式查询

import org.apache.hadoop.hbase.filter.RegexStringComparator
import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SubstringComparator
import org.apache.hadoop.hbase.filter.RowFilter

直接拷贝上面的四句话

hbase(main):018:0> import org.apache.hadoop.hbase.filter.RegexStringComparator
=> [Java::OrgApacheHadoopHbaseFilter::RegexStringComparator]
hbase(main):019:0> import org.apache.hadoop.hbase.filter.CompareFilter
=> [Java::OrgApacheHadoopHbaseFilter::CompareFilter]
hbase(main):020:0> import org.apache.hadoop.hbase.filter.SubstringComparator
=> [Java::OrgApacheHadoopHbaseFilter::SubstringComparator]
hbase(main):021:0> import org.apache.hadoop.hbase.filter.RowFilter
=> [Java::OrgApacheHadoopHbaseFilter::RowFilter]
hbase(main):027:0> scan 'stu', {FILTER => RowFilter.new(CompareFilter::CompareOp.valueOf('EQUAL'),RegexStringComparator.new('^c\d+_[a-z]\d+$'))}
ROW                                  COLUMN+CELL
 c1_s1                               column=base:name, timestamp=1588153968060, value=jack
 c1_s2                               column=base:name, timestamp=1588153968114, value=jack2
 c1_s3                               column=base:name, timestamp=1588153968207, value=jack3
 c2_s1                               column=base:name, timestamp=1588153968324, value=tom1
 c2_s2                               column=base:name, timestamp=1588153968367, value=tom2
 c2_s2                               column=base:weight, timestamp=1588154167692, value=70kg
 c2_s3                               column=base:height, timestamp=1588154125060, value=1.70m
 c2_s3                               column=base:name, timestamp=1588153968409, value=tom3
 c2_s3                               column=base:weight, timestamp=1588154124202, value=85kg
6 row(s)
Took 0.0385 seconds

感觉不到变化

hbase(main):036:0> put 'stu','c3_s55','base:name','Lucy'
hbase(main):037:0> scan 'stu', {FILTER => RowFilter.new(CompareFilter::CompareOp.valueOf('EQUAL'),RegexStringComparator.new('^c\d+_s55$'))}
ROW                                  COLUMN+CELL
 c3_s55                              column=base:name, timestamp=1588162870203, value=Lucy
1 row(s)
Took 0.0082 seconds

This entry was posted in JAVA and tagged . Bookmark the permalink.
月小升QQ 2651044202, 技术交流QQ群 178491360
首发地址：月小升博客 – https://java-er.com/blog/hbase-scan-all-command/
无特殊说明，文章均为月小升原创，欢迎转载，转载请注明本文地址，谢谢
您的评论是我写作的动力.

Leave a Reply

专题-HBase

  1. Hbase scan 查询命令大全，前缀，模糊，正则
  2. Hbase 命令行 shell基础
  3. Hbase数据例子，及shell操作例子
  4. Java 实现 Hbase Scan 按行键范围扫描
  5. Java 连接Hbase代码 读取，写入，建库
  6. php 操作Hbase函数手册
  7. php连接Hbase 通过thrift
  8. 单机Hbase安装及基本命令
  9. 安装thrift
  10. 理解thrift的php和python版本教程

近期文章

  1. php8 编译zip
  2. docker 快速操作笔记
  3. 软件概要设计和详细设计的区别
  4. Spring AOP理解
  5. Spring IOC的理解
  6. Java的双亲委派模型
  7. JVM中引用的分类
  8. 柔性事务是什么，分布式环境为何要放弃传统事务
  9. Spring循环依赖如何解决的
  10. 程序员如何赚钱，财富自由

标签

    Android Apache cookie frp FTP google HBase html5 iphone java java基础教程 java教程 jQuery linux Linux命令 mac mac软件 MongoDB mysql neo4j nginx nosql php php-fpm php函数 python redis shell SVN swift Ubuntu Windows wordpress wordpress插件 xcode zabbix 值得记忆 图片 多线程 密码 插件 数据库 日志 服务器 缓存

分类

  1. JAVA
  2. Linux
  3. MAC
  4. Perl
  5. PHP
  6. Python
  7. WEB前端
  8. Windows
  9. 互联网观察
  10. 吐槽
  11. 技术管理
  12. 数学与算法
  13. 数据库
  14. 日常随笔
  15. 月小升软件
  16. 电商运营
  17. 科技精品
  18. 程序视点
  19. 网络推广
  20. 运营
  21. 阅读笔记
  22. 高并发与大数据