php调用Hbase 通过thrift

系统版本: CentOS 7.4
JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64
Habse路径: /backup/soft/hbase-2.2.3
Thrift路径: /backup/soft/thrift-0.13.0
PHP版本:php 5.6


第一步:配置Hbase

参考https://java-er.com/blog/danji-hbase-install-command/

mkdir -p /backup/soft/hbase-2.2.3/data
chmod -R 777 /backup/soft/hbase-2.2.3/data

配置文件

./conf/hbase-site.xml
<configuration>
 <property>
    <name>hbase.rootdir</name>
    <value>/backup/soft/hbase-2.2.3/data</value>
  </property>
</configuration>

启动

/bin/start-hbase.sh

第二步:要hbase的源码

要下载hbase-2.2.3-src.tar.gz

存放路径/backup/soft/src/hbase-2.2.3-src.tar.gz

解压缩

tar zxvf hbase-2.2.3-src.tar.gz

源码路径

/backup/soft/src/hbase-2.2.3

第三步:安装thrift

参考 https://java-er.com/blog/install-thrift/

最终路径 /backup/soft/thrift-0.13.0

第四步:用thrift创建php文件

cd /backup/soft/src/hbase-2.2.3
 find ./ -name Hbase.thrift
./hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
cd ./hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/
thrift -gen php ./Hbase.thrift
[root@iz2ze9n3z2fl9zj9hrv58vz thrift]# ls
Hbase.thrift  gen-php

多个gen-php

[root@iz2ze9n3z2fl9zj9hrv58vz gen-php]# ls
Hbase
[root@iz2ze9n3z2fl9zj9hrv58vz gen-php]# cd Hbase/
[root@iz2ze9n3z2fl9zj9hrv58vz Hbase]# ls
AlreadyExists.php                      Hbase_getTableRegions_result.php
BatchMutation.php                      Hbase_getVerTs_args.php
ColumnDescriptor.php                   Hbase_getVerTs_result.php
HbaseClient.php                        Hbase_getVer_args.php
HbaseIf.php                            Hbase_getVer_result.php
Hbase_append_args.php                  Hbase_get_args.php
Hbase_append_result.php                Hbase_get_result.php
Hbase_atomicIncrement_args.php         Hbase_incrementRows_args.php
Hbase_atomicIncrement_result.php       Hbase_incrementRows_result.php
Hbase_checkAndPut_args.php             Hbase_increment_args.php
Hbase_checkAndPut_result.php           Hbase_increment_result.php
Hbase_compact_args.php                 Hbase_isTableEnabled_args.php
Hbase_compact_result.php               Hbase_isTableEnabled_result.php
Hbase_createTable_args.php             Hbase_majorCompact_args.php
Hbase_createTable_result.php           Hbase_majorCompact_result.php
Hbase_deleteAllRowTs_args.php          Hbase_mutateRowTs_args.php
Hbase_deleteAllRowTs_result.php        Hbase_mutateRowTs_result.php
Hbase_deleteAllRow_args.php            Hbase_mutateRow_args.php
Hbase_deleteAllRow_result.php          Hbase_mutateRow_result.php
Hbase_deleteAllTs_args.php             Hbase_mutateRowsTs_args.php
Hbase_deleteAllTs_result.php           Hbase_mutateRowsTs_result.php
Hbase_deleteAll_args.php               Hbase_mutateRows_args.php
Hbase_deleteAll_result.php             Hbase_mutateRows_result.php
Hbase_deleteTable_args.php             Hbase_scannerClose_args.php
Hbase_deleteTable_result.php           Hbase_scannerClose_result.php
Hbase_disableTable_args.php            Hbase_scannerGetList_args.php
Hbase_disableTable_result.php          Hbase_scannerGetList_result.php
Hbase_enableTable_args.php             Hbase_scannerGet_args.php
Hbase_enableTable_result.php           Hbase_scannerGet_result.php
Hbase_getColumnDescriptors_args.php    Hbase_scannerOpenTs_args.php
Hbase_getColumnDescriptors_result.php  Hbase_scannerOpenTs_result.php
Hbase_getRegionInfo_args.php           Hbase_scannerOpenWithPrefix_args.php
Hbase_getRegionInfo_result.php         Hbase_scannerOpenWithPrefix_result.php
Hbase_getRowTs_args.php                Hbase_scannerOpenWithScan_args.php
Hbase_getRowTs_result.php              Hbase_scannerOpenWithScan_result.php
Hbase_getRowWithColumnsTs_args.php     Hbase_scannerOpenWithStopTs_args.php
Hbase_getRowWithColumnsTs_result.php   Hbase_scannerOpenWithStopTs_result.php
Hbase_getRowWithColumns_args.php       Hbase_scannerOpenWithStop_args.php
Hbase_getRowWithColumns_result.php     Hbase_scannerOpenWithStop_result.php
Hbase_getRow_args.php                  Hbase_scannerOpen_args.php
Hbase_getRow_result.php                Hbase_scannerOpen_result.php
Hbase_getRowsTs_args.php               IOError.php
Hbase_getRowsTs_result.php             IllegalArgument.php
Hbase_getRowsWithColumnsTs_args.php    Mutation.php
Hbase_getRowsWithColumnsTs_result.php  TAppend.php
Hbase_getRowsWithColumns_args.php      TCell.php
Hbase_getRowsWithColumns_result.php    TColumn.php
Hbase_getRows_args.php                 TIncrement.php
Hbase_getRows_result.php               TRegionInfo.php
Hbase_getTableNames_args.php           TRowResult.php
Hbase_getTableNames_result.php         TScan.php
Hbase_getTableRegions_args.php

不少网上的例子含有有一个Hbase-Type.php 估计是之前版本的文件

第五步:建立一个php测试文件夹

要点:最早看到各种引入文件都报错,所以这一步相当重要,拷贝正确的文件,总结的说,两个文件包是必须的。

mkdir /backup/soft/phptest -p
cd /backup/soft/phptest

拷贝那个gen-php进来

cp /backup/soft/src/hbase-2.2.3/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/gen-php ./ -R

拷贝thrift下php源代码进来

cp /backup/soft/thrift-0.13.0/lib/php/lib ./ -R
 
[root@iz2ze9n3z2fl9zj9hrv58vz phptest]# ls
gen-php  lib

第六步:建立一个test.php

#!/usr/bin/env php
<?php
error_reporting(E_ALL);
 
$GLOBALS['THRIFT_ROOT'] = "/backup/soft/phptest/lib";
$GLOBALS['HBase_PHP'] = "/backup/soft/phptest/gen-php";
 
require_once $GLOBALS['THRIFT_ROOT'] . '/Transport/TTransport.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Protocol/TProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Protocol/TBinaryProtocolAccelerated.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Transport/TBufferedTransport.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Type/TMessageType.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Factory/TStringFuncFactory.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/StringFunc/TStringFunc.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/StringFunc/Core.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Type/TType.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Exception/TException.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Exception/TTransportException.php';
require_once $GLOBALS['THRIFT_ROOT'] . '/Exception/TProtocolException.php';
 
require_once $GLOBALS['HBase_PHP'] . '/Hbase/HbaseIf.php';
require_once $GLOBALS['HBase_PHP'] . '/Hbase/HbaseClient.php';
require_once $GLOBALS['HBase_PHP'] . '/Hbase/Hbase_getTableNames_args.php';
require_once $GLOBALS['HBase_PHP'] . '/Hbase/Hbase_getTableNames_result.php';
 
 
 
use Thrift\Protocol\TBinaryProtocol;
use Thrift\Transport\TSocket;
use Thrift\Transport\THttpClient;
use Thrift\Transport\TBufferedTransport;
use Thrift\Exception\TException;
 
use Hbase\HbaseClient;
use Hbase\ColumnDescriptor;
use Hbase\Mutation;
 
 
 
$socket = new TSocket('localhost', 9090);
 
 
$transport = new TBufferedTransport($socket);
$protocol = new TBinaryProtocol($transport);
 $client = new HbaseClient($protocol);
 $transport->open();
 
echo "----list tables----\n";
$tables = $client->getTableNames();
foreach ($tables as $name) {
    var_dump($tables);
}
$transport->close();
?>

第七步:确认Hbase, Hbase thrift启动

cd /backup/soft/hbase-2.2.3
[root@iz2ze9n3z2fl9zj9hrv58vz hbase-2.2.3]# bin/start-hbase.sh

Hbase 启动thrift

 ./hbase-daemon.sh start thrift
[root@iz2ze9n3z2fl9zj9hrv58vz hbase-2.2.3]# jps
31960 ThriftServer
32202 HMaster
32475 Jps

看到这三个说明真的启动了

简单建立一个表

[root@iz2ze9n3z2fl9zj9hrv58vz hbase-2.2.3]# bin/hbase shell
 
 
hbase(main):001:0> create 'test', 'cf'

第八步:回到php目录

cd /backup/soft/phptest
 
 
[root@iz2ze9n3z2fl9zj9hrv58vz phptest]# php test.php 
----list tables----
array(1) {
  [0]=>
  string(4) "test"
}

完成

#遇到错误1.

hbase(main):001:0> create 'test', 'cf'
 
ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
	at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2806)
	at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2014)
	at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:659)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)

报错
先启动regionserver,然后在启动Hmaster

./hbase-daemon.sh start regionserver

然后执行:

./hbase-daemon.sh start master
 
./hbase zkcli
 
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, hbase]
[zk: localhost:2181(CONNECTED) 1] rmr /hbase
[zk: localhost:2181(CONNECTED) 2] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 3]

重启动HBase搞定。
—后记:如果电脑可以重启,直接重启电脑就省掉了上面的步骤。也起效

#遇到错误2. 文件引入错误缺少php类包

2020.01.04 下午14:22 另外一个新错误
php Test.php
—-list tables—-

当我执行$tables = $client->getTableNames();报错,我去翻看HBase 下的php文件发现了Hbase_getTableNames_args 文件(我已经感觉到php应该有一次性加载一个文件夹的工作模式,太久不写代码,人都老了。代码越发更新)

require_once $GLOBALS['HBase_PHP'] . '/Hbase/Hbase_getTableNames_args.php';

Fatal error: Class ‘Hbase\Hbase_getTableNames_result’ not found in /root/HBaseCode/thrift/gen-php/Hbase/HbaseClient.php on line 379
说明需要引入一个文件

require_once $GLOBALS['HBase_PHP'] . '/Hbase/Hbase_getTableNames_result.php';

#端口查看

netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      4397/nginx: worker  
tcp        0      0 0.0.0.0:81              0.0.0.0:*               LISTEN      4397/nginx: worker  
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1208/sshd           
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      4397/nginx: worker  
tcp        0      0 0.0.0.0:40033           0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 172.17.121.16:43458     0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 0.0.0.0:45698           0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 0.0.0.0:2181            0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 172.17.121.16:45703     0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      1976/php-fpm: maste 
tcp        0      0 0.0.0.0:32969           0.0.0.0:*               LISTEN      7538/java           
tcp        0      0 0.0.0.0:3690            0.0.0.0:*               LISTEN      3839/svnserve       
tcp6       0      0 :::3306                 :::*                    LISTEN      2171/mysqld         
udp        0      0 0.0.0.0:62873           0.0.0.0:*                           728/dhclient        
udp        0      0 0.0.0.0:68              0.0.0.0:*                           728/dhclient        
udp        0      0 172.17.121.16:123       0.0.0.0:*                           817/ntpd            
udp        0      0 127.0.0.1:123           0.0.0.0:*                           817/ntpd            
udp        0      0 0.0.0.0:123             0.0.0.0:*                           817/ntpd            
udp6       0      0 :::44296                :::*                                728/dhclient        
udp6       0      0 :::123                  :::*                                817/ntpd

Hbase 启动thrift

./hbase-daemon.sh start thrift

多了个进程

tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      8006/java
telnet localhost 9090
 
telnet localhost 9090
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

通了

后记,在没有按照thrift服务的mac电脑上,也可以启动

 ./hbase-daemon.sh start thrift

进程

jps
23107 ThriftServer
23146 Jps
22511 HMaster

但是php就是不生效,报错为

----list tables----
 
Fatal error: Uncaught Thrift\Exception\TTransportException: TSocket: timed out reading 4 bytes from localhost:9090 in /Users/ge/Desktop/HBaseCode/lib/php/Thrift/Transport/TSocket.php:300
Stack trace:

所以月小升推测,必须在电脑上安装了Thrift服务,这个php代码才会起效。


This entry was posted in JAVA, Linux, PHP and tagged , . Bookmark the permalink.
月小升QQ 2651044202, 技术交流QQ群 178491360
首发地址:月小升博客https://java-er.com/blog/php-hbase-thrift/
无特殊说明,文章均为月小升原创,欢迎转载,转载请注明本文地址,谢谢
您的评论是我写作的动力

Leave a Reply

Your email address will not be published. Required fields are marked *

*

*