Due to my newest project I built a php client to access the HBase and Hive services within a hadoop cluster.
Those services are accessible via thrift a high performance protocol for back end services.
As building a client with thrift is not that easy I decided to put my HBase and Hive php thrift client packages online for others.
Links:
Hadoop
Hive
HBase
Thrift
How it works
Start the HBase and Hive Thrift server via shell:
|
1 2 |
hive --service hiveserver
/usr/lib/hbase/bin/hbase-daemon.sh start thrift |
Download HBase and Thrift php client package and write your own client:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
//thrift php
$GLOBALS['THRIFT_ROOT'] = dirname(__FILE__).'/thrift/src';
require_once( $GLOBALS['THRIFT_ROOT'].'/Thrift.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php' );
//hbase thrift
require_once dirname(__FILE__).'/thrift/Hbase.php';
//hive thrift
require_once dirname(__FILE__).'/thrift/ThriftHive.php';
/*
HBase php thrift client
*/
//open connection
$socket = new TSocket( 'localhost', 9090 );
$transport = new TBufferedTransport( $socket );
$protocol = new TBinaryProtocol( $transport );
$client = new HbaseClient( $protocol );
$transport->open();
//show all tables
$tables = $client->getTableNames();
foreach ( $tables as $name ) {
echo( " found: {$name}\n" );
}
//Create a table
try {
$columns = array(new ColumnDescriptor( array(
'name' =>; 'colFamily:',
'maxVersions' => 10) ));
$client->createTable("tableName", $columns );
} catch ( AlreadyExists $ae ) {
echo( "WARN: {$ae->;message}\n" );
}
//insert data to table
$mutations = array(
new Mutation( array(
'column' => 'colFamily:Col',
'value' => 'value123'
) ),
);
$client->mutateRow( "tableName", "ID_1237846634624", $mutations );
//get table data
$row = $client->getRow("tableName", "ID_1237846634624");
/*
Hive php thrift client
*/
//open connection
$transport = new TSocket("localhost", 10000);
$protocol = new TBinaryProtocol($transport);
$client = new ThriftHiveClient($protocol);
$transport->open();
//show tables
$client->execute('SHOW TABLES');
$tables = $client->fetchAll();
foreach ($tables as $name){
echo( " found: {$name}\n" );
}
//Create Hive table with Hbase table mapping
$mapping = 'CREATE EXTERNAL TABLE tableName(Col String, Col1 String)
STORED BY \'org.apache.hadoop.hive.hbase.HBaseStorageHandler\'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = "colFamily:Col, colFamily:Col")
TBLPROPERTIES("hbase.table.name" = "tableName")';
$client->execute($mapping);
//Query table
$client->execute('SELECT * FROM tableName Limit 10');
var_dump($client->fetchAll()); |
