« Posts tagged client

HBase and Hive Thrift PHP Client

December 10, 2011 / posted in php / No Comments

Due to my newest project I built a php client to access the HBase and Hive services within a hadoop cluster.

Those services are accessible via thrift a high performance protocol for back end services.

As building a client with thrift is not that easy I decided to put my HBase and Hive php thrift client packages online for others.

Links:
Hadoop
Hive
HBase
Thrift

How it works

Start the HBase and Hive Thrift server via shell:

Shell
1
2
hive --service hiveserver
/usr/lib/hbase/bin/hbase-daemon.sh start thrift

 

Download HBase and Thrift php client package and write your own client:

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
//thrift php
$GLOBALS['THRIFT_ROOT'] = dirname(__FILE__).'/thrift/src';
require_once( $GLOBALS['THRIFT_ROOT'].'/Thrift.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php' );
require_once( $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php' );
//hbase thrift
require_once dirname(__FILE__).'/thrift/Hbase.php';
//hive thrift
require_once dirname(__FILE__).'/thrift/ThriftHive.php';
/*
HBase php thrift client
*/
//open connection
$socket = new TSocket( 'localhost', 9090 );
$transport = new TBufferedTransport( $socket );
$protocol = new TBinaryProtocol( $transport );
$client = new HbaseClient( $protocol );
$transport->open();
//show all tables
$tables = $client->getTableNames();
foreach ( $tables as $name ) {
echo( " found: {$name}\n" );
}
//Create a table
try {
$columns = array(new ColumnDescriptor( array(
'name' =>; 'colFamily:',
'maxVersions' => 10) ));
$client->createTable("tableName", $columns );
} catch ( AlreadyExists $ae ) {
echo( "WARN: {$ae->;message}\n" );
}
//insert data to table
$mutations = array(
new Mutation( array(
'column' => 'colFamily:Col',
'value' => 'value123'
) ),
);
$client->mutateRow( "tableName", "ID_1237846634624", $mutations );
//get table data
$row = $client->getRow("tableName", "ID_1237846634624");
/*
Hive php thrift client
*/
//open connection
$transport = new TSocket("localhost", 10000);
$protocol = new TBinaryProtocol($transport);
$client = new ThriftHiveClient($protocol);
$transport->open();
//show tables
$client->execute('SHOW TABLES');
$tables = $client->fetchAll();
foreach ($tables as $name){
echo( " found: {$name}\n" );
}
//Create Hive table with Hbase table mapping
$mapping = 'CREATE EXTERNAL TABLE tableName(Col String, Col1 String)
STORED BY \'org.apache.hadoop.hive.hbase.HBaseStorageHandler\'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = "colFamily:Col, colFamily:Col")
TBLPROPERTIES("hbase.table.name" = "tableName")';
$client->execute($mapping);
//Query table
$client->execute('SELECT * FROM tableName Limit 10');
var_dump($client->fetchAll());