Due to my newest project I built a php client to access the HBase and Hive services within a hadoop cluster.
Those services are accessible via thrift a high performance protocol for back end services.
As building a client with thrift is not that easy I decided to put my HBase and Hive php thrift client packages online for others.
Links:
Hadoop
Hive
HBase
Thrift
How it works
Start the HBase and Hive Thrift server via shell:
|
1 2 |
hive --service hiveserver /usr/lib/hbase/bin/hbase-daemon.sh start thrift |
Download HBase and Thrift php client package and write your own client:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
//thrift php $GLOBALS['THRIFT_ROOT'] = dirname(__FILE__).'/thrift/src'; require_once( $GLOBALS['THRIFT_ROOT'].'/Thrift.php' ); require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php' ); require_once( $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php' ); require_once( $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php' ); //hbase thrift require_once dirname(__FILE__).'/thrift/Hbase.php'; //hive thrift require_once dirname(__FILE__).'/thrift/ThriftHive.php'; /* HBase php thrift client */ //open connection $socket = new TSocket( 'localhost', 9090 ); $transport = new TBufferedTransport( $socket ); $protocol = new TBinaryProtocol( $transport ); $client = new HbaseClient( $protocol ); $transport->open(); //show all tables $tables = $client->getTableNames(); foreach ( $tables as $name ) { echo( " found: {$name}\n" ); } //Create a table try { $columns = array(new ColumnDescriptor( array( 'name' =>; 'colFamily:', 'maxVersions' => 10) )); $client->createTable("tableName", $columns ); } catch ( AlreadyExists $ae ) { echo( "WARN: {$ae->;message}\n" ); } //insert data to table $mutations = array( new Mutation( array( 'column' => 'colFamily:Col', 'value' => 'value123' ) ), ); $client->mutateRow( "tableName", "ID_1237846634624", $mutations ); //get table data $row = $client->getRow("tableName", "ID_1237846634624"); /* Hive php thrift client */ //open connection $transport = new TSocket("localhost", 10000); $protocol = new TBinaryProtocol($transport); $client = new ThriftHiveClient($protocol); $transport->open(); //show tables $client->execute('SHOW TABLES'); $tables = $client->fetchAll(); foreach ($tables as $name){ echo( " found: {$name}\n" ); } //Create Hive table with Hbase table mapping $mapping = 'CREATE EXTERNAL TABLE tableName(Col String, Col1 String) STORED BY \'org.apache.hadoop.hive.hbase.HBaseStorageHandler\' WITH SERDEPROPERTIES ("hbase.columns.mapping" = "colFamily:Col, colFamily:Col") TBLPROPERTIES("hbase.table.name" = "tableName")'; $client->execute($mapping); //Query table $client->execute('SELECT * FROM tableName Limit 10'); var_dump($client->fetchAll()); |
Comments (0)