python 调用HBase的简单实例

2019/7/13 23:18:54

本文主要是介绍python 调用HBase的简单实例,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!

新来的一个工程师不懂HBase,java不熟,python还行,我建议他那可以考虑用HBase的thrift调用,完成目前的工作。

首先,安装thrift

下载thrift,这里,我用的是thrift-0.7.0-dev.tar.gz 这个版本

tar xzf thrift-0.7.0-dev.tar.gz
cd thrift-0.7.0-dev
sudo ./configure --with-cpp=no --with-ruby=no
sudo make
sudo make install

然后,到HBase的源码包里,找到

src/main/resources/org/apache/hadoop/hbase/thrift/

执行

thrift --gen py Hbase.thrift
mv gen-py/hbase/ /usr/lib/python2.4/site-packages/ (根据python版本可能有不同)

我这里写了些调用的脚本,供大家参考

from unittest import TestCase, main
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import ColumnDescriptor, Mutation, BatchMutation
class HBaseTester:

  def __init__(self, netloc, port, table="staftesttable"):
    self.tableName = table

    self.transport = TTransport.TBufferedTransport(
      TSocket.TSocket(netloc, port))
    self.protocol = TBinaryProtocol.TBinaryProtocol(self.transport)
    self.client = Hbase.Client(self.protocol)
    self.transport.open()

    tables = self.client.getTableNames()
    if self.tableName not in tables:
      self.__createTable()

  def __del__(self):
    self.transport.close()

  def __createTable(self):
    name = ColumnDescriptor(name='name')
    foo = ColumnDescriptor(name='foo')

    self.client.createTable(self.tableName,
                [name,foo])
  
  def put(self,key,name,foo):
    name = Mutation(column="name:v", value=name)
    foo = Mutation(column="foo:v",value=foo)
    
    self.client.mutateRow(self.tablename,key,[name,foo])
  
  def scanner(self,column):
    scanner = client.scannerOpen(self.tablename,"",[column])
				r = client.scannerGet(scanner)
				result= []
    while r:
     print r[0]
     result.append(r[0])
     r = client.scannerGet(scanner)
    print "Scanner finished"
    return result
   
  
    
    
    
     
class TestHBaseTester(TestCase):
  
  def setUp(self):
    self.writer = HBaseTester("localhost", 9090)

  def tearDown(self):
    name = self.writer.tableName
    client = self.writer.client
    client.disableTable(name)
    client.deleteTable(name)


  def testCreate(self):
    tableName = self.writer.tableName
    client = self.writer.client
    self.assertTrue(self.writer.tableName in client.getTableNames())
    columns =['name:','foo:']
    for i in client.getColumnDescriptors(tableName):
      self.assertTrue(i in columns)
      
  def testPut(self):
    self.writer.put("r1","n1","f1")
    self.writer.put("r2","n2","f2")
    self.writer.put("r3","n3","")
    self.writer.scanner("name:") 
    
if __name__ == "__main__":
  main()

以上这篇python 调用HBase的简单实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持找一找教程网。



这篇关于python 调用HBase的简单实例的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!


扫一扫关注最新编程教程