python 调用HBase的简单实例
2019/7/13 23:18:54
本文主要是介绍python 调用HBase的简单实例,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
新来的一个工程师不懂HBase,java不熟,python还行,我建议他那可以考虑用HBase的thrift调用,完成目前的工作。
首先,安装thrift
下载thrift,这里,我用的是thrift-0.7.0-dev.tar.gz 这个版本
tar xzf thrift-0.7.0-dev.tar.gz cd thrift-0.7.0-dev sudo ./configure --with-cpp=no --with-ruby=no sudo make sudo make install
然后,到HBase的源码包里,找到
src/main/resources/org/apache/hadoop/hbase/thrift/
执行
thrift --gen py Hbase.thrift
mv gen-py/hbase/ /usr/lib/python2.4/site-packages/ (根据python版本可能有不同)
我这里写了些调用的脚本,供大家参考
from unittest import TestCase, main from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import ColumnDescriptor, Mutation, BatchMutation class HBaseTester: def __init__(self, netloc, port, table="staftesttable"): self.tableName = table self.transport = TTransport.TBufferedTransport( TSocket.TSocket(netloc, port)) self.protocol = TBinaryProtocol.TBinaryProtocol(self.transport) self.client = Hbase.Client(self.protocol) self.transport.open() tables = self.client.getTableNames() if self.tableName not in tables: self.__createTable() def __del__(self): self.transport.close() def __createTable(self): name = ColumnDescriptor(name='name') foo = ColumnDescriptor(name='foo') self.client.createTable(self.tableName, [name,foo]) def put(self,key,name,foo): name = Mutation(column="name:v", value=name) foo = Mutation(column="foo:v",value=foo) self.client.mutateRow(self.tablename,key,[name,foo]) def scanner(self,column): scanner = client.scannerOpen(self.tablename,"",[column]) r = client.scannerGet(scanner) result= [] while r: print r[0] result.append(r[0]) r = client.scannerGet(scanner) print "Scanner finished" return result class TestHBaseTester(TestCase): def setUp(self): self.writer = HBaseTester("localhost", 9090) def tearDown(self): name = self.writer.tableName client = self.writer.client client.disableTable(name) client.deleteTable(name) def testCreate(self): tableName = self.writer.tableName client = self.writer.client self.assertTrue(self.writer.tableName in client.getTableNames()) columns =['name:','foo:'] for i in client.getColumnDescriptors(tableName): self.assertTrue(i in columns) def testPut(self): self.writer.put("r1","n1","f1") self.writer.put("r2","n2","f2") self.writer.put("r3","n3","") self.writer.scanner("name:") if __name__ == "__main__": main()
以上这篇python 调用HBase的简单实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持找一找教程网。
这篇关于python 调用HBase的简单实例的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-09-27使用python 将ETH账户的资产打散
- 2024-09-26Python编程基础
- 2024-09-2610 种方法写出更好的 Python 代码
- 2024-09-25Python编程基础详解
- 2024-09-25Python编程入门教程
- 2024-09-25从零开始使用Python构建LLaMA 3
- 2024-09-23Python中理解和使用树形结构的简单教程
- 2024-09-23Python 编程基础入门
- 2024-09-18初探Python股票自动化交易:入门指南
- 2024-09-18Python量化入门:轻松掌握量化分析基础与实战