线程锁实现多线程读取mongo 数据库库
2021/5/24 19:24:53
本文主要是介绍线程锁实现多线程读取mongo 数据库库,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
# encoding:utf-8 import threading import pymongo import redis from bson import ObjectId limit_page = 1000 lock = threading.Lock() class Up: def __init__(self): self.LastId = '' self.startClient = pymongo.MongoClient('192.168.1.50').TianYan.ShangBiao_old self.redisclient = redis.StrictRedis(host='192.168.1.50', db=10) def get_mongo_info(self): lock.acquire() if self.LastId: result = self.startClient.find({"_id": {"$gt": ObjectId(self.LastId)}}).limit(limit_page).sort('_id', 1) else: result = self.startClient.find().limit(limit_page).sort('_id', 1) message = list(result) try: self.LastId = message[-1]['_id'] except: print("数据导出完毕") return message else: return message finally: lock.release() def find_info(self): while True: message = self.get_mongo_info() if message ==[]: break for info in message: company_id = info['company_id'] print(company_id) self.redisclient.sadd('trademark', company_id) def run(self): Threads = [] for _ in range(50): t2 = threading.Thread(target=self.find_info) t2.start() Threads.append(t2) for t in Threads: t.join() if __name__ == '__main__': up = Up() up.run()
这篇关于线程锁实现多线程读取mongo 数据库库的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-06-26解决google chrome helper 内存占用较高!
- 2024-04-01got an unexpected keyword argument
- 2024-03-30维多利亚的秘密 golang入坑系统
- 2024-03-29mongodb sort by date
- 2024-03-29go swagger
- 2024-03-25mongodb cdc
- 2024-03-25how to use go in vscode
- 2024-03-22mongooseserverselectionerror: connect econnrefused ::1:27017
- 2024-03-21pymongo insert_many
- 2024-03-18projection mongodb