python队列是线程安全的吗_python – 为什么我的多进程队列看起来不是线程安全的?...
我正在構建一個運行另一個
Python程序的監視程序計時器,如果它無法從任何線程中找到簽入,則關閉整個程序.這樣,它最終將能夠控制所需的通信端口.計時器的代碼如下:
from multiprocessing import Process, Queue
from time import sleep
from copy import deepcopy
PATH_TO_FILE = r'.\test_program.py'
WATCHDOG_TIMEOUT = 2
class Watchdog:
def __init__(self, filepath, timeout):
self.filepath = filepath
self.timeout = timeout
self.threadIdQ = Queue()
self.knownThreads = {}
def start(self):
threadIdQ = self.threadIdQ
process = Process(target = self._executeFile)
process.start()
try:
while True:
unaccountedThreads = deepcopy(self.knownThreads)
# Empty queue since last wake. Add new thread IDs to knownThreads, and account for all known thread IDs
# in queue
while not threadIdQ.empty():
threadId = threadIdQ.get()
if threadId in self.knownThreads:
unaccountedThreads.pop(threadId, None)
else:
print('New threadId < {} > discovered'.format(threadId))
self.knownThreads[threadId] = False
# If there is a known thread that is unaccounted for, then it has either hung or crashed.
# Shut everything down.
if len(unaccountedThreads) > 0:
print('The following threads are unaccounted for:\n')
for threadId in unaccountedThreads:
print(threadId)
print('\nShutting down!!!')
break
else:
print('No unaccounted threads...')
sleep(self.timeout)
# Account for any exceptions thrown in the watchdog timer itself
except:
process.terminate()
raise
process.terminate()
def _executeFile(self):
with open(self.filepath, 'r') as f:
exec(f.read(), {'wdQueue' : self.threadIdQ})
if __name__ == '__main__':
wd = Watchdog(PATH_TO_FILE, WATCHDOG_TIMEOUT)
wd.start()
我還有一個小程序來測試看門狗功能
from time import sleep
from threading import Thread
from queue import SimpleQueue
Q_TO_Q_DELAY = 0.013
class QToQ:
def __init__(self, processQueue, threadQueue):
self.processQueue = processQueue
self.threadQueue = threadQueue
Thread(name='queueToQueue', target=self._run).start()
def _run(self):
pQ = self.processQueue
tQ = self.threadQueue
while True:
while not tQ.empty():
sleep(Q_TO_Q_DELAY)
pQ.put(tQ.get())
def fastThread(q):
while True:
print('Fast thread, checking in!')
q.put('fastID')
sleep(0.5)
def slowThread(q):
while True:
print('Slow thread, checking in...')
q.put('slowID')
sleep(1.5)
def hangThread(q):
print('Hanging thread, checked in')
q.put('hangID')
while True:
pass
print('Hello! I am a program that spawns threads!\n\n')
threadQ = SimpleQueue()
Thread(name='fastThread', target=fastThread, args=(threadQ,)).start()
Thread(name='slowThread', target=slowThread, args=(threadQ,)).start()
Thread(name='hangThread', target=hangThread, args=(threadQ,)).start()
QToQ(wdQueue, threadQ)
正如您所看到的,我需要將線程放入queue.Queue,而單獨的對象會慢慢將queue.Queue的輸出提供給多處理隊列.相反,如果我將線程直接放入多處理隊列中,或者在put之間沒有QToQ對象休眠,則多處理隊列將鎖定,并且在看門狗端看起來總是為空.
現在,由于多處理隊列應該是線程和進程安全的,我只能假設我在實現中搞砸了一些東西.我的解決方案似乎有效,但也覺得hacky足夠我覺得我應該解決它.
我正在使用Python 3.7.2,如果重要的話.
總結
以上是生活随笔為你收集整理的python队列是线程安全的吗_python – 为什么我的多进程队列看起来不是线程安全的?...的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 体系结构的意思是什么
- 下一篇: C语言字符串压缩之ZSTD算法怎么使用