標簽： python奇淫技巧

最優(yōu)線程數(shù)

Ncpu=CPU的數(shù)量

Ucpu=目標CPU使用率

W/C=等待時間與計算時間的比率

為保持處理器達到期望的使用率，最優(yōu)的線程池的大小等于

$$Nthreads=Ncpu*Ucpu*（1+W/C$$

cpu密集型任務，即$W<

如果希望CPU利用率為100%，則$Nthreads=Ncpu$

IO密集型任務，即系統(tǒng)大部分時間在跟I/O交互，而這個時間線程不會占用CPU來處理，即在這個時間范圍內，可以由其他線程來使用CPU，因而可以多配置一些線程。

混合型任務，二者都占有一定的時間

線城池

對于任務數(shù)量不斷增加的程序，每有一個任務就生成一個線程，最終會導致線程數(shù)量的失控。對于任務數(shù)量不端增加的程序，固定線程數(shù)量的線程池是必要的。

方法一：使用threadpool模塊

threadpool是一個比較老的模塊了，支持py2 和 py3 。

import threadpool

import time

def sayhello (a):

print("hello: "+a)

time.sleep(2)

def main():

global result

seed=["a","b","c"]

start=time.time()

task_pool=threadpool.ThreadPool(5)

requests=threadpool.makeRequests(sayhello,seed)

for req in requests:

task_pool.putRequest(req)

task_pool.wait()

end=time.time()

time_m = end-start

print("time: "+str(time_m))

start1=time.time()

for each in seed:

sayhello(each)

end1=time.time()

print("time1: "+str(end1-start1))

if __name__ == '__main__':

main(

方法二：使用concurrent.futures模塊

from concurrent.futures import ThreadPoolExecutor

import time

import time

from concurrent.futures import ThreadPoolExecutor, wait, as_completed

ll = []

def sayhello(a):

print("hello: "+a)

ll.append(a)

time.sleep(0.8)

def main():

seed=["a","b","c","e","f","g","h"]

start1=time.time()

for each in seed:

sayhello(each)

end1=time.time()

print("time1: "+str(end1-start1))

start2=time.time()

with ThreadPoolExecutor(2) as executor:

for each in seed:

executor.submit(sayhello,each)

end2=time.time()

print("time2: "+str(end2-start2))

def main2():

seed = ["a", "b", "c", "e", "f", "g", "h"]

executor = ThreadPoolExecutor(max_workers=10)

f_list = []

for each in seed:

future = executor.submit(sayhello, each)

f_list.append(future)

wait(f_list)

print(ll)

print('主線程結束')

def main3():

seed = ["a", "b", "c", "e", "f", "g", "h"]

with ThreadPoolExecutor(max_workers=2) as executor:

f_list = []

for each in seed:

future = executor.submit(sayhello, each)

f_list.append(future)

wait(f_list,return_when='ALL_COMPLETED')

print(ll)

print('主線程結束')

if __name__ == '__main__':

main3()

方法三：使用vthread模塊

demo1

import vthread

@vthread.pool(6)

def some(a,b,c):

import time;time.sleep(1)

print(a+b+c)

for i in range(10):

some(i,i,i)

demo2:分組線程池

import vthread

pool_1 = vthread.pool(5,gqueue=1) # open a threadpool with 5 threads named 1

pool_2 = vthread.pool(2,gqueue=2) # open a threadpool with 2 threads named 2

@pool_1

def foolfunc1(num):

time.sleep(1)

print(f"foolstring1, test3 foolnumb1:{num}")

@pool_2

def foolfunc2(num):

time.sleep(1)

print(f"foolstring2, test3 foolnumb2:{num}")

@pool_2

def foolfunc3(num):

time.sleep(1)

print(f"foolstring3, test3 foolnumb3:{num}")

for i in range(10): foolfunc1(i)

for i in range(4): foolfunc2(i)

for i in range(2): foolfunc3(i)

總結

以上是生活随笔為你收集整理的python并发处理list数据_3种方式实现python多线程并发处理的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。