知行编程网知行编程网  2022-06-23 05:00 知行编程网 隐藏边栏 |   抢沙发  14 
文章评分 0 次,平均分 0.0

一篇文章浅析Python自带的线程池和进程池


我们都知道,不管是Java,还是C++,还是Go,还是Python,都是有线程这个概念的。

但是我们知道线程是不能随便创建的就像每招一个员工一样是有代价的无限制招人肯定最后各种崩溃。

所以通常情况下我们会引出线程池这个概念。

本质就是我就招了几个固定的员工给他们派活,某一个人的活干完了再去任务中心领取新的活。

防止任务太多一次性招太多工人最后系统崩溃。

开心一刻

理想的多线程

一篇文章浅析Python自带的线程池和进程池

实际的多线程

一篇文章浅析Python自带的线程池和进程池

from concurrent.futures import ...

可能也是因为线程池这个东西用的越来越多了吧从Python3.2+之后就成了内置模块

对的直接就能使用不需要pip进行安装什么的。

concurrent.futures下面主要有俩接口。

  • ThreadPoolExecutor 线程池。

  • ProcessPoolExecutor进程池。

这里可没有什么所谓的异步池

个人看法:虽然异步的性能很高但是目前除了Go以外其他实现的都不是太好用法上面有些怪异当然你们可以说我菜我承认。

线程池

示例代码

执行结果

一篇文章浅析Python自带的线程池和进程池

发现的问题

其实这个就是并发不要怀疑但是你有没有发现个问题main先执行这说明啥?

这说明main跑完之后是不管子线程的死活的。

那能不能设置一下所有的子线程都执行完之后main函数在执行完?

当然可以需要一个参数即可。

要完成上述的问题我们需要一个参数加上这个参数之后。

就可以让主线程等待所有子线程执行完之后主线程再执行完

示例代码

主要就是13行的pool.shutdown()

执行结果

一篇文章浅析Python自带的线程池和进程池

这次结果就是我们想要的了hhh!!!

add_done_callback可以理解为是回调函数线程执行完之后自动调用指定的回调函数。

并且能拿到线程执行函数的返回值

有什么用我也没用过怪我才疏学浅叭。

示例代码

注意:第172728行代码!

执行效果

一篇文章浅析Python自带的线程池和进程池

我想可能通常用在一些善后工作叭。


多进程方式

其实通过上述几个例子我们基本是知道怎么使用上面这个线程池了。

但是都知道Python的线程因为GIL(全局解释器锁)的原因是不能并发到多个物理核心上的。

所以是IO密集型的像爬虫,读写文件使用线程池是ok的。

但是如果说我就是野就是头铁非要用Python做计算型应用像图片压缩、视频流推送那没办法需要使用多进程池方式。

其实通过concurrent这个接口可以很方便的创建进程池只需要修改两个地方。

只需要修改这俩地方即可其他和上述用法一摸一样。


总结

本篇主要讲的是Python自带的线程池进程池

比较有特色的是ThreadPoolExecutorProcessPoolExecutor的接口是一样的。

只需要修改导入的包就行。

concurrent的接口主要有pool.submit(),pool.shutdown(),futrue.add_done_callback()

基本这几个都够自己用了。

如果在操作过程中有任何问题,欢迎在留言区吱一声。




近期热门文章


  • 30 行代码实现蚂蚁森林自动收能量(附送源码)

  • Linux 最常用命令:简单易学,但能解决 95% 以上的问题

  • 2021了,最流行的密码依旧是123456

  • 真香!用VS Code直接浏览GitHub代码 !

  • 从万众期待到口碑扑街!唐探3令人失望,用Python来分析一下大家的评论

  • 用Python分析了某大学2000千条表白墙,甜到发腻,原来脱单的秘密是……

  • 值得收藏!20个Python奇技淫巧,让你受益匪浅!

  • 这款IDEA插件刷爆了朋友圈,网友:这用起来有点酸爽~

  • 央视最美记者王冰冰火爆B站,单个视频1800万播放,都说了啥?

<p data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;min-height: 1em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><br style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"  /></p><ul class="list-paddingleft-2" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;overflow-wrap: break-word !important;"><section data-tools="135编辑器" data-id="94250" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;font-variant-numeric: normal;font-variant-east-asian: normal;letter-spacing: 0.544px;line-height: 27.2px;border-width: 0px;border-style: none;border-color: initial;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-tools="135编辑器" data-id="94250" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;letter-spacing: 0.544px;border-width: 0px;border-style: none;border-color: initial;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-tools="135编辑器" data-id="94250" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;letter-spacing: 0.544px;border-width: 0px;border-style: none;border-color: initial;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;border-width: 0px;border-style: none;border-color: initial;text-align: right;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)" style="max-width: 100%;display: inline-block;clear: both;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-brushtype="text" data-darkmode-bgimage="1" data-style="padding: 18px 15px 20px 10px; background-size: 100% 100%; background-image: url('https://www.zkxjob.com/wp-content/uploads/2022/06/wxsync-2022-06-a2a8a5e1e58f30392066a170034ee027.png'); color: rgb(86, 146, 214); text-align: center; letter-spacing: 1.5px; background-repeat: no-repeat;" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16072664870629="1" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16073544711184="1" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgb(86, 146, 214)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)|rgb(86, 146, 214)" data-darkmode-bgimage-16114842087086="1" style="padding: 18px 15px 20px 10px;max-width: 100%;background-size: 100% 100%;background-image: url('https://www.zkxjob.com/wp-content/uploads/2022/06/wxsync-2022-06-a2a8a5e1e58f30392066a170034ee027.png');color: rgb(86, 146, 214);text-align: center;letter-spacing: 1.5px;background-repeat: no-repeat;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-darkmode-bgimage="1" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16072664870629="1" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16073544711184="1" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgb(86, 146, 214)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)|rgb(86, 146, 214)" data-darkmode-bgimage-16114842087086="1" style="max-width: 100%;display: flex;justify-content: center;align-items: center;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section data-darkmode-bgimage="1" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16072664870629="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16072664870629="1" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgba(163, 163, 163, 0.9)" data-darkmode-original-color-16073544711184="rgba(230, 230, 230, 0.9)" data-darkmode-bgimage-16073544711184="1" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgb(86, 146, 214)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)|rgb(86, 146, 214)" data-darkmode-bgimage-16114842087086="1" style="margin-left: 2px;max-width: 100%;width: 20px;box-sizing: border-box !important;overflow-wrap: break-word !important;"></section><p data-darkmode-bgimage="1" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgb(51, 51, 51)" data-darkmode-original-color-16072664870629="rgb(51, 51, 51)" data-darkmode-bgimage-16072664870629="1" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgb(51, 51, 51)" data-darkmode-original-color-16073544711184="rgb(51, 51, 51)" data-darkmode-bgimage-16073544711184="1" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgb(51, 51, 51)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)|rgb(86, 146, 214)|rgb(51, 51, 51)" data-darkmode-bgimage-16114842087086="1" style="max-width: 100%;min-height: 1em;font-size: 14px;color: rgb(51, 51, 51);text-align: right;line-height: 1.5em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span data-darkmode-bgimage="1" data-darkmode-bgcolor-16072664870629="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16072664870629="rgb(255, 255, 255)" data-darkmode-color-16072664870629="rgb(51, 51, 51)" data-darkmode-original-color-16072664870629="rgb(51, 51, 51)" data-darkmode-bgimage-16072664870629="1" data-darkmode-bgcolor-16073544711184="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16073544711184="rgb(255, 255, 255)" data-darkmode-color-16073544711184="rgb(51, 51, 51)" data-darkmode-original-color-16073544711184="rgb(51, 51, 51)" data-darkmode-bgimage-16073544711184="1" data-darkmode-bgcolor-16114842087086="rgb(25, 25, 25)" data-darkmode-original-bgcolor-16114842087086="#fff|rgb(255, 255, 255)" data-darkmode-color-16114842087086="rgb(51, 51, 51)" data-darkmode-original-color-16114842087086="#fff|rgba(230, 230, 230, 0.9)|rgb(86, 146, 214)|rgb(51, 51, 51)" data-darkmode-bgimage-16114842087086="1" style="max-width: 100%;font-family: 楷体, 楷体_GB2312, SimKai;letter-spacing: 0.5px;box-sizing: border-box !important;overflow-wrap: break-word !important;">你点的每个“在看”,我都认真当成了AI</span></p></section></section></section></section></section></section></section></ul>

本篇文章来源于: 菜鸟学Python

本文为原创文章,版权归所有,欢迎分享本文,转载请保留出处!

知行编程网
知行编程网 关注:1    粉丝:1
这个人很懒,什么都没写

发表评论

表情 格式 链接 私密 签到
扫一扫二维码分享