Zhanxing Zhu
北京大学 数学科学学院助理教授
https://www.zhihu.com/question/385326992/answer/1164005349
当然随着逐渐发展，还有不同新的有意思的新方向、新应用出现。不过最终深度学习这个框架和方法最终会像最小二乘法一样，被广泛应用，成为标准工具，但会不断的有新的东西能挖出来。
-
Probing Brain Activation Patterns by Dissociating Semantics and Syntax in Sentences, AAAI 2020 -
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations, ICML 2019 -
Diverse Image-to-Image Translation via Disentangled Representations, ECCV 2018 -
大象：VAE系解纠缠：从VAE到βVAE，再到β-TCVAE：https://zhuanlan.zhihu.com/p/62312210
-
Unsupervised Data Augmentation for Consistency Training
-
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
-
MixMatch: A Holistic Approach to Semi-Supervised Learning
-
AutoAugment: Learning Augmentation Policies from Data
-
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
-
模型压缩和蒸馏。这块感觉做得不太够，bert做出来，效果真的好，但训练本身只是有钱人的游戏，如何能把模型的体积降下来，耗时降下来，无论是训练侧还是推理侧，都有很大的价值。其实，Hinton大神在14年已经提出了蒸馏的概念，近期也有很多关于bert的压缩，比较火的albert，新鲜出炉的Fastbert，但是感觉下降到理想水平还有一段距离。
-
样本依赖问题。绝大部分的优秀模型都依赖大量数据，其实很多人已经在这块尝试了很多方法降低这种依赖，然而其实做的人远远不如做模型的多，感觉工作很多能做但是出结果的似乎都不是很多，例如主动学习一查标杆文献基本都只有10年前后甚至更早的东西，博客谈的也只是科普（也不知道是不是我看的不够多）。 -
主动学习，边学边调整样本，但是个人看似乎还是依赖人工打标。 -
数据增强，通过特定规则增加一些数据，当然这些规则本身会带入过拟合信息。 -
半监督、弱监督、无监督。个人感觉这块有一定收益吧。
-
迁移学习、联邦学习、pre-training。谨慎的拿来注意，就问这么一个问题，所有的事情都得从零开始吗？从10开始然后做不香吗，github搬代码不香吗，能有一个地方能让自下载模型改吧改吧就完事，是不是很好，于是这些相关的思路，我们是有必要去发扬、优化的。
<pre style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="margin-right: 8px;margin-left: 8px;max-width: 100%;letter-spacing: 0.544px;white-space: normal;color: rgb(0, 0, 0);font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;widows: 1;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><strong style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;letter-spacing: 0.5px;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><strong style="max-width: 100%;font-size: 16px;letter-spacing: 0.544px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;letter-spacing: 0.5px;box-sizing: border-box !important;overflow-wrap: break-word !important;">—</span></strong>完<strong style="max-width: 100%;font-size: 16px;letter-spacing: 0.544px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;letter-spacing: 0.5px;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><strong style="max-width: 100%;font-size: 16px;letter-spacing: 0.544px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;letter-spacing: 0.5px;box-sizing: border-box !important;overflow-wrap: break-word !important;">—</span></strong></span></strong></span></strong></section><section style="max-width: 100%;letter-spacing: 0.544px;white-space: normal;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;widows: 1;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section powered-by="xiumi.us" style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="margin-top: 15px;margin-bottom: 25px;max-width: 100%;opacity: 0.8;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="max-width: 100%;letter-spacing: 0.544px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section powered-by="xiumi.us" style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="margin-top: 15px;margin-bottom: 25px;max-width: 100%;opacity: 0.8;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;color: rgb(127, 127, 127);font-size: 12px;font-family: sans-serif;line-height: 25.5938px;letter-spacing: 3px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;color: rgb(0, 0, 0);box-sizing: border-box !important;overflow-wrap: break-word !important;"><strong style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="max-width: 100%;font-size: 16px;font-family: 微软雅黑;caret-color: red;box-sizing: border-box !important;overflow-wrap: break-word !important;">为您推荐</span></strong></span></section><section style="margin: 5px 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;font-family: sans-serif;letter-spacing: 0px;opacity: 0.8;line-height: normal;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="font-size: 14px;">为什么魂斗罗只有128KB却可以实现那么长的剧情？</span><br style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;" /></section><section style="margin: 5px 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;font-family: sans-serif;letter-spacing: 0px;opacity: 0.8;line-height: normal;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="color: rgb(87, 107, 149);max-width: 100%;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;">MIT最新深度学习入门课，安排起来！</span></section><section style="margin: 5px 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;font-family: sans-serif;letter-spacing: 0px;opacity: 0.8;line-height: normal;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="color: rgb(87, 107, 149);max-width: 100%;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;">有了这个神器，轻松用 Python 写个 App</span></section><section style="margin: 5px 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;font-family: sans-serif;letter-spacing: 0px;opacity: 0.8;line-height: normal;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="color: rgb(87, 107, 149);max-width: 100%;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;">「最全」实至名归，NumPy 官方早有中文教程</span><br style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;" /></section><section style="margin: 5px 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;font-family: sans-serif;letter-spacing: 0px;opacity: 0.8;line-height: normal;box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="color: rgb(87, 107, 149);max-width: 100%;font-size: 14px;box-sizing: border-box !important;overflow-wrap: break-word !important;">我为什么鼓励你读计算机领域的博士？</span></section><p style="margin-right: 8px;margin-bottom: 5px;margin-left: 8px;padding-right: 0em;padding-left: 0em;max-width: 100%;min-height: 1em;color: rgb(127, 127, 127);font-size: 12px;font-family: sans-serif;line-height: 1.75em;letter-spacing: 0px;box-sizing: border-box !important;overflow-wrap: break-word !important;"><br /></p></section></section></section></section></section></section></section></section>
本篇文章来源于: 深度学习这件小事
本文为原创文章，版权归知行编程网所有，欢迎分享本文，转载请保留出处！
内容反馈