ai人工智能的本質和未來

The future looks towards running deep learning algorithms on more compact devices as any improvements in this space make for big leaps in the usability of AI.

未來的趨勢是在更緊湊的設備上運行深度學習算法，因為該領域的任何改進都將使AI的可用性取得重大飛躍。

If a Raspberry Pi could run large neural networks, then artificial intelligence could be deployed in a lot more places.

如果Raspberry Pi可以運行大型神經網絡，那么人工智能可以部署在更多地方。

Recent research in the field of economising AI has led to a surprisingly easy solution to reduce the size of large neural networks. It’s so simple, it could fit in a tweet:

在節省AI領域中的最新研究已導致出乎意料的簡單解決方案，以減小大型神經網絡的大小。 它非常簡單，可以在一條推文中顯示 ：

Train the Neural Network to Completion

訓練神經網絡完成

Globally prune the 20% of weights with the lowest magnitudes.

在全球范圍內修剪最低重量的20％重量。

Retrain with learning rate rewinding for the original training time.

以原始訓練時間的學習率倒帶進行再訓練。

Iteratively repeat steps 2 and 3 until the desired sparsity is reached.

反復重復步驟2和3，直到達到所需的稀疏度。

Further, if you keep repeating this procedure, you can get the model as tiny as you want. However, it’s pretty certain that you’ll lose some model accuracy along the way.

此外，如果繼續重復此過程，則可以根據需要獲得最小的模型。 但是，可以肯定的是，您將在此過程中損失一些模型精度。

This line of research grew out of the an ICLR paper last year (Frankle and Carbin’s Lottery Ticket Hypothesis) which showed that a DNN could perform with only 1/10th of the number of connections if the right subnetwork was found in training.

這項研究源于去年的ICLR論文(Frankle和Carbin的彩票假設 )，該論文表明，如果在訓練中找到正確的子網，則DNN只能執行連接數量的1/10的操作。

The timing of this finding coincides well with reaching new limitations in computational requirements. Yes, you can send a model to train on the cloud but for seriously big networks, along with considerations of training time, infrastructure and energy usage — more efficient methods are desired because they’re just easier to handle and manage.

這一發現的時機恰好與在計算要求上達到新的限制相吻合。 是的，您可以發送模型在云上進行訓練，但對于大型網絡，需要考慮訓練時間，基礎架構和能源使用情況，因此需要更高效的方法，因為它們更易于操作和管理。

Bigger AI models are more difficult to train and to use, so smaller models are preferred.

較大的AI模型更難訓練和使用，因此較小的模型是首選。

Following this desire for compression, pruning algorithms came back into the picture following the success of the ImageNet competition. Higher performing models were getting bigger and bigger but many researchers proposed techniques try keep them smaller.

隨著對壓縮的渴望，隨著ImageNet競賽的成功，修剪算法重新出現 。 性能更高的模型變得越來越大，但是許多研究人員提出了一些技術，試圖將它們縮小。

Yuhan Du on 玉函杜上UnsplashUnsplash

Song Han of MIT, developed a pruning algorithm for neural networks called AMC (AutoML for model compression) which removed redundant neurons and connections, when then the model is retrained to retain its initial accuracy level. Frankle took this method and developed it further by rewinding the pruned model to its initial weights and retrained it at a faster initial rate. Finally, in the ICLR study above, the researchers found that the model could be rewound to its early training rate and without playing with any parameters or weights.

麻省理工學院的宋瀚 ( Song Han)開發了一種稱為AMC( 用于模型壓縮的AutoML )的神經網絡修剪算法，該算法刪除了多余的神經元和連接，然后對其進行了重新訓練以保持其初始精度水平。 Frankle采用了這種方法，并通過將修剪后的模型重繞到其初始權重并以更快的初始速率對其進行了重新訓練來進一步開發了該方法。 最后，在上述ICLR研究中，研究人員發現該模型可以倒退至其早期訓練速度，而無需使用任何參數或權重。

Generally as the model gets smaller, the accuracy gets worse however this proposed model performs better than both Han’s AMC and Frankle’s rewinding method.

通常，隨著模型變小，精度會變差，但是此提議的模型的性能優于Han的AMC和Frankle的倒帶方法。

Now it’s unclear why this model works as well as it does, but the simplicity of it is easy to implement and also doesn’t require time-consuming tuning. Frankle says: “It’s clear, generic, and drop-dead simple.”

現在還不清楚為什么該模型能夠像它一樣運作良好，但是它的簡單性易于實現，并且不需要費時的調整。 弗蘭克(Frankle)說：“這很清楚，通用并且很簡單。”

Model compression and the concept of economising machine learning algorithms is an important field that we can make further gains in. Leaving models too large reduces the applicability and usability of them (I mean, you can keep your algorithm sitting in an API in the cloud) but there are so many constraints in keeping them local.

模型壓縮和節省機器學習算法的概念是我們可以進一步獲益的重要領域。模型過大會降低模型的適用性和可用性(我的意思是，您可以將算法保留在云中的API中)但是將它們保持在本地存在很多限制。

For most industries, models are often limited in their usability because they may be too big or too opaque. The ability to discern why a model works so well will not only enhance the ability to make better models, but also more efficient models.

對于大多數行業來說，模型的可用性通常受到限制，因為模型可能太大或太不透明。 辨別模型為何運作良好的能力不僅可以增強制作更好模型的能力，而且可以提高效率。

For neural nets, the models are so big because you want the model to naturally develop connections, which are being driven by the data. It’s hard for a Human to understand these connections but regardless, the understanding the model can chop out useless connections.

對于神經網絡，模型是如此之大，因為您希望模型自然地建立由數據驅動的連接。 對于人類而言，很難理解這些連接，但是無論如何，對模型的理解都可以消除無用的連接。

The golden nugget would be to have a model that can reason — so a neural network which trains connections based on logic, thereby reducing the training time and final model size, however, we’re some time away from having an AI that controls the training of AI.

金塊將是擁有一個可以推理的模型-因此，一個基于邏輯來訓練連接的神經網絡，從而減少了訓練時間和最終模型的大小，但是，我們距離控制訓練的AI還有一段距離AI。

Thanks for reading, and please let me know if you have any questions!

感謝您的閱讀，如果您有任何疑問，請告訴我！

Keep up to date with my latest articles here!

在這里了解我的最新文章！

翻譯自: https://towardsdatascience.com/the-future-of-ai-is-in-model-compression-145158df5d5e

ai人工智能的本質和未來

總結

以上是生活随笔為你收集整理的ai人工智能的本质和未来_人工智能的未来在于模型压缩的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。