神碼ai人工智能寫作機器人

A brief overview of the state-of-the-art in training ML models on devices. For a more comprehensive survey, read our full paper on this topic.

關于在設備上訓練ML模型的最新技術的簡要概述。 要進行更全面的調查，請閱讀 有關此主題的完整論文 。

We are surrounded by smart devices: from mobile phones and watches to glasses, jewelry, and even clothes. But while these devices are small and powerful, they are merely the tip of a computing iceberg that starts at your fingertips and ends in giant data and compute centers across the world. Data is transmitted from devices to the cloud where it is used to train models that are then transmitted back to be deployed back on the device. Unless used for learning simple concepts like wake words or recognizing your face to unlock your phone, machine learning is computationally expensive and data has no choice but to travel these thousands of miles before it can be turned into useful information.

智能設備包圍著我們 ：從手機和手表到眼鏡，珠寶，甚至衣服。 但是，盡管這些設備體積小巧，功能強大，但它們僅是計算冰山的一角，它觸手可及，并遍布全球的巨型數據和計算中心。 數據從設備傳輸到云，在云中用于訓練模型，然后將模型傳輸回去再部署回設備上。 除非用于學習諸如喚醒單詞之類的簡單概念或識別您的面部以解鎖手機，否則機器學習在計算上是昂貴的，數據別無選擇，只能經過數千英里才能轉化為有用的信息。

This journey from device to data center and back to device has its drawbacks. The privacy and security of user data is probably the most obvious as this data needs to be transmitted to the cloud and stored there, most often, indefinitely. Transmission of user data is open to interference and capture, and stored data leaves open the possibility of unauthorized access. But there are other significant drawbacks. Cloud-based AI and ML models have higher latencies, cost more to implement, lack autonomy, and, depending on the frequency of model updates, are often less personalized.

從設備到數據中心再回到設備的過程有其缺點。 用戶數據的隱私性和安全性可能是最明顯的，因為該數據需要無限期地傳輸到云并存儲在云中。 用戶數據的傳輸容易受到干擾和捕獲，存儲的數據使未經授權的訪問成為可能。 但是還有其他重大缺陷。 基于云的AI和ML模型具有更高的延遲，更高的實現成本，缺乏自治性，并且根據模型更新的頻率，通常不那么個性化。

As devices become more powerful, it becomes possible to address the drawbacks of the cloud model by moving some or all of the model development onto the device itself. This transfer of model development on to the device is usually referred to as Edge Learning or On-device Learning. The biggest roadblock to doing Edge Learning is model training which is the most computationally expensive part of the model development process especially in the age of deep learning. Speeding up training is possible either by adding more resources to the device or using these resources more effectively or some combination of the two.

隨著設備功能越來越強大，可以通過將部分或全部模型開發移至設備本身來解決云模型的缺點。 將模型開發轉移到設備上的過程通常稱為邊緣學習或設備上學習。 進行邊緣學習的最大障礙是模型訓練，這是模型開發過程中計算上最昂貴的部分，尤其是在深度學習時代。 通過向設備添加更多資源或更有效地使用這些資源或兩者的某種組合，可以加快培訓速度。

This transfer of model development on to the device is usually referred to as Edge Learning or On-device Learning.

將模型開發轉移到設備上的過程通常稱為邊緣學習或設備上學習。

Fig 1: A hierarchical view of the various approaches to edge/on-device learning. The boxes in grey are the topics covered in this article and corresponding paper. Image by Author圖1：邊緣/設備上學習的各種方法的層次結構圖。 灰色框是本文和相應論文中涉及的主題。 圖片作者

Fig 1 gives a hierarchical view of the ways to improve model training on devices. On the left are the hardware approaches that work with the actual chipsets. Fundamental research in this area aims at improving existing chip design (by developing chips with more compute and memory, and lower power consumption and footprint) or developing new designs with novel architectures that speed up model training. While hardware research is a fruitful avenue for improving on-device learning, it is an expensive process that requires large capital expenditure to build laboratories and fabrication facilities, and usually involves long timescales for development.

圖1給出了改進設備模型訓練的方法的分層視圖。 左側是與實際芯片組配合使用的硬件方法。 該領域的基礎研究旨在改進現有芯片設計(通過開發具有更多計算和內存以及更低功耗和占用空間的芯片)或開發具有新穎架構的新設計來加快模型訓練的速度。 雖然硬件研究是改善設備上學習的有效途徑，但它是一個昂貴的過程，需要大量資本支出來建立實驗室和制造設施，并且通常涉及較長的開發時間。

Software approaches encompass a large part of current work in this field. Every machine learning algorithm depends on a small set of computing libraries for efficient execution of a few key operations (such as Multiply-Add in the case of neural networks). The libraries that support these operations are the interface between the hardware and the algorithms and allow for algorithm development that is not based on any specific hardware architecture. However, these libraries are heavily tuned to the unique aspects of the hardware on which the operations are executed. This dependency limits the amount of improvement that can be gained by new libraries. The algorithms part of software approaches gets the most attention when it comes to improving ML on the edge as it involves the development and improvement of the machine learning algorithms themselves.

軟件方法涵蓋了該領域當前的大部分工作。 每種機器學習算法都依賴于一小組計算庫來有效執行一些關鍵操作(例如在神經網絡的情況下為乘加)。 支持這些操作的庫是硬件和算法之間的接口，并允許不基于任何特定硬件體系結構的算法開發。 但是，這些庫在很大程度上針對執行操作的硬件的獨特方面進行了調整。 這種依賴性限制了新庫可以實現的改進量。 當涉及到邊緣機器學習的改進時，軟件方法的算法部分將引起最多的關注，因為它涉及機器學習算法本身的開發和改進。

Finally, theoretical approaches help direct new research on ML algorithms. These approaches improve our understanding of existing techniques and their generalizability to new problems, environments, and hardware.

最后，理論方法有助于指導有關ML算法的新研究。 這些方法提高了我們對現有技術的理解以及它們對新問題，環境和硬件的一般性。

This article focuses on developments in algorithms and theoretical approaches. While hardware and computing libraries are equally important, given the long lead times for novel hardware and the interdependency between hardware and libraries, the state-of-the-art changes faster in the algorithms and theoretical spaces.

本文重點介紹算法和理論方法的發展。 盡管硬件和計算庫同等重要，但鑒于新型硬件的交貨時間較長以及硬件和庫之間的相互依賴關系，最新技術在算法和理論空間方面的變化更快。

演算法 (Algorithms)

Most of the work in on-device ML has been on deploying models. Deployment focuses on improving model size and inference speed using techniques like model quantization and model compression. For training models on devices, there needs to be advances in areas such as model optimization and Hyperparameter Optimization (HPO). But, advances in these fields improve accuracy and the rate of convergence, often at the expense of compute and memory usage. To improve model training on devices, it is important to have training techniques that are aware of the resource constraints under which these techniques will be run.

設備上ML的大部分工作都是在部署模型上進行的。 部署著重于使用模型量化和模型壓縮等技術來提高模型大小和推理速度。 對于在設備上訓練模型，需要在模型優化和超參數優化(HPO)等領域取得進步。 但是，這些領域的進步通常會以計算和內存使用為代價，提高準確性和收斂速度。 為了改善設備上的模型訓練，重要的是要有訓練技術，這些訓練技術應知道將在這些資源下運行這些資源。

To improve model training on devices, it is important to have training techniques that are aware of the resource constraints under which these techniques will be run.

為了改善設備上的模型訓練，重要的是要有訓練技術，這些訓練技術應知道將在這些資源下運行這些資源。

The mainstream approach to doing such resource-aware model training is to design ML algorithms that satisfy a surrogate software-centric resource constraint instead of a standard loss function. Such surrogate measures are designed to approximate the hardware constraints through asymptotic analysis, resource profiling, or resource modeling. For a given software-centric resource constraint, state-of-art algorithm designs adopt one of the following approaches:

進行這種資源感知模型訓練的主流方法是設計滿足替代軟件中心資源約束而不是標準損失函數的ML算法。 此類替代措施旨在通過漸近分析，資源配置文件或資源建模來近似估計硬件約束。 對于給定的以軟件為中心的資源約束，最新的算法設計采用以下方法之一：

Lightweight ML Algorithms — Existing algorithms, such as linear/logistic regression or SVMs, have low resource footprints and need no additional modifications for resource constrained model building. This low footprint makes these techniques an easy and obvious starting point for building resource-constrained learning models. However, in cases where the available device’s resources are smaller than the resource footprint of the selected lightweight algorithm, this approach will fail. Additionally, in many cases, lightweight ML algorithms result in models with low complexity that may fail to fully capture the underlying process resulting in underfitting and poor performance.

輕量級ML算法 -現有的算法(例如線性/邏輯回歸或SVM)具有較低的資源占用量，并且無需進行其他修改即可構建資源受限的模型。 這種低占用空間使這些技術成為構建資源受限的學習模型的簡單而明顯的起點。 但是，如果可用設備的資源小于所選輕量算法的資源占用量，則此方法將失敗。 此外，在許多情況下，輕量級ML算法導致模型的復雜度較低，可能無法完全捕獲基礎過程，從而導致擬合不足和性能不佳。

Reducing Model complexity — A better approach to control the size (memory footprint) and computation complexity of the learning algorithm is by constraining the model architecture (for e.g. by selecting a smaller hypothesis class). This approach has the added advantage that these models can be trained using traditional optimization routines. Apart from model building, this is one of the dominant approaches for deploying resource efficient models for model inference. Most importantly, this approach extends to even Deep Neural Networks (DNNs) where, as evidenced by Fig 2, there has been a slow but steady progression towards smaller, faster, leaner architectures. This progression has been helped by the increased use of Neural Architecture Search (NAS) techniques that show a preference for smaller, more efficient networks. Compared to the lightweight ML algorithms approach, model complexity reduction techniques can accommodate a broader class of ML algorithms and can more effectively capture the underlying process.

降低模型復雜度 —控制學習算法的大小(內存占用)和計算復雜度的更好方法是約束模型架構(例如，通過選擇較小的假設類別)。 這種方法的另一個優點是可以使用傳統的優化例程來訓練這些模型。 除模型構建外，這是為模型推理部署資源高效模型的主要方法之一。 最重要的是，這種方法甚至擴展到了深度神經網絡(DNN)，如圖2所示，在向更小，更快，更精簡的架構發展的過程中，進展緩慢但穩定。 越來越多地使用神經體系結構搜索(NAS)技術幫助實現了這一進步，這些技術顯示出對更小，更高效的網絡的偏愛。 與輕量級ML算法相比，模型復雜度降低技術可以容納更多種類的ML算法，并且可以更有效地捕獲底層過程。

Fig 2. Ball chart of the chronological evolution of model complexity. Top-1 accuracy is measured on the ImageNet dataset. The model complexity is represented by FLOPS and reflected by the ball size. The accuracy and FLOPS are taken from original publications of the models. The time of the model is when the associated publication is first made available online. Image by Junyao Guo.圖2.模型復雜度按時間順序演變的球狀圖。 Top-1準確性是在ImageNet數據集上測量的。 模型的復雜性由FLOPS表示，并由球的大小反映出來。 精度和FLOPS取自模型的原始出版物。 模型的時間是相關的出版物首次在線提供的時間。 郭俊堯

Modifying optimization routines — The most significant of the algorithmic advances is the design of optimization routines specifically for resource-efficient model building where resource constraints are incorporated during the model building (training) phase. Instead of limiting the model architectures beforehand, these approaches can adapt optimization routines to fit the resource constraints for any given model architecture (hypothesis class).

修改優化例程 -最先進的算法是專門針對資源效率較高的模型構建的優化例程的設計，在模型構建(訓練)階段將資源約束納入其中。 這些方法可以預先使用優化例程以適應任何給定模型體系結構(假設類)的資源約束，而不是預先限制模型體系結構。

Resource-constrained model-centric optimization routines focus on improving the performance of models that will be quantized after training either through stochastic rounding, weight initialization, or by introducing quantization error into gradient updates. Also prevalent are layer-wise training and techniques that trade computation for memory, both of which try to reduce the computational requirements associated with training DNNs. In certain cases, this approach can also dynamically modify the architecture to fit the resource constraints. Although this approach provides a wider choice of the class of models, the design process is still tied to a specific problem type (classification, regression, etc.) and depends on the selected method/loss function (linear regression, ridge regression for regression problems).

資源受限的以模型為中心的優化例程著重于提高模型的性能，這些模型將在訓練后通過隨機舍入，權重初始化或將量化誤差引入梯度更新來進行量化。 分層訓練和以內存換取計算的技術也很普遍，它們都試圖減少與訓練DNN相關的計算要求。 在某些情況下，此方法還可以動態修改體系結構以適應資源限制。 盡管此方法提供了更多的模型類別選擇，但設計過程仍與特定的問題類型(分類，回歸等)相關，并且取決于所選的方法/損失函數(線性回歸，針對回歸問題的嶺回歸) )。

Resource-constrained generic optimization routines such as Buckwild! And SWALP focuses on reducing the resource-footprint for model training by using low-precision arithmetic for gradient computations. An alternative line of work involves implementing fixed point Quadratic Programs (QP) such as QSGD or QSVRG for solving linear Model Predictive Control (MPC). Most of these algorithms involve modifying fast gradient methods for convex optimization to obtain a suboptimal solution in a finite number of iterations under resource-constrained settings .

資源受限的通用優化例程，例如Buckwild !！ SWALP致力于通過使用低精度算術進行梯度計算來減少模型訓練的資源占用。 另一種工作方式是實施定點二次程序(QP)，例如QSGD或QSVRG，以解決線性模型預測控制(MPC)。 這些算法大多數都涉及修改快速梯度方法以進行凸優化，以在資源受限的設置下以有限次數的迭代獲得次優解。

Data Compression — Rather than constraining the model size/complexity, data compression approaches target building models on compressed data. The goal is to limit the memory usage via reduced data storage and computation through fixed per-sample computation cost. A more generic approach includes adopting advanced learning settings that accommodates algorithms with smaller sample complexity. However, this is a broader research topic and is not just limited to on-device learning.

數據壓縮 - 數據壓縮不是限制模型的大小/復雜性，而是針對壓縮數據構建目標模型。 目的是通過減少數據存儲和通過固定的每樣本計算成本進行計算來限制內存使用。 更為通用的方法包括采用高級學習設置，以適應樣本復雜度較小的算法。 但是，這是一個更廣泛的研究主題，而不僅限于設備上的學習。

New protocols for data observation — Finally, completely novel approaches are possible that completely change the traditional data observation protocol (like the availability of i.i.d data in batch or online settings). These approaches are guided by an underlying resource-constrained learning theory which captures the interplay between resource constraints and the goodness of the model in terms of the generalization capacity. Compared to the above approaches, this framework provides a generic mechanism to design resource-constrained algorithms for a wider range of learning problems applicable to any method/loss function targeting that problem type.

數據觀察的新協議 —最后，完全新穎的方法可能會徹底改變傳統的數據觀察協議(例如，批量或在線設置中的iid數據的可用性)。 這些方法以一種潛在的資源受限學習理論為指導，該理論從泛化能力的角度捕獲了資源約束與模型優度之間的相互作用。 與上述方法相比，此框架提供了一種通用機制來設計資源受限算法，用于更廣泛的學習問題，適用于針對該問題類型的任何方法/損失函數。

ChallengesThe major challenge in algorithms research is proper software-centric characterization of the hardware constraints and the appropriate use of this characterization for better metric designs. If hardware dependencies are not properly abstracted away, the same model and algorithm can have very different performance profiles on different hardware. While novel loss functions can take such dependencies into account, it is still a relatively new field of study. The assumption in many cases is that the resource budget available for training does not change but that is usually never the case. Our everyday devices are often multi-tasking — checking emails, social media, messaging people, playing videos… the list goes on. Each of these apps and services are constantly vying for resources at any given moment in time. Taking this changing resource landscape into account is an important challenge for effective model training on the edge.

挑戰算法研究中的主要挑戰是對硬件約束進行正確的以軟件為中心的表征，以及為更好的度量設計而適當使用此表征。 如果沒有正確抽象出硬件依賴關系，則相同的模型和算法在不同的硬件上可能具有非常不同的性能。 盡管新穎的損失函數可以考慮這種依賴性，但它仍然是一個相對較新的研究領域。 許多情況下的假設是，可用于培訓的資源預算不會改變，但通常永遠不會改變。 我們的日常設備通常是多任務處理的-檢查電子郵件，社交媒體，消息傳遞者，播放視頻...等等。 這些應用程序和服務中的每一個都在任何給定的時間不斷爭奪資源。 考慮到這種不斷變化的資源格局，這是在邊緣進行有效模型訓練的重要挑戰。

Finally, improved methods for model profiling are needed to more accurately calculate an algorithm’s resource consumption. Current approaches to such measurements are abstract and focus on applying software engineering principles such as asymptotic analysis or low-level measures like FLOPS or MACs (Multiply-Add Computations). None of these approaches give a holistic idea of resource requirements and in many cases represent an insignificant portion of the total resources required by the system during learning.

最后，需要用于模型分析的改進方法來更準確地計算算法的資源消耗。 當前進行此類測量的方法是抽象的，并且側重于應用軟件工程原理，例如漸近分析或諸如FLOPS或MAC(乘加計算)的低級測量。 這些方法都沒有一個全面的資源需求概念，并且在許多情況下，它們在學習過程中只占系統所需總資源的很小一部分。

理論 (Theory)

Every learning algorithm is based on an underlying theory that guarantees certain aspects of its performance. Research in this area focuses mainly on Learnability — the development of frameworks to analyze the statistical aspects (i.e. error guarantees) of algorithms. While traditional machine learning theories underlie most current approaches, developing newer notions of learnability that include resource constraints will help us better understand and predict how algorithms will perform under resource-constrained settings. There are two broad categories of theories into which most of the existing resource-constrained algorithms can be divided

每種學習算法均基于保證其性能某些方面的基礎理論。 該領域的研究主要集中在可學習性上 —開發用于分析算法統計方面(即錯誤保證)的框架。 盡管傳統的機器學習理論是大多數當前方法的基礎，但發展包括資源約束在內的更新的可學習性概念將有助于我們更好地理解和預測算法在資源受限的環境下的性能。 有兩大類理論可將大多數現有的資源受限算法分為

Traditional Learning Theories — Most existing resource-constrained algorithms are designed following traditional machine learning theory (like PAC Learning Theory, Mistake Bounds, Statistical Query). A limitation of this approach is that such theories are built mainly for analyzing the error guarantees of the algorithm used for model estimation. The effect of resource constraints on the generalization capability of the algorithm is not directly addressed through such theories. For example, algorithms developed using the approach of reducing the model complexity typically adopts a two-step approach. First, the size of the hypothesis class is constrained beforehand to those that use fewer resources. Next, an algorithm is designed guaranteeing the best-in-class model within that hypothesis class. What is missing in such frameworks is the direct interplay between the error guarantees and the resource constraints.

傳統學習理論 -大多數現有資源受限的算法都是按照傳統的機器學習理論(如PAC學習理論，誤區，統計查詢)設計的。 這種方法的局限性在于，建立這種理論主要是為了分析用于模型估計的算法的誤差保證。 通過這種理論不能直接解決資源約束對算法泛化能力的影響。 例如，使用降低模型復雜度的方法開發的算法通常采用兩步法。 首先，假設類的大小事先限制為使用較少資源的類。 接下來，設計一種算法，以確保該假設類別內的同類最佳模型。 這種框架中缺少的是錯誤保證和資源約束之間的直接相互作用。

Resource-constrained learning theories — Newer learning theories try to overcome the drawbacks of traditional theories especially since new research has shown that it may be impossible to learn a hypothesis class under resource constrained settings. Most of the algorithms from earlier that assume new protocols for data observation fall in this category of resource-constrained theories. Typically, such approaches modify the traditional assumption of i.i.d data being presented in a batch or streaming fashion and introduces a specific protocol of data observability that limits the memory/space footprint used by the approach. These theories provide a platform to utilize existing computationally efficient algorithms under memory-constrained settings to build machine learning models with strong error guarantees. Prominent resource-constrained learning theories include Restricted Focus of Attention (RFA), newer Statistical Query (SQ) based learning paradigms, and graph-based approaches that model the hypothesis class as a hypothesis graph. Branching programs translate the learning algorithm under resource constraints (memory) in the form of a matrix (as opposed to a graph) where there is a connection between the stability of the matrix norm (in the form of an upper bound on its maximum singular value) and the learnability of the hypothesis class with limited memory. Although such theory-motivated design provides a generic framework through which algorithms can be designed for a wide range of learning problems, to date, very few algorithms based on these theories have been developed.

資源受限的學習理論 -較新的學習理論試圖克服傳統理論的弊端，特別是因為新的研究表明，在資源受限的環境下學習假設類是不可能的。 早先的大多數假設使用新協議進行數據觀察的算法都屬于這種資源受限的理論。 通常，此類方法修改了以批量或流方式呈現iid數據的傳統假設，并引入了數據可觀察性的特定協議，該協議限制了該方法使用的內存/空間占用量。 這些理論提供了一個平臺，可以在內存受限的設置下利用現有的高效計算算法來構建具有強大錯誤保證的機器學習模型。 突出的資源受限學習理論包括限制注意力集中(RFA)，更新的基于統計查詢(SQ)的學習范例，以及將假設類別建模為假設圖的基于圖的方法。 分支程序在資源約束(內存)下以矩陣(而不是圖)的形式轉換學習算法，其中矩陣范數的穩定性(以其最大奇異值的上限形式)之間存在聯系)和假設類的可記憶性有限。 盡管這種基于理論的設計提供了一個通用的框架，通過該框架可以針對各種學習問題設計算法，但迄今為止，基于這些理論的算法很少。

ChallengesPerhaps the biggest drawback to theoretical research is that while it is flexible enough to apply across classes of algorithms and hardware systems, it is limited due to the inherent difficulty of such research and the need to implement a theory in the form of an algorithm before its utility can be realized.

挑戰理論研究的最大缺點可能是，盡管它具有足夠的靈活性以適用于各種算法和硬件系統，但由于這種研究的固有困難以及需要以算法的形式實現理論而受到限制。它的效用可以實現。

結論 (Conclusion)

A future full of smart devices was the stuff of science fiction when we slipped the first iPhones into our pockets. Thirteen years later, devices have become much more capable and now promise the power of AI and ML right at our fingertips. However, these new-found capabilities are a facade propped up by massive computational resources (data centers, compute clusters, 4G/5G networks etc) that bring AI and ML to life. But devices can only be truly powerful on their own when it is possible to sever the lifeline that extends between them and the cloud. And that requires the ability to train machine learning models on these devices rather than in the cloud.

當我們將第一批iPhone放入口袋時，科幻小說充滿了未來的智能設備。 十三年后，設備變得更加強大，現在可以在指尖獲得AI和ML的強大功能。 但是，這些新發現的功能是使AI和ML栩栩如生的大量計算資源(數據中心，計算集群，4G / 5G網絡等)支撐的立面。 但是，只有在可以切斷在設備與云之間延伸的生命線的情況下，設備才能真正發揮真正的強大功能。 這就要求能夠在這些設備上而不是在云中訓練機器學習模型。

Training ML models on a device has so far remained an academic pursuit, but with the increasing number of smart devices and improved hardware, there is interest in performing learning on the device itself. In the industry, this interest is fueled mainly by hardware manufacturers promoting AI-specific chipsets that are optimized for certain mathematical operations, and startups providing ad hoc solutions to certain niche domains mostly in computer vision and IoT. From an AI/ML perspective, most of the activity lies in two areas — the development of algorithms that can train models under resource constraints and the development of theoretical frameworks that provide guarantees about the performance of such algorithms.

迄今為止，在設備上訓練ML模型仍是一項學術追求，但是隨著智能設備和硬件的改進，人們對在設備上進行學習感興趣。 在行業中，這種興趣主要是由硬件制造商推動的，這些制造商推廣了針對特定數學運算進行了優化的AI專用芯片組，并且初創公司主要在計算機視覺和IoT中為某些特定領域提供了臨時解決方案。 從AI / ML的角度來看，大多數活動都在兩個領域中：可以在資源約束下訓練模型的算法的開發以及為此類算法的性能提供保證的理論框架的開發。

At the algorithmic level, it is clear that current efforts are mainly targeted at either utilizing already lightweight machine learning algorithms or modifying existing algorithms in ways that reduce resource utilization. There are a number of challenges before we can consistently train models on the edge including the need for decoupling algorithms from the hardware, and designing effective loss functions and metrics that capture resource constraints. Also important are an expanded focus on traditional as well as advanced ML algorithms with low sample complexity and dealing with situations where the resource budget is dynamic rather than static. Finally, the availability of an easy and reliable way to profile algorithm behavior under resource constraints will speed up the entire development process.

在算法級別，很明顯，當前的工作主要針對利用已經很輕量級的機器學習算法或以減少資源利用的方式修改現有算法。 在我們不斷地在邊緣訓練模型之前，存在許多挑戰，包括需要將算法與硬件解耦，以及設計有效的損失函數和指標以捕獲資源約束的需求。 同樣重要的是，應進一步關注具有低樣本復雜度的傳統以及高級ML算法，并應對資源預算是動態而非靜態的情況。 最后，在資源限制下提供一種簡單可靠的方法來描述算法行為的方法將加速整個開發過程。

Learning theory for resource-constrained algorithms is focused on the un-learnability of an algorithm under resource constraints. The natural step forward is to identify techniques that can instead provide guarantees on the learnability of an algorithm and the associated estimation error. Existing theoretical techniques also mainly focus on the space(memory) complexity of these algorithms and not their compute requirements. Even in cases where an ideal hypothesis class can be identified that satisfies resource constraints, further work is needed to select the optimal model from within that class.

資源受限算法的學習理論集中于資源約束下算法的不可學習性。 向前邁出的自然步伐是確定可以替代地為算法的可學習性和相關的估計誤差提供保證的技術。 現有的理論技術也主要集中在這些算法的空間(存儲器)復雜度上，而不是它們的計算要求上。 即使在可以確定滿足資源約束的理想假設類別的情況下，也需要進一步的工作以從該類別中選擇最佳模型。

Despite these difficulties, the future for machine learning on the edge is exciting. Model sizes, even for deep neural networks, have been trending down. Major platforms such as Apple’s Core/CreateML support the retraining of models on the device. While the complexity and training regimen of models continue to grow, it is within the realm of possibility that we will continue to see a push to offload computation from the cloud to the device for reasons of privacy and security, cost, latency, autonomy, and better personalization.

盡管存在這些困難，但是邊緣機器學習的未來還是令人興奮的。 甚至對于深度神經網絡，模型的大小一直在下降。 蘋果的Core / CreateML等主要平臺支持對設備上的模型進行再培訓。 盡管模型的復雜性和訓練方案不斷增長，但出于隱私和安全性，成本，延遲，自治性和安全性的考慮，我們將繼續看到將計算從云上卸載到設備的可能性。更好的個性化。

This article was written with contributions from Sauptik Dhar, Junyao Guo, Samarth Tripathi, Jason Liu, Vera Serdiukova, and Mohak Shah.

本文是由Sauptik Dhar，Guunyao Guo，Samarth Tripathi，Jason Liu，Vera Serdiukova和Mohak Shah撰寫的。

If you are interested in a more comprehensive survey of edge learning, read our full paper on this topic.

如果您對邊緣學習的更全面的研究感興趣，請閱讀我們 關于該主題的全文 。

翻譯自: https://towardsdatascience.com/from-cloud-to-device-the-future-of-ai-and-machine-learning-on-the-edge-78009d0aee9

神碼ai人工智能寫作機器人

總結

以上是生活随笔為你收集整理的神码ai人工智能写作机器人_从云到设备到边缘的人工智能和机器学习的未来的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。