當前位置：首頁 > 编程语言 > python >内容正文

python

问答机器人接口python_设计用于机器学习工程的Python接口

發布時間：2023/12/15 python 40 豆豆

生活随笔收集整理的這篇文章主要介紹了问答机器人接口python_设计用于机器学习工程的Python接口小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

問答機器人接口python

In order to do machine learning engineering, a model must first be deployed, in most cases as a prediction API. In order to make this API work in production, model serving infrastructure must first be built. This includes load balancing, scaling, monitoring, updating, and much more.

為了進行機器學習工程，必須首先部署模型，在大多數情況下，模型應作為預測API。為了使該API在生產中起作用，必須首先構建模型服務基礎結構。這包括負載平衡，擴展，監視，更新等等。

At first glance, all of this work seems familiar. Web developers and DevOps engineers have been automating microservice infrastructure for years now. Surely, we can repurpose their tools?

乍一看，所有這些工作似乎都很熟悉。 Web開發人員和DevOps工程師多年來一直在實現微服務基礎架構的自動化。當然，我們可以重新利用他們的工具嗎？

Unfortunately, we can’t—at least, not precisely.

不幸的是，我們不能-至少不是精確地。

While ML infrastructure is similar to traditional DevOps, it is just ML-specific enough to make standard DevOps tools suboptimal. This is why we built Cortex, our open source platform for machine learning engineering.

雖然ML基礎結構類似于傳統的DevOps，但它僅是特定于ML的，足以使標準DevOps工具欠佳。這就是為什么我們構建了Cortex(我們的機器學習工程開源平臺)的原因。

At a very high level, Cortex is designed to make it easy to deploy a model locally or to the cloud, automating all the underlying infrastructure. A central component of the platform is the Predictor interface—a programmable Python interface through which developers can write prediction APIs.

在很高的層次上，Cortex旨在簡化在本地或云中部署模型的過程，從而使所有基礎架構自動化。平臺的中心組件是Predictor接口，這是一個可編程的Python接口，開發人員可以通過該接口編寫預測API。

Designing a Python interface specifically for serving predictions as web requests was a challenge we spent months on (and are still improving). Here, I want to share some of the design principles we’ve developed so far:

設計一個專門用于將預測作為Web請求提供服務的Python接口是我們花了幾個月(并且還在不斷改進)的挑戰。在這里，我想分享我們到目前為止開發的一些設計原則：

1. Predictor只是一個Python類 (1. A Predictor is just a Python class)

At the core of Cortex’s architecture is our concept of a Predictor, which is essentially a prediction API, including all of the request handling code and dependencies. The Predictor interface enforces some simple requirements for those prediction APIs.

Cortex體系結構的核心是Predictor的概念， Predictor本質上是一種預測API，包括所有請求處理代碼和依賴項。 Predictor接口對這些預測API提出了一些簡單的要求。

Because Cortex takes a microservices approach to model serving, the Predictor interface is strictly concerned with two things:

由于Cortex采用微服務方法來建模服務，因此Predictor接口嚴格關注兩件事：

Initializing models
初始化模型
Serving predictions
服務預測

In that spirit, Cortex’s Predictor interface requires two functions, __init__() and predict(), that do more or less what you’d expect:

本著這種精神，Cortex的Predictor接口需要兩個函數__init__()和predict() ，它們或多或少地實現了您的期望：

After initialization, you can think of a Predictor as just a Python object, whose single predict() function gets called when users query an endpoint.

初始化之后，您可以將Predictor視為一個Python對象，當用戶查詢端點時，將調用該對象的單個predict()函數。

One of the big benefits of this approach is that it is intuitive to anyone with software engineering experience. There’s no need to touch your data pipeline or model training code. A model is just a file, and a Predictor is just an object that imports it and runs a predict() method.

這種方法的最大好處之一是，它對于具有軟件工程經驗的任何人都是直觀的。無需觸摸數據管道或模型訓練代碼。模型只是一個文件，而Predictor只是一個導入它并運行predict()方法的對象。

Beyond its syntactic appeal, however, the approach offers some key benefits in how it complements Cortex’s broader approach.

但是，除了語法上的吸引力外，該方法在補充Cortex的更廣泛方法方面提供了一些關鍵優勢。

2.預測只是一個HTTP請求 (2. A prediction is just an HTTP request)

One of the complexities of building an interface for serving predictions in production is that inputs will almost certainly differ, at least in format, from the model’s training data.

構建用于為生產中的預測提供服務的界面的復雜性之一是，輸入至少與模型的訓練數據至少在格式上肯定會有所不同。

This works on two levels:

這在兩個級別上起作用：

The body of a POST request is not a NumPy array or whatever data structure your model was trained to process.
POST請求的主體不是NumPy數組，也不是模型經過訓練可以處理的任何數據結構。
Machine learning engineering is all about using models to build software, which often times means using models on data they were not trained to process, e.g. using GPT-2 to write folk music.
機器學習工程就是使用模型來構建軟件，這通常意味著對未經訓練的數據使用模型，例如使用GPT-2 編寫民間音樂。

The Predictor interface, therefore, cannot be opinionated about the inputs and outputs of a prediction API. A prediction is just an HTTP request, and the developer is free to process it however they want. If, for example, they want to deploy a multi-model endpoint and query different models based on request params, they can do that:

因此，對于預測API的輸入和輸出，不能使用Predictor接口。預測只是一個HTTP請求，開發人員可以根據需要隨意處理它。例如，如果他們想部署多模型端點并根據請求參數查詢不同模型，則可以這樣做：

And while this interface gives developers freedom in terms of what they can do with their API, it also provides some natural scoping that enables Cortex to be more opinionated on the infrastructure side.

盡管此接口使開發人員可以自由使用API??進行操作，但它還提供了一些自然的作用域，使Cortex在基礎架構方面更具主張。

For example, under the hood Cortex uses FastAPI to setup request routing. There are a number of processes Cortex sets up at this layer that relate to autoscaling, monitoring, and other infrastructure features which could become very complex if developers were required to implement routing.

例如， Cortex在后臺使用FastAPI設置請求路由。 Cortex在此層設置了許多與自動縮放，監視和其他基礎結構功能有關的過程，如果需要開發人員實施路由，這些過程可能會變得非常復雜。

But, because each API has a single predict() method, every API has the same number of routes—one. Being able to assume this allows Cortex to do a lot more at the infrastructure level without limiting engineers.

但是，由于每個API都有一個單獨的predict()方法，所以每個API都有相同數量的路由-一個。能夠做到這一點使Cortex可以在基礎架構級別上做更多的事情，而不會限制工程師。

3.服務模型只是微服務 (3. A served model is just a microservice)

Scale is a chief concern for anyone using machine learning in production. Models can get huge (GPT-2 is roughly 6 GB), are computationally expensive, and can have high latency. Especially for realtime inference, scaling up to handle traffic is a challenge—even more so if you’re budget constrained.

對于在生產中使用機器學習的任何人來說，規模都是一個主要問題。模型可能會變得很大(GPT-2大約為6 GB)，計算量很大，并且可能具有高延遲。尤其是對于實時推理，擴展規模以處理流量是一個挑戰，如果您的預算有限，就更是如此。

To solve for this, Cortex treats Predictors as microservices, which can be horizontally scaled. More specifically, when a developer hits $ cortex deploy, Cortex containerizes the API, spins up a cluster provisioned for inference, and deploys. Then, it exposes the API as a web service behind a load balancer, and configures autoscaling, updating, and monitoring:

為了解決這個問題，Cortex將Predictors視為可以水平縮放的微服務。更具體地說，當開發人員點擊$ cortex deploy ，Cortex將容器化API，旋轉為推理而配置的集群，然后進行部署。然后，它將API作為Web服務公開在負載均衡器后面，并配置自動縮放，更新和監視：

Cortex DocsCortex Docs

The Predictor interface is fundamental to this process, even though it “just” a Python interface.

Predictor接口是此過程的基礎，即使它“只是” Python接口也是如此。

What the Predictor interface does is enforce a packaging of code in such a way that it becomes a single, atomic unit of inference. All the request handling code you need for a single API is contained within a Predictor. This makes it easy for Cortex to scale Predictors:

Predictor接口的作用是強制執行代碼打包，以使其成為單個原子的推理單元。單個API所需的所有請求處理代碼都包含在Predictor中。這使Cortex輕松縮放預測變量：

Cortex GitHubCortex GitHub

In this way, engineers don’t have to do any extra work—unless they want to tweak things, of course—to prepare an API for production. A Cortex deployment is production-ready by default.

這樣，工程師不必做任何額外的工作-當然，除非他們想進行調整-可以為生產準備API。默認情況下，Cortex部署可用于生產。

機器學習工程的界面-強調“工程” (An interface for machine learning engineering—emphasis on the “engineering”)

A constant theme in all of the above points is the balance of flexibility and ease-of-use. If the goal is to allow machine learning engineers to build whatever they want, Cortex needs to be:

上述所有方面的不變主題是靈活性和易用性之間的平衡。如果目標是允許機器學習工程師構建他們想要的任何東西，那么Cortex需要：

Flexible enough for engineers to implement any idea.
足夠靈活，工程師可以實施任何想法。
Opinionated enough to automate the infrastructure work that obstructs engineers.
自以為是足以使阻礙工程師的基礎架構工作自動化。

Striking this balance is a constant challenge, particularly as the world of machine learning engineering is young and constantly changing.

達到這種平衡是一個持續的挑戰，尤其是在機器學習工程世界還處于不斷變化之中的今天。

However, by focusing solely on what it takes to build software out of models—the “engineering” in machine learning engineering—we believe we can walk that tightrope.

但是，通過僅專注于從模型中構建軟件的需要(機器學習工程中的“工程”)，我們相信我們可以走鋼絲。

If contributing to the machine learning engineering ecosystem sounds interesting to you, we’re always happy to meet new contributors at Cortex.

如果對您的機器學習工程生態系統做出貢獻聽起來很有趣，我們很高興在Cortex結識新的貢獻者。

翻譯自: https://towardsdatascience.com/designing-a-python-interface-for-machine-learning-engineering-ae308adc4412