如果您不将Docker用于数据科学项目,那么您将生活在1985年
重點(diǎn) (Top highlight)
One of the hardest problems that new programmers face is understanding the concept of an ‘environment’. An environment is what you could say, the system that you code within. In principal it sounds easy, but later on in your career you begin to understand just how difficult it is to maintain.
新程序員面臨的最困難的問題之一是了解“環(huán)境”的概念。 您可以說的是環(huán)境,即您在其中編碼的系統(tǒng)。 從原則上講,這聽起來很容易,但是在職業(yè)生涯的后期,您開始了解維護(hù)的難易程度。
The reason being is that libraries and IDE’s and even the Python Code itself goes through updates and version changes, then sometimes, you’ll update one library, and a separate piece of code will fail, so you’ll need to go back and fix it.
原因是庫(kù)和IDE甚至Python代碼本身都會(huì)進(jìn)行更新和版本更改,因此有時(shí)您將更新一個(gè)庫(kù),而另一段代碼將失敗,因此您需要返回并進(jìn)行修復(fù)它。
Moreover, if we have multiple projects being developed at the same time, there can be dependency conflicts, which is when things really get ugly as code fails directly because of another piece of code.
而且,如果我們同時(shí)開發(fā)多個(gè)項(xiàng)目,則可能存在依賴沖突,這是當(dāng)代碼由于另一段代碼而直接失敗時(shí),事情變得非常難看。
Also, say you want to share a project to a team mate working on a different OS, or even ship your project that you’ve built on your Mac to a production server on a different OS, would you have to reconfigure your code? Yes, you probably will have to.
另外,假設(shè)您想與在不同OS上工作的團(tuán)隊(duì)共享一個(gè)項(xiàng)目,或者甚至將在Mac上構(gòu)建的項(xiàng)目運(yùn)送到在不同OS上的生產(chǎn)服務(wù)器上,是否需要重新配置代碼? 是的,您可能必須這樣做。
So to mitigate any of these issues, containers were proposed as a method to separate projects and the environments that they exist within. A container is basically a place where an environment can run, separate to everything else on the system. Once you define what’s in your container, it becomes so much easier to recreate the environment, and even share the project with teammates.
因此,為了緩解這些問題中的任何一個(gè),提出了將containers作為一種將項(xiàng)目及其所處環(huán)境分開的方法。 一個(gè) container 基本上是一個(gè)可以運(yùn)行環(huán)境的地方,與系統(tǒng)上的所有其他地方分開。 一旦定義了container,中的container,就可以輕松地重新創(chuàng)建環(huán)境,甚至與隊(duì)友共享項(xiàng)目。
要求 (Requirements)
To get started, we need to install a few things to get set up:
首先,我們需要安裝一些東西進(jìn)行設(shè)置:
Windows or macOS: Install Docker Desktop
Windows或macOS: 安裝Docker桌面
Linux: Install Docker and then Docker Compose
Linux:先安裝Docker ,再安裝Docker Compose
容器化Python服務(wù) (Containerise a Python service)
Let’s imagine we’re creating a Flask service called server.py and let’s say the contents of the file are as follows:
假設(shè)我們正在創(chuàng)建一個(gè)名為server.py的Flask服務(wù),并假設(shè)文件的內(nèi)容如下:
from flask import Flaskserver = Flask(__name__)@server.route("/")
def hello():
return "Hello World!"if __name__ == "__main__":
server.run(host='0.0.0.0')
Now as I said above, we need to keep a record of the dependencies for our code so for this, we can create a requirements.txt file that can contain the following requirement:
現(xiàn)在,如上所述,我們需要記錄代碼的依賴關(guān)系,因此,我們可以創(chuàng)建一個(gè)requirements.txt文件,其中可以包含以下要求:
Flask==1.1.1So our package has the following structure:
因此,我們的軟件包具有以下結(jié)構(gòu):
app├─── requirements.txt
└─── src
└─── server.py
The structure is pretty logical (source kept is kept in a separate directory). To execute our Python program, all is left to do is to install a Python interpreter and run it.
該結(jié)構(gòu)非常合理(源代碼保存在單獨(dú)的目錄中)。 要執(zhí)行我們的Python程序,剩下要做的就是安裝一個(gè)Python解釋器并運(yùn)行它。
Now to run the program, we could run it locally but suppose we have 15 projects we’re working through — it makes sense to run it in a container to avoid any conflicts with any other projects.
現(xiàn)在要運(yùn)行該程序,我們可以在本地運(yùn)行它,但假設(shè)我們正在處理15個(gè)項(xiàng)目-在容器中運(yùn)行它以避免與任何其他項(xiàng)目發(fā)生任何沖突都是有意義的。
Let’s move onto containerisation.
讓我們進(jìn)入集裝箱化。
Photo by Victoire Joncheray on Unsplash Victoire Joncheray在Unsplash上拍攝的照片Docker文件 (Dockerfile)
To run Python code, we pack the container as a Docker image and then run a container based on it. So as follows:
要運(yùn)行Python代碼,我們將容器打包為Docker image ,然后基于該容器運(yùn)行一個(gè)容器。 因此如下:
Then create an image by the Docker builder
然后通過Docker構(gòu)建器創(chuàng)建image
The simple docker run <image> command then creates a container that is running an app
簡(jiǎn)單的docker run <image>命令然后創(chuàng)建一個(gè)運(yùn)行應(yīng)用程序的容器
Dockerfile的分析 (Analysis of a Dockerfile)
A Dockerfile is a file that contains instructions for assembling a Docker image (saved as myimage):
Dockerfile是一個(gè)文件,其中包含有關(guān)組裝Docker映像(保存為myimage )的說明:
# set base image (host OS)FROM python:3.8# set the working directory in the container
WORKDIR /code# copy the dependencies file to the working directory
COPY requirements.txt .# install dependencies
RUN pip install -r requirements.txt# copy the content of the local src directory to the working directory
COPY src/ .# command to run on container start
CMD [ "python", "./server.py" ]
A Dockerfile is compiled line by line so the builder generates an image layer and stacks it upon previous images.
Dockerfile是逐行編譯的,因此構(gòu)建器會(huì)生成圖像層并將其堆疊在先前的圖像上。
We can also observe in the output of the build command the Dockerfile instructions being executed as steps.
我們還可以在build命令的輸出中觀察到作為步驟執(zhí)行的Dockerfile指令。
$ docker build -t myimage .Sending build context to Docker daemon 6.144kBStep 1/6 : FROM python:3.8
3.8.3-alpine: Pulling from library/python
…
Status: Downloaded newer image for python:3.8.3-alpine
---> 8ecf5a48c789Step 2/6 : WORKDIR /code
---> Running in 9313cd5d834d
Removing intermediate container 9313cd5d834d
---> c852f099c2f9Step 3/6 : COPY requirements.txt .
---> 2c375052ccd6Step 4/6 : RUN pip install -r requirements.txt
---> Running in 3ee13f767d05
…
Removing intermediate container 3ee13f767d05
---> 8dd7f46dddf0Step 5/6 : COPY ./src .
---> 6ab2d97e4aa1Step 6/6 : CMD python server.py
---> Running in fbbbb21349be
Removing intermediate container fbbbb21349be
---> 27084556702b
Successfully built 70a92e92f3b5
Successfully tagged myimage:latest
Then, we can see that the image is in the local image store:
然后,我們可以看到該圖像在本地圖像存儲(chǔ)中:
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZE
myimage latest 70a92e92f3b5 8 seconds ago 991MB
During development, we may need to rebuild the image for our Python service multiple times and we want this to take as little time as possible.
在開發(fā)過程中,我們可能需要多次重建Python服務(wù)的映像,并且我們希望這樣做花費(fèi)盡可能少的時(shí)間。
Note: Docker and virtualenv are quite similar but different. Virtualenv only allows you to switch between Python Dependencies but you’re stuck with your host OS. However with Docker, you can swap out the entire OS — install and run Python on any OS (think Ubuntu, Debian, Alpine, even Windows Server Core). Therefore if you work in a team and want to future proof your technology, use Docker. If you don’t care about it — venv is fine, but remember it’s not future proof. Please reference this if you still want more information.
注意: Docker和virtualenv非常相似,但有所不同。 Virtualenv只允許您在Py??thon依賴關(guān)系之間進(jìn)行切換,但是您對(duì)主機(jī)OS感到Virtualenv 。 但是,使用Docker ,您可以換出整個(gè)OS -在任何OS上安裝并運(yùn)行Python(請(qǐng)考慮使用Ubuntu,Debian,Alpine甚至Windows Server Core)。 因此,如果您在團(tuán)隊(duì)中工作,并且希望將來驗(yàn)證您的技術(shù),請(qǐng)使用Docker 。 如果您不關(guān)心它, venv很好,但是請(qǐng)記住,這并不是未來的證明。 如果您仍需要更多信息,請(qǐng)參考此內(nèi)容。
There you have it! We’ve shown how to containerise a Python service. Hopefully, this process will make it a lot easier and gives your project a longer shelf life as it’ll be less likely to come down with code-bugs as dependencies change.
你有它! 我們已經(jīng)展示了如何容器化Python服務(wù)。 希望這個(gè)過程將使它變得更容易,并為您的項(xiàng)目提供更長(zhǎng)的保存期限,因?yàn)殡S著依賴關(guān)系的改變,代碼錯(cuò)誤的可能性將降低。
Thanks for reading, and please let me know if you have any questions!
感謝您的閱讀,如果您有任何疑問,請(qǐng)告訴我!
Keep up to date with my latest articles here!
在這里了解我的最新文章!
翻譯自: https://towardsdatascience.com/youre-living-in-1985-if-you-don-t-use-docker-for-your-data-science-projects-858264db0082
總結(jié)
以上是生活随笔為你收集整理的如果您不将Docker用于数据科学项目,那么您将生活在1985年的全部?jī)?nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 做梦梦到自己辞职了是什么意思
- 下一篇: 梦到自己不来姨妈是怎么回事