不忘初心，砥礪前行?

作者 | 陌無崖

轉載請聯系授權?

Go + Service = One Goliath Project

Kevin Dangoor on December 20th

Khan Academy is embarking on a huge effort to rebuild our server software on a more modern stack in Go.
可汗學院正在努力著用Go重建著我們的服務器軟件

At Khan Academy, we don’t shy away from a challenge. After all, we’re a non-profit with a mission to provide a “free world-class education to anyone, anywhere”. Challenges don’t get much bigger than that.

在可汗學院，我們不會回避挑戰。畢竟，我們是一家非營利性組織，其使命是向“任何地方的任何人提供免費的世界一流的教育”。挑戰不會比這大得多。

Our mission requires us to create and maintain software to provide tools which help teachers and coaches who work with students, and a personalized learning experience both in and out of school. Millions of people rely on our servers each month to provide a wide variety of features we’ve built up over the past ten years.

我們的使命要求我們創建和維護軟件，以提供可幫助與學生一起工作的教師和教練的工具，并提供校內外的個性化學習體驗。每月有數百萬人依靠我們的服務器來提供我們在過去十年中建立的各種功能。

Ten years is a long time in technology! We chose Python as our backend server language and it has been a productive choice for us. Of course, ten years ago we chose Python 2 because Python 3 was still very new and not well supported.

十年是很長的技術！我們選擇Python作為我們的后端服務器語言，這對我們來說是一種高效的選擇。當然，十年前我們選擇了Python 2，因為Python 3仍然很新，并且沒有得到很好的支持。

The Python 2 end-of-life

Now, in 2019, Python 3 versions are dominant and the Python Software Foundation has said that Python 2 reaches its official end-of-life on January 1, 2020, so that they can focus their limited time fully on the future. Undoubtedly, there are still millions of lines of Python 2 out there, but the truth is undeniable: Python 2 is on its way out.

現在，在2019年，Python 3版本占據了主導地位，Python軟件基金會表示Python 2將于2020年1月1日正式終止使用壽命，以便他們可以將有限的時間完全用于未來。毫無疑問，仍然有數百萬行的Python 2，但事實是不可否認的：Python 2即將問世。

Moving from Python 2 to 3 is not an easy task. Beyond that hurdle, which has been widely written about elsewhere, we also have a bunch of other APIs in libraries we use which have undergone huge changes.

從Python 2遷移到3并非易事。除了在其他地方廣為介紹的這一障礙之外，我們在使用的庫中還有許多其他API，它們已經發生了巨大的變化。

All of these differences mean that we’d have to split our code to run in at least two services (the old Python 2 codebase and the Python 3 replacement) which can coexist during the transition.

所有這些差異意味著我們必須將代碼拆分為至少可以在過渡期間共存的兩個服務（舊的Python 2代碼庫和Python 3替代品）中運行。

For all of that work, we’d receive these benefits:

• Likely a 10-15% boost in backend server code performance

• Python 3’s language features

對于所有這些工作，我們將獲得以下好處：

• 后端服務器代碼性能可能提高10-15％

• Python 3的語言功能

Other languages

Given all of the work required and the relatively small benefits, we wanted to consider other options. We started using Kotlin for specific jobs within Khan Academy a year ago. Its performance benefits have saved us money, which we can apply in other ways to help people around the world learn. If we moved from Python to a language that is an order of magnitude faster, we can both improve how responsive our site isanddecrease our server costs dramatically.

鑒于所需的所有工作和相對較小的收益，我們希望考慮其他選擇。一年前，我們開始將Kotlin用于可汗學院內的特定工作。它的性能優勢為我們節省了金錢，我們可以采用其他方式來幫助世界各地的人們學習。如果我們從Python遷移到速度快一個數量級的語言，那么我們既可以改善站點的響應速度，也可以大大降低服務器成本。

Moving to Kotlin was an appealing alternative. While we were at it, we decided to dig deeper into other options. Looking at the languages that have first-class support in Google App Engine, another serious contender appeared: Go. Kotlin is a very expressive language with an impressive set of features. Go, on the other hand, offers simplicity and consistency. The Go team is focused on making a language which helps teams reliably ship software over the long-term.

搬到Kotlin是一個吸引人的選擇。在此期間，我們決定更深入地研究其他選擇。查看在Google App Engine中具有一流支持的語言，出現了另一個嚴重的競爭者：Go。Kotlin是一種非常富有表現力的語言，具有令人印象深刻的功能。另一方面，Go提供了簡單性和一致性。Go團隊致力于開發一種可幫助團隊長期可靠地發布軟件的語言。

As individuals writing code, we can iterate faster due to Go’s lightning quick compile times. Also, members of our team have years of experience and muscle memory built around many different editors. Go is better supported than Kotlin by a broad range of editors.

當個人編寫代碼時，由于Go閃電般的快速編譯時間，我們可以更快地進行迭代。同樣，我們團隊的成員具有多年的經驗，并且圍繞許多不同的編輯者都具有肌肉記憶力。與Gotlin相比，Go得到了廣泛的編輯者的更好支持。

Finally, we ran a bunch of tests around performance and found that Go and Kotlin (on the JVM) perform similarly, with Kotlin being perhaps a few percent ahead. Go, however, used a lot less memory, which means that it can scale down to smaller instances.

最后，我們圍繞性能進行了一系列測試，發現Go和Kotlin（在JVM上）的性能相似，其中Kotlin可能領先百分之幾。但是，Go使用的內存更少，這意味著它可以縮減為更小的實例。

We still like Python, but the dramatic performance difference which Go brings to us is too big to ignore, and we think we’ll be able to better support a system running on Go over the years. Moving to Go will undeniably be more effort than moving to Python 3, but the performance win alone makes it worth it.

我們仍然喜歡Python，但是Go帶給我們的巨大的性能差異太大了，不容忽視，并且我們認為多年來我們將能夠更好地支持在Go上運行的系統。無疑，向Go遷移比向Python 3遷移要付出更多的努力，但是僅憑性能方面的勝利就值得。

From monolith to services

With a few exceptions, our servers have historically all run the same code and can respond to a request for any part of Khan Academy. We use separate services for storing data and managing caches, but the logic for any request can be easily traced through our code and is the same regardless of which server responds.

除了少數例外，我們的服務器歷來都運行相同的代碼，并且可以響應可汗學院任何部分的請求。我們使用單獨的服務來存儲數據和管理緩存，但是任何請求的邏輯都可以通過我們的代碼輕松跟蹤，并且無論哪個服務器響應，邏輯都是相同的。

When a function calls another in a program, those calls are extremely reliable and very fast. This is a fundamental advantage of monoliths. Once you break up your logic into services, you’re putting slower, more fragile boundaries between parts of your code. You also have to consider how, exactly, that communication is going to happen. Do you put a publish/subscribe bus in between? Make direct HTTP or gRPC calls? Dispatch via some gateway?

當一個函數在程序中調用另一個函數時，這些調用非常可靠且非常快。這是很大的基本優點。將邏輯分解為服務后，您將在代碼的各個部分之間放慢速度，變得更脆弱。您還必須考慮通信將如何發生。您是否在兩者之間放置發布/訂閱總線？進行直接HTTP或gRPC調用？通過某些網關發送？

Even recognizing this added complexity, we re breaking up our monolith into services. There s an element of necessity to it, because new Go code would have to run in a separate process at least from our existing Python.

即使意識到這種增加的復雜性，我們也將整體細分為服務。這有一定的必要性，因為新的Go代碼必須至少與我們現有的Python在單獨的進程中運行。

The added complexity of services is balanced by a number of big benefits:

• By having more services which can be deployed independently, deployment and test runs can move more quickly for a single service, which means engineers will be able to spend less of their time on deployment activities. It also means they’ll be able to get changes out more quickly when needed.

• We can have more confidence that a problem with a deployment will have a limited impact on other parts of the site.

• By having separate services, we can also choose the right kinds of instances and hosting configuration needed for each service, which helps to optimize both performance and cost.

服務的復雜性增加了許多好處：

• 通過擁有更多可以獨立部署的服務，部署和測試運行可以針對單個服務更快地進行移動，這意味著工程師將能夠花費更少的時間在部署活動上。這也意味著他們將能夠在需要時更快地獲取更改。

• 我們可以更有信心，部署問題對站點其他部分的影響有限。

• 通過擁有單獨的服務，我們還可以選擇每種服務所需的正確種類的實例和托管配置，這有助于優化性能和成本。

We posted a series of blog posts (part 1, part 2, part 3) about how we had performed a significant refactoring of our Python code, drawing boundaries and creating constraints around which code could import which other code. Those boundaries provided a starting point for thinking about how we’d break our code into services. Craig Silverstein and Ben Kraft led an effort to figure out an initial set of services and how we would need to accommodate the boundaries between them.

我們發布了一系列博客文章（第1部分，第2部分，第3部分），介紹了我們如何對Python代碼進行重大重構，繪制邊界并圍繞哪些代碼可以導入其他代碼創建約束。這些界限為思考如何將代碼分解為服務提供了一個起點。克雷格·西爾弗斯坦（Craig Silverstein）和本·卡夫（Ben Kraft）致力于找出最初的服務集，以及我們將如何適應它們之間的界限。

In our current monolith, code is free to read and update any data models it needs to. To keep things sane, we made some rules around data access from services, but that’s a topic for another day.

在當前我們的服務，代碼可以自由讀取和更新所需的任何數據模型。為了使事情保持理智，我們制定了一些有關通過服務訪問數據的規則，但這又是一個話題。

Cleaning house

Ten years is a long time in technology. GraphQL didn’t exist in 2009, and two years ago we decided to migrate all of our HTTP GET APIs to GraphQL, later deciding to also adopt GraphQL mutations. We adopted React just after it was introduced, and it has spread to much of our web frontend. Google Cloud has grown in breadth of features. Server architectures have moved in the direction of independently deployable services.

十年是很長的技術。GraphQL在2009年不存在，兩年前，我們決定將所有HTTP GET API遷移到GraphQL，后來決定也采用GraphQL突變。引入React之后，我們就采用了React，它已經擴展到我們的許多Web前端。Google Cloud的功能廣泛。服務器體系結構已朝著可獨立部署的服務方向發展。

We’re going to do a lot of housecleaning in Python. We’re very aware of the second-system effect and our goal with this work is not to “create the perfect system” but rather to make it easier to port to Go. We started some of these technical migrations earlier, and some of them will continue on past the point at which our system is running in Go, but the end result will be more modern and coherent.

我們將在Python中進行大量的內部清理工作。我們非常了解這對第二系統的影響，我們開展這項工作的目標不是“創建完美的系統”，而是使其更容易移植到Go。我們較早地開始了其中的一些技術遷移，其中一些將繼續運行到我們的系統在Go中運行的地步，但最終結果將更加現代和一致。

• We’ll only generate web pages via React server side rendering, eliminating the Jinja server-side templating we’ve been using

• We’ll use GraphQL federation to dispatch requests to our services (and to our legacy Python code during the transition)

• Where we need to offer REST endpoints, we’ll do so through a gateway that converts the request to GraphQL

• We will rely more heavily on Fastly, our CDN provider, to enable more requests to be served quickly, closer to our users, and without requiring our server infrastructure to handle the request at all

• We’re going to deprecate some largely unused, outdated features that are an ongoing maintenance burden and would slow down our path forward

• 我們只會通過React服務器端渲染生成網頁，從而消除了我們一直在使用的Jinja服務器端模板。

• 我們將使用GraphQL聯合將請求分派到我們的服務（以及過渡期間的舊Python代碼）

• 在需要提供REST端點的地方，我們將通過網關將請求轉換為GraphQL。

• 我們將棄用一些未使用的，過時的功能，這些功能會帶來持續的維護負擔，并且會拖慢我們的前進速度

There are other things we might want to fix, but we re making choices that ultimately will help us complete the project more quickly and safely.

我們可能還需要修復其他問題，但是我們正在做出選擇，最終將幫助我們更快，更安全地完成項目。

What’s not changing

Everything I’ve described to this point is a huge amount of change, but there is a lot that we’re not changing. As much as possible, we’re going to port our logic straight from Python to Go, just making sure the code looks like idiomatic Go when it’s done.

到目前為止，我所描述的一切都是巨大的變化，但是很多事情我們都沒有改變。我們將盡可能地將邏輯從Python直接移植到Go，只需確保完成后代碼看起來像慣用的Go。

We’ve been using Google App Engine since day 1, and it has worked well for us and scaled automatically as we’ve grown. So, we’re going to keep using App Engine for our new Go services. We’re using Google Cloud Datastore as our database for the site, which is also staying the same. This also applies to the variety of other Google Cloud service we use, which have been performing well and scaling with our needs.

從第一天開始，我們就一直在使用Google App Engine，它對我們來說運作良好，并且隨著我們的成長而自動擴展。因此，我們將繼續為新的Go服務使用App Engine。我們將Google Cloud Datastore用作網站的數據庫，并且保持不變。這也適用于我們使用的其他各種Google Cloud服務，這些服務表現良好并且可以根據我們的需求進行擴展。

The plan

As of December 2019, we have our first few Go services running in production behind an Apollo GraphQL gateway. These services are pretty small today, because the way we’re doing the migration is very incremental. This incremental switchover is another good topic to talk about on another day (subscribe to our RSS feed or our Twitter account to read new posts as they go live).

截至2019年12月，我們在Apollo GraphQL網關后面開始運行了幾個Go服務。這些服務今天很小，因為我們進行遷移的方式是漸進式的。漸進式切換是另一天要討論的好話題（訂閱我們的RSS feed或我們的Twitter帳戶，以在新帖子上線時閱讀它們）。

For us, 2020 is going to be filled with technical challenge and opportunity: Converting a large Python monolith to GraphQL-based services in Go. We’re excited about this project, which we’ve named Goliath (you can probably imagine all of the “Go-” names we considered!). It’s a once in a decade opportunity to take a revolutionary step forward, and a big example of how we live our "We champion quality" engineering principle.

對我們來說，2020年將充滿技術挑戰和機遇：在Go中將大型Python整體組件轉換為基于GraphQL的服務。我們對這個名為Goliath的項目感到非常興奮（您可以想象我們考慮過的所有“ Go-”名稱！）。這是十年一次的革命性機遇，也是我們如何踐行“以質量為先”的工程原則的一個重要例子。

If you’re also excited about this opportunity, check out our careers page. As you can imagine, we’re hiring engineers!

如果您也對此機會感到興奮，請查看我們的職業頁面。正如你所想的那樣，我們正在招聘工程師！

本文為關于Golang相關博客文章的外文翻譯來源請點擊閱讀原文

END

今日推薦閱讀

RabbitMQ系列筆記廣播模式和路由模式?
RabbitMQ系列筆記入門篇

RabbitMQ系列筆記work模式

RabbitMQ系列筆記work模式

protoc語法詳解及結合grpc定義服務

Golang中Model的使用

基于Nginx和Consul構建高可用及自動發現的Docker服務架構

▼關注我，一起成長

????主要分享 學習心得、筆記、隨筆▼

????

總結

以上是生活随笔為你收集整理的可汗学院为什么选择Go的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。