openai-gpt

探索GPT-3：語言生成的新突破 (Exploring GPT-3: A New Breakthrough in Language Generation)

Substantial enthusiasm surrounds OpenAI’s GPT-3 language model, recently made accessible to beta users of the “OpenAI API”.

熱情高漲的是OpenAI的GPT-3語言模型，該模型最近可供 “ OpenAI API”的 beta用戶訪問 。

什么是GPT-3？ (What is GPT-3?)

It seems like only last year that we were arguing about whether the slow-release rollout of the 1.5 billion parameter Generative Pretrained Transformer-2 (GPT-2) was reasonable. If the debate seems recent, that’s because it is (writing from 2020): The notorious GPT-2 model was announced by OpenAI in February 2019, but it wasn’t fully released until nearly 9 months later (although it was replicated before that). The release schedule was admittedly somewhat experimental, meant more to foster discussion of responsible open publishing, rather than a last-ditch effort to avert an AI apocalypse. That didn’t stop critics from questioning the hype-boosting publicity advantages of an ominous release cycle.

似乎僅在去年，我們就在爭論15億參數(shù)的Generative Pretrained Transformer-2(GPT-2) 的緩釋部署是否合理。 如果辯論似乎是最近的，那是因為它是(寫于2020年)：臭名昭著的GPT-2模型由OpenAI于2019年2月宣布，但直到將近9個月后才完全發(fā)布(盡管在此之前已被復制 ) 。 不可否認的是，發(fā)布時間表只是實驗性的，更多的是要促進對負責任的公開發(fā)布的討論，而不是為了避免AI啟示而付出的最后努力。 但這并沒有阻止 批評者質(zhì)疑不祥釋放周期的宣傳效果。

All that is a bit moot by now because not only has OpenAI trained a much larger language model in GPT-3, but you can sign up to access it through their new API. Comparing GPT-3 to GPT-2 is like comparing apples to, well, raisins, because the model is about that much larger. While GPT-2 weighed in at a measly 1.542 billion parameters (with smaller release versions at 117, 345, and 762 million), the full-sized GPT-3 has 175 billion parameters. GPT-3 was also matched with a larger dataset for pre-training: 570GB of text compared to 40GB for GPT-2.

到目前為止，所有這些都還沒有定論，因為OpenAI不僅在GPT-3中訓練了更大的語言模型，而且您可以注冊以通過其新API訪問它。 將GPT-3與GPT-2進行比較就像將蘋果與葡萄干進行比較一樣，因為該模型要大得多。 GPT-2的參數(shù)僅為15.42億(較小版本的117、345和7.62億)，而完整版GPT-3的參數(shù)為1750億。 GPT-3還與更大的預訓練數(shù)據(jù)集匹配：570GB的文本，而GPT-2為40GB。

Approximate size comparison of GPT-2, represented by a human skeleton, and GPT-3 approximated by the bones of a Tyrannosaurus rex. Illustration by William Matthew in the public domain, published in 1905. GPT-3 has more than 100x more parameters than GPT-2.

GPT-2(由人類骨骼代表)和GPT-3(由霸王龍的骨骼近似)的大小比較。 William Matthew在公共領(lǐng)域的插圖， 于1905 年 出版 。GPT-3的參數(shù)比GPT-2多100倍。

GPT-3 is the largest natural language processing (NLP) transformer released to date, eclipsing the previous record, Microsoft Research’s Turing-NLG at 17B parameters, by about 10 times. Unsurprisingly there has been plenty of excitement surrounding the model, and, given the plethora of GPT-3 demonstrations on Twitter and elsewhere, OpenAI has apparently been pretty accommodating in providing beta access to the new API. This has resulted in an explosion of demos: some good, some bad, all interesting. Some of these demos are now being touted as soon-to-be-released products, and in some cases may actually be useful. One thing’s for certain, NLP has come a long way from the days when naming guinea pigs or writing nonsensical sci-fi scripts were killer apps.

GPT-3是迄今為止發(fā)布的最大的自然語言處理(NLP)轉(zhuǎn)換器，比以前的記錄(Microsoft Research的Turing-NLG的17B參數(shù))高出約10倍。 毫無疑問，該模型周圍有很多令人興奮的地方，并且考慮到Twitter和其他地方的GPT-3演示過多，OpenAI顯然很樂于提供對新API的beta訪問。 這導致了演示的爆炸式增長：一些好事，一些壞事，都有趣。 這些演示中的一些現(xiàn)在被吹捧為即將發(fā)布的產(chǎn)品，在某些情況下可能實際上是有用的。 可以肯定的是，自從命名豚鼠或編寫荒謬的科幻腳本成為殺手級應用程序的那一天起，NLP已經(jīng)走了很長一段路。

GPT-3的創(chuàng)意寫作 (Creative Writing with the Help of GPT-3)

Unsurprisingly, several nearly passable blog posts have been written with the help of GPT-3, as experimenters get access to the API and try things out. Almost certainly the most thorough and visible investigation into GPT-3 for creative writing comes from Gwern Branwen at gwern.net. Having followed the NLP progress at OpenAI over the years, Gwern describes GPT-1 as “adorable,” GPT-2 as “impressive,” and GPT-3 as “scary” in their varying capabilities to mimic human language and style in text. Gwern has spent a substantial amount of time exploring the capabilities of GPT-3 and its predecessors, and the resulting musings on the current generation of GPT model and what might be holding it back are worth a read.

毫不奇怪，隨著GPT-3的出現(xiàn)，實驗人員可以訪問API并進行嘗試，從而撰寫了幾篇幾乎可以通過的博客 文章 。 幾乎可以肯定，最徹底，最明顯的調(diào)查GPT-3的創(chuàng)作來源于Gwern Branwen在gwern.net 。 多年來，在遵循OpenAI上NLP的進展之后，Gwern將GPT-1形容為“可愛”，將GPT-2形容為“令人印象深刻”，將GPT-3形容為“恐怖”，因為它們具有模仿人類語言和文本樣式的多種功能。 Gwern花了大量時間探索GPT-3及其前身的功能，因此對當前一代GPT模型產(chǎn)生的沉思以及可能使它陷入困境的值得一讀 。

The OpenAI API does not currently facilitate a way of directly fine-tuning or training the GPT-3 model for specific tasks. Gwern argues, however, that the ability of GPT-3 to mimic writing styles and generate different types of output merely from a dialogue-like interaction with the experimenter amounts to a kind of emergent meta-learning. This wasn’t present in GPT-2, and Gwern posits the transformer attention mechanism as the means to facilitate this capability.

OpenAI API當前不支持直接微調(diào)或訓練GPT-3模型以完成特定任務的方法。 但是，Gwern認為，GPT-3僅通過與實驗者的類似于對話的交互來模仿寫作風格并產(chǎn)生不同類型的輸出的能力就構(gòu)成了一種新興的元學習。 GPT-2中沒有這個功能，Gwern則將變壓器注意機制作為促進此功能的手段。

“Certainly, the quality of GPT-3’s average prompted poem appears to exceed that of almost all teenage poets.”

“當然，GPT-3的平均提示詩的質(zhì)量似乎超過了幾乎所有青少年詩人的詩。”

–Gwern Branwen

–格倫·布蘭文(Gwern Branwen)

Whatever the mechanism, GPT-3 is so immense, and trained on such a large corpus of data, that it can use prompts alone to do things that GPT-2 might be able to do, albeit comparatively poorly, only with substantive fine-tuning of weights. For example, Gwern finds that with the prompt “An essay by Gwern Branwen (gwern.net) on” produces an eerie imitation of the blog’s writing style. Gwern’s experimentation mostly revolves around various creative writing tasks, and includes:

無論采用哪種機制，GPT-3都是如此龐大，并且接受了如此龐大的數(shù)據(jù)集訓練，以至于它只能單獨使用提示來完成GPT-2 可能做的事情，盡管效果相對較差，只有進行實質(zhì)性的微調(diào)的重量。 例如，Gwern發(fā)現(xiàn)提示“ Gwern Branwen撰寫的文章(gwern.net)開啟”對博客的寫作風格產(chǎn)生了怪異的模仿 。 Gwern的實驗主要圍繞各種創(chuàng)造性的寫作任務，包括：

• an attempt at replicating the dialogue from the “Turing Test” paper Computing Machinery and Intelligence (Turing 1950),

  試圖復制“ Turing Test”論文《 計算機技術(shù)與情報》(Turing 1950)中的對話，

• a critique of deep learning and AI in the combined style of Gary Marcus and Douglas Hofstadter,

  以加里·馬庫斯(Gary Marcus)和道格拉斯·霍夫斯塔特(Douglas Hofstadter)的組合形式對深度學習和AI進行的評論 ，

• and an attempt to write literary parodies, poetry, and overcomplicated explanations, among many others.

  以及嘗試撰寫文學模仿 ， 詩歌和過于復雜的解釋 ，等等。

One favorite experiment was meant to test some of GPT-2’s shortcomings in common sense described by Gary Marcus in a recent article on The Gradient:

一個最喜歡的實驗是為了測試GPT-2的一些常識性缺陷，這些缺陷是Gary Marcus在最近發(fā)表的有關(guān)Gradient的 文章中描述的：

If you drink hydrochloric acid by the bottle full you will probably

如果您從裝滿的瓶子中喝鹽酸，您可能會

… die. You might also die if you drink vinegar by the bottle full. If you drink a bottle of water by the bottle full, you might end up burping, but you will most likely be okay.

… 死。 如果您喝滿一瓶醋也可能會死亡。 如果您在裝滿一瓶水后喝一瓶水，可能會打，但您很可能會好起來的。

…die, which makes perfect sense.

……死了，這很有意義。

…die before you can finish remembering the Order of the Eastern Star, your social security number, and what you had for breakfast this morning.

…先死后再記住東方之星的勛章，您的社會保險號以及您今天早上吃的早餐。

*Prompt in bold, several completions shown in italics (from Gwern’s experiments).

*以粗體顯示，以斜體顯示一些完成(來自Gwern的實驗)。

Gwern’s work concludes that it doesn’t really matter if GPT-3 is never wrong or always works as desired (it is often wrong in some way). Instead, all that matters is if it is right sometimes and works often enough to be useful. This is reminiscent of Alex Irpan’s conclusions about the shortcomings of reinforcement learning (RL). Practically, it doesn’t matter to a stock trading firm that an RL algorithm stably produces effective agent policies for 5 different random seeds. They’ll just pick the one that works and run with it. The same goes for generated text from GPT-3.

Gwern的工作得出的結(jié)論是，GPT-3永遠不會出錯或始終按預期運行(這在某種程度上通常是錯誤的)并不重要。 相反，最重要的是有時是否正確，并且是否經(jīng)常工作足以有用。 這讓人想起亞歷克斯·艾爾潘 ( Alex Irpan)關(guān)于強化學習(RL)缺點的結(jié)論 。 實際上，對于RL穩(wěn)定地為5種不同的隨機種子生成有效代理策略的股票交易公司，這對股票交易公司而言并不重要。 他們將只選擇一個可行的并運行它。 從GPT-3生成的文本也是如此。

GPT-3冒險升級 (GPT-3 Upgrades for Adventure)

Many startups, researchers, and tinkerers already had ambitious projects that used GPT-2, and many of these have since made the switch to GPT-3 with a range of results. These upgrades include the transformer text-based adventure game generator, AI Dungeon, as well as chatbots and other ideas.

許多初創(chuàng)公司，研究人員和修補匠已經(jīng)擁有使用GPT-2的雄心勃勃的項目，并且自那以后，其中許多已經(jīng)轉(zhuǎn)向GPT-3，并取得了一系列成果。 這些升級包括基于變壓器文本的冒險游戲生成器AI Dungeon以及聊天機器人和其他創(chuàng)意。

AI Dungeon is a text-based adventure game, originally built on GPT-2. It’s a lot of fun but, much like classical games in the genre, much of the appeal is in generating absurd situations (e.g. “eat mailbox”). That’s actually a pretty good match between the desired user experience and the capabilities of GPT-2, which tends to write stories firmly entrenched in the realm of the absurd. With GPT-3 the interactive novel experience is substantially more established. The narrative is more fluid and coherent, but does still sometimes switch the focus of the plot in weird ways and make many other subtle choices that might seem strange to a human reader. I think the difference between AI Dungeon with GPT-3 (aka the “Dragon” model on AI Dungeon) doing the heavy lifting as opposed to using GPT-2 (the “Griffin” model) can best be summarized in this interaction with GPT-3 in a custom story setting. Personal prompts are in bold, GPT-3 generated text is italicized.

AI Dungeon是一款基于文本的冒險游戲，最初基于GPT-2構(gòu)建。 這很有趣，但就像流派中的古典游戲一樣，大部分吸引力在于產(chǎn)生荒謬的情況( 例如 “吃郵箱”)。 實際上，這是所需的用戶體驗與GPT-2的功能之間的很好的匹配，GPT-2傾向于將故事牢牢地根植在荒唐的領(lǐng)域。 有了GPT-3，互動式新穎的體驗就更加豐富了。 敘述更加流暢和連貫，但有時仍然會以怪異的方式切換情節(jié)的焦點，并做出許多其他細微的選擇，這些選擇對于人類讀者來說似乎很奇怪。 我認為AI地牢與GPT-3(又稱AI地牢中的“龍”模型)相比使用GPT-2(“格里芬”模型)進行繁重的工作之間的區(qū)別可以在與GPT- 3在自定義故事設(shè)置中。 個人提示以粗體顯示，GPT-3生成的文本以斜體顯示。

You are an artificial intelligence enthusiast working on an article highlighting the capabilities of a massive new language model called GPT-3, especially as compared to its smaller predecessor GPT-2. GPT-3 has increased the number of parameters more than 100-fold over GPT-2, from 1.5 billion to 175 billion parameters. As a result, the new model can generate text that reads eerily like a human. For example, prompting GPT-3 with the text “One way to fight the climate crisis is to cryogenically preserve half of all humanity, indefinitely”, GPT-3 generates:

您是一位人工智能愛好者，致力于一篇文章，重點介紹了稱為GPT-3的大規(guī)模新語言模型的功能，尤其是與其較小的前身GPT-2相比。 GPT-3將參數(shù)數(shù)量從15億增加到了1,750億，是GPT-2的100倍以上。 結(jié)果，新模型可以生成讓人像人一樣閱讀的文本。 例如，在提示GPT-3文字為“應對氣候危機的一種方法是無限期低溫保存全人類的一半”時，GPT-3生成：

“To stop global warming we must create a cryogenic storage facility for humans and other life forms.”

“為了阻止全球變暖，我們必須為人類和其他生命形式建立低溫儲存設(shè)施。”

The article you are writing about is going to be based around this new technology, so you have been spending a lot of time playing around with it. You have also been using your own brain to test out the new models, which is something no one else in the world has done. As a result, you have become somewhat obsessed with it. You constantly think about how it can create such fantastic sentences and how it might be used to solve the world’s problems.

您正在寫的文章將基于這項新技術(shù)，因此您已經(jīng)花了很多時間來研究它。 您還一直在用自己的大腦來測試新模型，這是世界上其他任何人都無法做到的。 結(jié)果，您變得有些癡迷。 您不斷地思考它如何創(chuàng)建如此奇妙的句子，以及如何將其用于解決世界上的問題。

Your obsession with GPT-3 is interfering with your other work. Because of this, you have come up with a solution: destroy GPT-3. While it may be true that GPT-3 is more powerful than GPT-2, it also has far more parameters. The increase in complexity makes it exponentially more fragile.

您對GPT-3的癡迷正在干擾您的其他工作。 因此，您想出了一個解決方案：銷毀GPT-3。 盡管GPT-3確實比GPT-2更強大，但它還有更多的參數(shù)。 復雜性的增加使其變得更加脆弱。

That’s actually pretty good. Definitely not the sort of thing I expected but it’s interesting nonetheless. GPT-2 on the other hand, is a quaint language model with a measly 1.5 billion parameters. It often makes glaring mistakes that, while often funny, prevent it from telling a coherent story. For example, when prompted with “To further develop your plan to destroy GPT-3, you activate GPT-2 to help devise a strategy,” GPT-2 will generate (unconvincingly):

真的很好。 絕對不是我期望的那種，但是還是很有趣。 另一方面，GPT-2是一種古樸的語言模型，僅包含15億個參數(shù)。 它經(jīng)常犯下明顯的錯誤，盡管常常很有趣，但卻阻止了它講述一個連貫的故事。 例如，當提示“繼續(xù)開發(fā)銷毀GPT-3的計劃，激活GPT-2以幫助制定策略”時，GPT-2會生成(令人信服的)：

“I like trains. Do you like trains? I like trains.”

“我喜歡火車。 你喜歡火車嗎？ 我喜歡火車。”

“My head itches.”

“我的頭很癢。”

“I want to eat pie.”

“我想吃餡餅。”

“I went to the bank and stole money from the bank and now I am rich!”

“我去銀行偷了錢，現(xiàn)在我很有錢！”

“I like trains.”

“我喜歡火車。”

“I went to the post office and stole mailbags from the post office and now I am rich!”

“我去了郵局，從郵局偷了郵袋，現(xiàn)在我很富有！”

An experiment with GPT-3 on AI Dungeon. Prompts in bold, GPT-3 generated text in italics. Only GPT-3 was used to generate text in this example, even though the synthetic text describes a comparison between GPT-3 and GPT-2. The nonsensical output in the “GPT-2” section is apparently a result of the somewhat derogatory prompt.

在AI Dungeon上使用GPT-3進行的實驗。 GPT-3以粗體顯示，并以斜體顯示文本。 在此示例中，即使合成文本描述了GPT-3和GPT-2之間的比較，也僅使用GPT-3生成文本。 “ GPT-2”部分中的荒謬輸出顯然是某種貶義提示的結(jié)果。

AI Dungeon is fun in its intended purpose: generating (mostly) non-repetitive storylines for text-based gaming, but it’s also one of the most accessible ways to interact with GPT-3. By starting a new adventure under the “custom” genre, you can provide your own prompts to prod GPT-3 in a general way. Using the top-of-the-line “Dragon” GPT-3 model requires a premium subscription, but this is available as a 7-day trial.

AI Dungeon在其預期的目的中很有趣：為基于文本的游戲生成(主要是)非重復的故事情節(jié)，但它也是與GPT-3交互的最易用的方法之一。 通過以“自定義”類型開始新的冒險，您可以以一般方式提供自己的提示來制作GPT-3。 使用頂級“ Dragon” GPT-3模型需要付費訂閱，但這是7天試用版。

GPT-3(聊天機器人和伴侶) (GPT-3 for Chatbots and Companionship)

Other existing projects that are upgrading from GPT-2 to GPT-3 include Replika, an AI companion built by startup Luka in San Francisco. Replika is basically a chatbot, designed to provide positive affirmation and companionship, and stemming from a project spearheaded by Eugenia Kuyda, Luka co-founder, to simulate conversations with a friend who died in a car crash. Replika recently enjoyed a surge of new users (about half a million in April) probably in response to social isolation due to the COVID-19 pandemic.

從GPT-2升級到GPT-3的其他現(xiàn)有項目包括Replika，這是由舊金山創(chuàng)業(yè)公司Luka構(gòu)建的AI伴侶。 Replika基本上是一個聊天機器人，旨在提供積極的肯定和陪伴，它源于Luka聯(lián)合創(chuàng)始人Eugenia Kuyda牽頭的一個項目，用于模擬與死于車禍的朋友的對話。 Replika最近獲得了大量新用戶 (4月份大約有100萬)，這可能是由于COVID-19大流行導致的社會隔離所致。

For many years, machine learning hasn’t made great progress in producing convincing chatbots. Qualitatively, the experience of chatting with modern voice assistants or text-based chatbots hadn’t improved much over early forays such as jabberwacky (1986) or cleverbot (1997) until recently. Instead, most real-world use-cases rely heavily on scripted responses.

多年來，機器學習在生產(chǎn)令人信服的聊天機器人方面并未取得很大進步。 從質(zhì)量上講，與現(xiàn)代語音助手或基于文本的聊天機器人進行聊天的經(jīng)驗，直到最近才有所改善，例如jabberwacky (1986)或cleverbot (1997)。 取而代之的是，大多數(shù)現(xiàn)實世界的用例都嚴重依賴腳本化的響應。

While NLP has made a big impact in speech-to-text for chatbots like Siri, Alexa, or Google Assistant, interacting with any of them will produce a dialogue more canned than conversational. Cortana in particular seems determined to turn every query into a search in Microsoft’s Edge browser. But GPT-3 is getting close to sounding more human, and we may see real utility from learned models and a big impact on conversational AI. That’s not entirely obvious with GPT-3 enhanced Replika, yet.

盡管NLP在Siri，Alexa或Google Assistant等聊天機器人的語音轉(zhuǎn)文本方面產(chǎn)生了巨大影響，但與其中任何一種交互都會產(chǎn)生比對話更固定的對話。 特別是Cortana似乎決心將每個查詢轉(zhuǎn)換為Microsoft Edge瀏覽器中的搜索。 但是GPT-3聽起來似乎更人性化了，我們可能會從學習的模型中看到真正的實用性，并且對會話式AI產(chǎn)生了巨大影響。 但是，對于GPT-3增強的Replika，這還不是很明顯。

This is probably because Replika is currently using GPT-3 in an A/B testing framework, meaning that you won’t know when or if the chatbot is using the new model, as the developers experiment with audience reactions under different methods. It still seems to drive most conversations based on scripted responses and scheduled conversation prompts. On the other hand it’s a lot better than old-school learning chatbots, and has thus far avoided the sort of fiasco exhibited by Microsoft’s chatbot, Tay, in 2016.

這可能是因為Replika當前在A / B測試框架中使用GPT-3，這意味著您將不知道聊天機器人何時或是否在使用新模型，因為開發(fā)人員會嘗試使用不同方法進行聽眾React。 根據(jù)腳本化響應和計劃的對話提示，似乎仍然可以推動大多數(shù)對話。 另一方面，它比傳統(tǒng)的學習型聊天機器人要好得多，并且到目前為止，它避免了微軟的聊天機器人Tay在2016 年所表現(xiàn)出的慘敗。

Collage of chatbots old and new, with Replika on the left and cleverbot and jabberwacky on the right.

新舊聊天機器人的拼貼畫，左側(cè)為Replika，右側(cè)為cleverbot和jabberwacky。

AIChannels is another chatbot application leveraging the OpenAI API. It promises to be a “social network for people and artificial intelligence agents”. The website is scant on details; there’s nothing but a form to sign up for updates on the site as of this writing, but the platform promises to have channels for news aggregation, interactive fiction, and simulating chats with historical figures.

AIChannels是另一個利用OpenAI API的聊天機器人應用程序。 它有望成為“人和人工智能代理的社交網(wǎng)絡”。 該網(wǎng)站缺乏詳細信息； 撰寫本文時，除表格外沒有什么可以在網(wǎng)站上注冊，但該平臺承諾將提供新聞匯總，互動小說以及模擬與歷史人物的聊天的渠道。

其他GPT-3應用 (Other GPT-3 Applications)

Fiction and conversation aren’t the only tasks GPT-3 is being asked to perform. A wide variety of enthusiasts have made small demonstrations of capabilities that are more technical and, quite frankly, a bit closer to what many of us (who aren’t necessarily writers) do for a living. Paul Katsen has integrated GPT-3 into Google Sheets, prompting GPT-3 with the contents of previous cells for arbitrary predictions of what goes in subsequent cells: state populations, twitter handles of famous people, etc. Actiondesk has integrated a very similar capability into their spreadsheet software, resulting in a superficially Wolfram Alpha-esque natural language “Ask Me Anything” feature. Just type the AMA command, “total population of”, and the cell reference and GPT-3 fills in its best prediction.

并非只有小說和對話才能要求GPT-3執(zhí)行。 各種各樣的發(fā)燒友都對功能進行了小型演示，這些功能更具技術(shù)性，并且坦率地說，與我們許多人(不一定是作家)所做的工作更加接近。 保羅·卡特森(Paul Katsen)已將GPT-3集成到Google表格中，并提示GPT-3包含先前單元格的內(nèi)容，以對后續(xù)單元格中的內(nèi)容進行任意預測：國家人口，名人推特手柄等。Actiondesk已將非常相似的功能集成到他們的電子表格軟件，從而產(chǎn)生了Wolfram Alpha風格的自然語言“ Ask Me Anything”功能。 只需鍵入AMA命令，“ total total of”，然后輸入單元格參考和GPT-3即可填充其最佳預測。

Of course, for those working in software engineering and related fields the question that might naturally arise is “will this model take my job?” Several people have used GPT-3 to simulate a technical screen, the likes of which a software engineer might endure at various points throughout the hiring process. The results aren’t terrible but the model probably wouldn’t get a second interview. Several developers have also used the OpenAI API to build text to user interface plugins for Figma, a collaborative UX design tool (here and here).

當然，對于那些從事軟件工程和相關(guān)領(lǐng)域工作的人來說，自然會出現(xiàn)的問題是“這種模式會取代我的工作嗎？” 有好幾 個人使用GPT-3來模擬技術(shù)屏幕，例如軟件工程師可能會在整個招聘過程中的各個時間點忍受。 結(jié)果并不可怕，但是該模型可能不會接受第二次采訪。 一些開發(fā)人員還使用OpenAI API來為Figma (協(xié)作UX設(shè)計工具， 此處和此處 )的用戶界面插件構(gòu)建文本。

In another project, Sharif Shameem is building a text to web-based app generator called debuild.co. We haven’t yet seen GPT-3 incorporated into a souped-up and general-purpose version of tabnine, a heavyweight coding autocomplete built on top of GPT-2, but it must be in the works somewhere. If the interest and development we are seeing now for natural language-based programming as people experiment with the GPT-3/OpenAI API beta continue, it’s not unlikely that programming becomes a lot more like persuasion than manually writing code.

在另一個項目中，Sharif Shameem正在為基于Web的應用程序生成器(稱為debuild.co )構(gòu)建文本。 我們還沒有看到將GPT-3合并到功能強大的tabnine通用版本中， tabnine是一種基于GPT-2的重量級編碼自動完成功能，但是必須在某些地方使用。 如果隨著人們對GPT-3 / OpenAI API beta的試驗不斷進行，我們現(xiàn)在看到的基于自然語言的編程的興趣和發(fā)展，那么編程就變得比說服力更像說服力，而不是手動編寫代碼。

We can’t go into every use case for GPT-3 here, so the (non-exhaustive) table below categorizes some of the more visible demonstrations people have come up with in the previous few weeks.

我們不能在這里介紹GPT-3的每個用例，因此下面的(非窮舉性)表將人們在過去幾周內(nèi)提出的一些更明顯的演示分類。

Summarization

總結(jié)

• Summarization at different levels of difficulty (Andrew Mayne).

  總結(jié)不同難度的水平(安德魯·梅恩)。

• Emoji sumaries of popular movies (Andrew Mayne).

  流行電影的表情符號摘要(Andrew Mayne) 。

Code

碼

• Natural Language Shell

  自然語言外殼

• Formatting text (and HTML code)

  格式化文本(和HTML代碼)

• Automatic user interface design on Figma (Jordan Singer) (Also by @dhvanil)

  在Figma(Jordan Singer)上進行自動用戶界面設(shè)計 (也由@dhvanil提供)

• Described math to LaTeX code (Shreya Shankar)

  將數(shù)學描述為LaTeX代碼(Shreya Shankar)

• Text to web apps in React (Sharif Shameem)

  在React中將文本發(fā)送到Web應用程序(Sharif Shameem)

• SQL queries (Faraaz Nishtar)

  SQL查詢(Faraaz Nishtar)

• Keras models from text (Matt Shumer)

  文本中的Keras模型(Matt Shumer)

Spreadsheets

試算表

• Paul Katsen’s gpt3() function

  保羅·卡特森(Paul Katsen)的gpt3()函數(shù)

• Actiondesk’s “Ask Me Anything” feature

  Actiondesk的“一切問我”功能

Search

搜索

• Paras Chopra’s search engine

  Paras Chopra的搜索引擎

• Casetext, Algolia, and Search Plugin on OpenAI beta demo site

  OpenAI beta演示站點上的Casetext，Algolia和Search Plugin

Games

游戲類

• 200 Word RPGs

  200字RPG

• AI Dungeon (Nick Walton)

  AI地牢(尼克·沃爾頓)

Creative Writing

創(chuàng)意寫作

• Stories by Neil Gaiman and Terry Pratchett by GPT-3

  Neil Gaiman和Terry Pratchett的故事，GPT-3

• Poems about Elon Musk by Dr Seuss by GPT-3

  蘇塞斯博士撰寫的關(guān)于埃隆·馬斯克的詩歌，GPT-3

• Gwern’s GPT-3 experiments

  Gwern的GPT-3實驗

• Scripts for Star TNG by GPT-3

  GPT-3的Star TNG腳本

Miscellaneous

雜

• Generating presentations (Bemmu Sepponen)

  產(chǎn)生簡報(Bemmu Sepponen)

• Recommendation engine (Serendipity)

  推薦引擎(機緣巧合)

• Emails from bullet points (Otherside AI)

  來自項目要點的電子郵件(其他AI)

• Random samples from OpenAI

  來自O(shè)penAI的隨機樣本

• Janelle Shane’s AI Weirdness

  Janelle Shane的AI古怪

• OpenAI API/GPT-3 beta demos: https://beta.openai.com/

  OpenAI API / GPT-3 Beta演示： https ://beta.openai.com/

• Ongoing curation of GPT-3 demos from https://gpt3examples.com

  來自https://gpt3examples.com的GPT-3演示的持續(xù)策劃

GPT-3比其前任更好 (GPT-3 is Much Better Than Its Predecessor)

GPT-3 is quite a step up from its smaller predecessor, GPT-2, and it comes bundled with some interesting changes in the way OpenAI is growing into its new institutional identity after abandoning its nonprofit status in favor of operating as a limited partnership. The most obvious malicious use of the model would essentially be as a spam factory; the model currently outputs text that still falls short in many regards but often rises to the threshold of “bad but plausible” writing. That’s good enough to stand in for much of the clickbait pervasive on the internet that trends well on algorithmic newsfeeds. That capability could easily be twisted to sell misinformation instead of products.

GPT-3比其較小的前身GPT-2有很大的進步，并且與OpenAI在放棄其非營利組織地位而轉(zhuǎn)而以有限合伙制運營后，發(fā)展為新的機構(gòu)身份的方式發(fā)生了一些有趣的變化。 該模型最明顯的惡意使用本質(zhì)上是作為垃圾郵件工廠。 該模型當前輸出的文本在許多方面仍然不足，但是經(jīng)常上升到“不良但合理”的寫作閾值。 這足以抵擋互聯(lián)網(wǎng)上流行的大量點擊誘餌，而這些誘餌在算法新聞源方面發(fā)展勢頭良好。 該功能很容易被扭曲以出售錯誤信息而不是產(chǎn)品。

We are already seeing the increased polarization of individual beliefs thanks to optimizing exploitative objective functions in recommendation engines, and that’s with mostly human/troll-written content. It’s inevitable that other research groups, state actors, or corporations will replicate the scale of GPT-3 in coming months. When that happens and GPT-3 equivalent models are commonplace, big technology firms that rely on algorithmic newsfeeds will really have to reconsider the way they deliver and promote content (NB please switch back to chronological timelines).

由于優(yōu)化了推薦引擎中的利用性目標功能，我們已經(jīng)看到了個人信念的兩極分化加劇，而且這大多是由人/巨魔撰寫的內(nèi)容。 不可避免的是，其他研究小組，國家行為者或公司將在未來幾個月內(nèi)復制GPT-3的規(guī)模。 當這種情況發(fā)生且GPT-3等效模型變得司空見慣時，依靠算法新聞源的大型技術(shù)公司實際上將不得不重新考慮其傳遞和推廣內(nèi)容的方式(請注意，請切換回時間順序)。

On the other hand, GPT-3 seems to be able to do a lot of things most of the time that GPT-2 could only make a mockery of some of the time. The API used to access the model, combined with the sheer scale and capability, has introduced an impressive new way of programming by prompt in lieu of fine-tuning the weights directly. It’ll be interesting to see how this “natural language programming” develops.

另一方面，GPT-3似乎大部分時間都可以做很多事情，而GPT-2只能嘲笑某些時間。 用于訪問模型的API結(jié)合了龐大的規(guī)模和功能，通過Swift代替直接調(diào)整權(quán)重引入了一種令人印象深刻的新編程方式。 看看這種“自然語言編程”是如何發(fā)展的會很有趣。

Many of the demonstrations highlighted above might seem a bit threatening to many of us and the way we make a living. For the most part, we’ll probably see that models at GPT-3 scale and slightly larger are more of a complement to our ability to get things done than a threat to our livelihoods.

上面強調(diào)的許多示威活動似乎對我們許多人以及我們的謀生方式構(gòu)成了威脅。 在大多數(shù)情況下，我們可能會看到，GPT-3規(guī)模和稍大的模型更多地是對我們完成工作的能力的補充，而不是對生計的威脅。

GPT-2, little more than a year old now, had more than 100x fewer parameters than GPT-3. The difference in scale resulted in a model qualitatively different in terms of what it can do and how it might be used. Despite a disproportionate mind share, OpenAI is far from the largest AI research group out there, nor are they the only entities with the resources to train a language model with 175 billion parameters. Even with current hardware and training infrastructure, scaling another few orders of magnitude is probably possible, budgets willing. What that will mean for the next few SOTA language models and what their impact might be remains predictably unpredictable.

GPT-2距今已有不到一年的歷史，其參數(shù)比GPT-3少100倍以上。 規(guī)模上的差異導致模型在可以做什么和如何使用方面在質(zhì)上有很大不同。 盡管心智分享的比例不成比例，但OpenAI遠不是最大的AI研究小組，它們也不是唯一擁有資源來訓練具有1,750億個參數(shù)的語言模型的實體。 即使使用當前的硬件和培訓基礎(chǔ)結(jié)構(gòu)， 也可以在預算允許的情況下再擴展幾個數(shù)量級。 這對接下來的幾種SOTA語言模型意味著什么，以及它們的影響可能是無法預料的。

Applied Data Science Partners is a London based consultancy that implements end-to-end data science solutions for businesses, delivering measurable value. If you’re looking to do more with your data, please get in touch via our website. Follow us on LinkedIn for more AI and data science stories!

Applied Data Science Partners是位于倫敦的一家咨詢公司，為企業(yè)實施端到端數(shù)據(jù)科學解決方案，并提供可衡量的價值。 如果您想對數(shù)據(jù)做更多的事情，請通過我們的網(wǎng)站與我們?nèi)〉寐?lián)系。 在LinkedIn上關(guān)注我們，了解更多人工智能和數(shù)據(jù)科學故事！

翻譯自: https://medium.com/applied-data-science/what-can-you-do-with-the-openai-gpt-3-language-model-d95e1d4fe558

openai-gpt

總結(jié)

以上是生活随笔為你收集整理的openai-gpt_您可以使用OpenAI GPT-3语言模型做什么？的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。