?關于?#今日arXiv精選?

這是「AI 學術前沿」旗下的一檔欄目，編輯將每日從arXiv中精選高質量論文，推送給讀者。

Fastformer: Additive Attention is All You Need

Category: NLP

Link:?https://arxiv.org/abs/2108.09084

Abstract

Transformer is a powerful model for text understanding. It is inefficient due to its quadratic complexity to input sequence length. In Fastformer, instead of modeling the pair-wise interactionsbetween tokens, we first use additive attention mechanism to model global contexts.

Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text ?Models

Category: NLP

Link:?https://arxiv.org/abs/2108.08877

Abstract

Sentence embeddings are broadly useful for language processing tasks. While T5 achieves impressive performance on language tasks cast assequence-to-sequence mapping problems, it is unclear how to produce sentences from encoder-decoder models. We investigate three methods for extracting T5 sentences.

Do Vision Transformers See Like Convolutional Neural Networks?

Category: Computer Vision

Link:?https://arxiv.org/abs/2108.0881

Abstract

Recent work has shown that (Vision) Transformer models (ViT) can achieve superior performance on image classification tasks. Are they acting like convolutional networks, or learning entirely different visual representations?

Is it Time to Replace CNNs with Transformers for Medical Images?

Category: Computer Vision

Link:?https://arxiv.org/abs/2108.09038

Abstract

Vision transformers (ViTs) have appeared as a competitive alternative to CNNs. ViTs possess several interestingproperties that could prove beneficial for medical imaging tasks. While CNNs perform better when trained from scratch, ViTs are on par with CNNs when pretrained on ImageNet.

Video Relation Detection via Tracklet based Visual Transformer

Category: Computer Vision

Link:?https://arxiv.org/abs/2108.08669

Abstract

Video Visual Relation Detection (VidVRD), has received significant attention over recent years. We apply the state-of-the-art video object tracklet detection pipeline MEGA and deepSORT. Then we perform VidVRD in a tracklet-based manner.

·

總結

以上是生活随笔為你收集整理的今日arXiv精选 | 近期必读的5篇Transformers相关论文的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。