Transformer architecture的解释

Go Forth And Transform

I hope you’ve found this a useful place to start to break the ice with the major concepts of the Transformer. If you want to go deeper, I’d suggest these next steps:

Read the Attention Is All You Need paper, the Transformer blog post (Transformer: A Novel Neural Network Architecture for Language Understanding), and the Tensor2Tensor announcement.
Watch Łukasz Kaiser’s talk walking through the model and its details
Play with the Jupyter Notebook provided as part of the Tensor2Tensor repo
Explore the Tensor2Tensor repo.

Follow-up works:

Depthwise Separable Convolutions for Neural Machine Translation
One Model To Learn Them All
Discrete Autoencoders for Sequence Models
Generating Wikipedia by Summarizing Long Sequences
Image Transformer
Training Tips for the Transformer Model
Self-Attention with Relative Position Representations
Fast Decoding in Sequence Models using Discrete Latent Variables
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

Acknowledgements

Thanks to Illia Polosukhin, Jakob Uszkoreit, Llion Jones , Lukasz Kaiser, Niki Parmar, and Noam Shazeer for providing feedback on earlier versions of this post.

Please hit me up on Twitter for any corrections or feedback.

转载自：http://jalammar.github.io/illustrated-transformer/

Transformer architecture的解释相关推荐

UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation
UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation UTNet:一种用于医学图像分割的混合transform ...
【论文笔记】TransFG: A Transformer Architecture for Fine-Grained Recognition
TransFG 简介与基于CNN的模型在细粒度任务上的对比 disadvantages of CNN advantage of Transformer 整体结构改进点 1.overlapping ...
论文解读TransFG: A Transformer Architecture for Fine-grained Recognition
此篇文章是transformer在细粒度领域的应用. 问题:Transformer还未应用在图像细分类领域中贡献点:1.vision transformer的输入把图像切分成patch,但是是没有o ...
论文笔记33 -- （细粒度识别）【CVPR2021】TransFG: A Transformer Architecture for Fine-grained Recognition
Transformer在细粒度领域的应用论文:点这里代码:点这里 Ju He,Jie-Neng Chen,Shuai Liu,Adam Kortylewski,Cheng Yang,Yutong ...
追溯XLNet的前世今生：从Transformer到XLNet
作者丨李格映来源 | 转载自CSDN博客导读:2019 年 6 月,CMU 与谷歌大脑提出全新 XLNet,基于 BERT 的优缺点,XLNet 提出一种泛化自回归预训练方法,在 20 个任务上超 ...
《Deep Learning With Python second edition》英文版读书笔记：第十一章DL for text: NLP、Transformer、Seq2Seq
文章目录第十一章:Deep learning for text 11.1 Natural language processing: The bird's eye view 11.2 Preparin ...
DL：深度学习算法(神经网络模型集合)概览之《THE NEURAL NETWORK ZOO》的中文解释和感悟(六)
DL:深度学习算法(神经网络模型集合)概览之<THE NEURAL NETWORK ZOO>的中文解释和感悟(六) 目录 DRN DNC NTM CN KN AN 相关文章 DL:深度学习 ...
自然语言处理模型：bert 结构原理解析——attention+transformer（翻译自：Deconstructing BERT）
原文:Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters 关于transformer 和attention的机 ...
LeViT: aVision Transformer in ConvNet‘s Clothing for Fast in
摘要我们设计了一系列图像分类架构,可以在高速模式下优化精度和效率之间的平衡.我们的工作利用了基于注意力的体系结构的最新发现,这种体系结构在高度并行处理硬件上具有竞争力.我们重温了大量文献中关于卷积神 ...

Transformer architecture的解释

Go Forth And Transform

Acknowledgements

Transformer architecture的解释相关推荐

最新文章

热门文章