10.7. Encoder-Decoder Seq2Seq for Machine Translation — Dive into Deep Learning 1.0.0-beta0 documentation thumbnail
10.7. Encoder-Decoder Seq2Seq for Machine Translation — Dive into Deep Learning 1.0.0-beta0 documentation
d2l.ai
the encoder RNN will take a variable-length sequence as input and transform it into a fixed-shape hidden state Then to generate the output sequence, one token at a time, the decoder model, consisting of a separate RNN, will predict each successive target token given both the input sequence and the p
1 Users
0 Comments
15 Highlights
0 Notes

Top Highlights

  • the encoder RNN will take a variable-length sequence as input and transform it into a fixed-shape hidden state
  • Then to generate the output sequence, one token at a time, the decoder model, consisting of a separate RNN, will predict each successive target token given both the input sequence and the preceding tokens in the output.
  • the special beginning-of-sequence token and the original target sequence, excluding the final token, are concatenated as input to the decoder, while the decoder output (labels for training) is the original target sequence, shifted by one token: “<bos>”, “Ils”, “regardent”, “.” → “Ils”, “regardent”, “.”, “<eos>”
  • In general, the encoder transforms the hidden states at all time steps into a context variable through a customized function � :
  • To predict the subsequent token

Domain

Ready to highlight and find good content?

Glasp is a social web highlighter that people can highlight and organize quotes and thoughts from the web, and access other like-minded people’s learning.