TransformerModel

Full name Microsoft.VisualBasic.MachineLearning.Transformer.TransformerModel Assembly Microsoft.VisualBasic.DeepLearning Members 7

Transformer architecture as described in "Attention is all you need"

00 Remarks

01 Syntax

Microsoft.VisualBasic.MachineLearning.Transformer.TransformerModel

02 Methods

Name	Overloads	Summary
.ctor	1	Creates a transformer translation model and prepares its embeddings from the provided training sentences.
Train	1	Trains the model with explicit backpropagation through time over the decoder steps.
Infer	2
Translate	1	Translate a batch of sentenses by generating one word at a time for each sentence with the decoder until max length or stopping character.
Backward	1	显式 BPTT：逆序回传各解码步，累加对 encoder 输出的梯度，再统一回传编码器与两侧词嵌入。
CloneSentences	1	深拷贝一组句子（BPTT 需要保存每一步解码输入的快照）。

03 Members

method .ctor #

#ctor(Int32, Int32, Int32, Int32, Int32, Int32, Int32, Double, List(Of List(Of String)), List(Of List(Of String)))

Creates a transformer translation model and prepares its embeddings from the provided training sentences.

Parameters

Name	Type	Description
`Nx`	`Int32`	Number of encoder and decoder layers.
`embeddingSize`	`Int32`	Width of the model.
`dk`	`Int32`	Dimension of the query and key projections per head.
`dv`	`Int32`	Dimension of the value projection per head.
`h`	`Int32`	Number of attention heads.
`dff`	`Int32`	Hidden width of the feed forward networks.
`batchSize`	`Int32`	Number of sentences per training batch.
`dropout`	`Double`	Dropout rate used during training.
`allEnglishSentences`	`List(Of List(Of String))`	The source language sentences used to build the source vocabulary.
`allSpanishSentences`	`List(Of List(Of String))`	The target language sentences used to build the target vocabulary.

method Train #

Train(Int32, Int32, Double, Int32, List(Of List(Of String)), List(Of List(Of String)))

Trains the model with explicit backpropagation through time over the decoder steps.

Parameters

Name	Type	Description
`nrEpochs`	`Int32`	Number of passes over the data set.
`nrTrainingSteps`	`Int32`	Number of decoder steps per batch.
`learningRate`	`Double`	The learning rate.
`batchSize`	`Int32`	Number of sentence pairs per batch.
`allEnglishSentences`	`List(Of List(Of String))`	The source language sentences.
`allSpanishSentences`	`List(Of List(Of String))`	The target language sentences.

method Infer overload 2 #

Infer(IEnumerable(Of String))

Translates a single sentence token by token.

Parameters

Name	Type	Description
`words`	`IEnumerable(Of String)`	The source sentence tokens.

Returns

The translated sentences, or Nothing when the source sentence contains a word that is not in the dictionary.

method Translate #

Translate(Int32, Boolean, List(Of List(Of String)), List(Of List(Of String)), List(Of List(Of String)))

Translate a batch of sentenses by generating one word at a time for each sentence with the decoder until max length or stopping character.

Returns

该批次的交叉熵损失（训练模式下按 sequenceLength * batchSize 归一化）

method Backward #

Backward

显式 BPTT：逆序回传各解码步，累加对 encoder 输出的梯度，再统一回传编码器与两侧词嵌入。

method CloneSentences #

CloneSentences(List(Of List(Of String)))

深拷贝一组句子（BPTT 需要保存每一步解码输入的快照）。

method Infer #

Infer()