Cache

Full name Microsoft.VisualBasic.MachineLearning.Transformer.EncoderLayer.Cache Assembly Microsoft.VisualBasic.DeepLearning Members 26

Forward intermediates of one encode pass, required by the backward pass.

01 Syntax

Microsoft.VisualBasic.MachineLearning.Transformer.EncoderLayer.Cache

02 Fields

Name	Overloads	Summary
Input	2	The input of this layer.
AttentionOutput	2	Output of the self attention sub layer.
AttentionDropped	2	Self attention output after dropout.
Normalized1	2	Result of the first AddNorm.
Norm1Mean	2	Mean used by the first layer normalization.
Norm1InvStd	2	Inverse standard deviation used by the first layer normalization.
FeedForwardOutput	2	Output of the feed forward sub layer.
FeedForwardDropped	2	Feed forward output after dropout.
Norm2Mean	2	Mean used by the second layer normalization.
Norm2InvStd	2	Inverse standard deviation used by the second layer normalization.
DropoutApplied	2	Indicates whether dropout was applied during this pass.
MhaCache	2	Forward cache snapshot of the self attention sub layer.
FfCache	2	Forward cache snapshot of the feed forward sub layer.

03 Members

field Input #

Input

The input of this layer.

field AttentionOutput #

AttentionOutput

Output of the self attention sub layer.

field AttentionDropped #

AttentionDropped

Self attention output after dropout.

field Normalized1 #

Normalized1

Result of the first AddNorm.

field Norm1Mean #

Norm1Mean

Mean used by the first layer normalization.

field Norm1InvStd #

Norm1InvStd

Inverse standard deviation used by the first layer normalization.

field FeedForwardOutput #

FeedForwardOutput

Output of the feed forward sub layer.

field FeedForwardDropped #

FeedForwardDropped

Feed forward output after dropout.

field Norm2Mean #

Norm2Mean

Mean used by the second layer normalization.

field Norm2InvStd #

Norm2InvStd

Inverse standard deviation used by the second layer normalization.

field DropoutApplied #

DropoutApplied

Indicates whether dropout was applied during this pass.

field MhaCache #

MhaCache

Forward cache snapshot of the self attention sub layer.

field FfCache #

FfCache

Forward cache snapshot of the feed forward sub layer.

field Input overload 2 #

Input

field AttentionOutput overload 2 #

AttentionOutput

field AttentionDropped overload 2 #

AttentionDropped

field Normalized1 overload 2 #

Normalized1

field Norm1Mean overload 2 #

Norm1Mean

field Norm1InvStd overload 2 #

Norm1InvStd

field FeedForwardOutput overload 2 #

FeedForwardOutput

field FeedForwardDropped overload 2 #

FeedForwardDropped

field Norm2Mean overload 2 #

Norm2Mean

field Norm2InvStd overload 2 #

Norm2InvStd

field DropoutApplied overload 2 #

DropoutApplied

field MhaCache overload 2 #

MhaCache

field FfCache overload 2 #

FfCache