Cache

Full name Microsoft.VisualBasic.MachineLearning.Transformer.MultiHeadAttention.Cache Assembly Microsoft.VisualBasic.DeepLearning Members 18

Forward intermediates of the attention computation, required by the backward pass.

01 Syntax

Microsoft.VisualBasic.MachineLearning.Transformer.MultiHeadAttention.Cache

02 Fields

Name	Overloads	Summary
Qf	2	The projected query tensors, one per head.
Kf	2	The projected key tensors, one per head.
Vf	2	The projected value tensors, one per head.
Probs	2	The attention probabilities (softmax output) of every head.
Concat	2	The heads after concatenation along the last dimension.
QueriesInput	2	The query side input (identical to the K/V input for self attention).
KInput	2	The key side input.
VInput	2	The value side input.
CrossAttention	2	Whether this is cross attention, i.e.

03 Members

field Qf #

The projected query tensors, one per head.

field Kf #

The projected key tensors, one per head.

field Vf #

The projected value tensors, one per head.

field Probs #

Probs

The attention probabilities (softmax output) of every head.

field Concat #

Concat

The heads after concatenation along the last dimension.

field QueriesInput #

QueriesInput

The query side input (identical to the K/V input for self attention).

field KInput #

KInput

The key side input.

field VInput #

VInput

The value side input.

field CrossAttention #

CrossAttention

Whether this is cross attention, i.e. queries and keys/values come from different inputs.

field Qf overload 2 #

field Kf overload 2 #

field Vf overload 2 #

field Probs overload 2 #

Probs

field Concat overload 2 #

Concat

field QueriesInput overload 2 #

QueriesInput

field KInput overload 2 #

KInput

field VInput overload 2 #

VInput

field CrossAttention overload 2 #

CrossAttention