Cache

Full name Microsoft.VisualBasic.DeepLearning.LLM.SwiGLUFeedForward.Cache Assembly Microsoft.VisualBasic.DeepLearning.LLM Members 8

前向传播的中间量缓存，供反向传播使用。

01 Syntax

Microsoft.VisualBasic.DeepLearning.LLM.SwiGLUFeedForward.Cache

02 Fields

03 Members

field Input #

Input

本层输入 [B, S, dModel]

field Gate #

Gate

门控支路的预激活值 x·Wg

field Up #

内容支路 x·Wu

field Activation #

Activation

门控后的激活值 SiLU(gate) ⊙ up

field Input overload 2 #

Input

field Gate overload 2 #

Gate

field Up overload 2 #

field Activation overload 2 #

Activation