nuget
Trainers take the generated output of activations and gradients in order to modify the weights in the network to make a better prediction the next time the network runs with a data block.
@author Daniel Persson (mailto.woden@gmail.com)
TrainerAlgorithm
01 Syntax
Microsoft.VisualBasic.MachineLearning.CNN.trainers.TrainerAlgorithm
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| .ctor | 1 | Creates a trainer algorithm with default hyper parameters. |
| SetKernel | 1 | Attaches the network to be trained. |
| train | 1 | Runs one training step: a forward pass, a backward pass and, every TrainerAlgorithm.batch_size samples, a weight update. |
| update | 1 | Applies the concrete update rule to one parameter of one parameter block. |
| initTrainData | 1 | Allows an update rule to allocate its additional per parameter accumulator for a parameter block. |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| learning_rate | 1 | Learning rate (alpha) applied to the weight update. |
| eps | 1 | Small constant added for numerical conditioning, avoiding division by zero. |
| momentum | 1 | Momentum factor used by the momentum based update rules. |
| batch_size | 1 | Gets the mini batch size; the weights are updated every TrainerAlgorithm.batch_size samples. |
| conv_net | 1 | Gets the convolutional network this trainer updates. |
| get_output | 1 | Gets the output activations produced by the most recent forward pass. |
04 Fields
| Name | Overloads | Summary |
|---|---|---|
| l1_decay | 1 | L1 and L2 regularization strengths applied during the weight update. |
| l2_decay | 1 | L1 and L2 regularization strengths applied during the weight update. |
| k | 1 | Iteration counter, incremented on every training step. |
| gsum | 1 | Per parameter accumulators used by the update rules (first and second moment estimates). |
| xsum | 1 | Per parameter accumulators used by the update rules (first and second moment estimates). |
05 Members
#ctor(
Int32, Single)Creates a trainer algorithm with default hyper parameters.
Parameters
| Name | Type | Description |
|---|---|---|
batch_size | Int32 | Number of samples accumulated before the weights are updated. |
l2_decay | Single | L2 regularization strength; the L1 decay is fixed at 0.001. |
SetKernel(ConvolutionalNN)
Attaches the network to be trained.
Parameters
| Name | Type | Description |
|---|---|---|
cnn | ConvolutionalNN | The network whose weights are updated by this trainer. |
Returns
This trainer, so the call can be chained.
Runs one training step: a forward pass, a backward pass and, every TrainerAlgorithm.batch_size samples, a weight update.
Parameters
| Name | Type | Description |
|---|---|---|
x | DataBlock | The input data block. |
y | Double() | The target output; a single element is treated as a class index, several elements as a regression target. |
checkpoints | PerformanceCounter | Optional performance counter used to time the individual steps. |
Returns
The loss and timing information of this training step.
update(
Int32, Int32, Double, Double())Applies the concrete update rule to one parameter of one parameter block.
Parameters
| Name | Type | Description |
|---|---|---|
i | Int32 | Index of the parameter block inside the network. |
j | Int32 | Index of the parameter inside the block. |
gij | Double | The raw batch gradient of that parameter. |
p | Double() | The parameter vector that is updated in place. |
initTrainData(BackPropResult)
Allows an update rule to allocate its additional per parameter accumulator for a parameter block.
Parameters
| Name | Type | Description |
|---|---|---|
bpr | BackPropResult | The parameter block that is about to be trained for the first time. |
learning_rate
Learning rate (alpha) applied to the weight update.
eps
Small constant added for numerical conditioning, avoiding division by zero.
momentum
Momentum factor used by the momentum based update rules.
batch_size
Gets the mini batch size; the weights are updated every TrainerAlgorithm.batch_size samples.
conv_net
Gets the convolutional network this trainer updates.
get_output
Gets the output activations produced by the most recent forward pass.
l1_decay
L1 and L2 regularization strengths applied during the weight update.
l2_decay
L1 and L2 regularization strengths applied during the weight update.
k
Iteration counter, incremented on every training step.
gsum
Per parameter accumulators used by the update rules (first and second moment estimates).
xsum
Per parameter accumulators used by the update rules (first and second moment estimates).