nuget
The rectified linear unit (ReLU) activation function: the input value which is less than the threshold will be clipped as the threshold value.
ReLU
00 Remarks
ReLU is the default activation function of the modern neural network models, as its derivative is cheap to calculate and it does not suffer from the vanishing gradient problem on the positive half-axis.
01 Syntax
Microsoft.VisualBasic.MachineLearning.ComponentModel.Activations.ReLU
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| .ctor | 2 | Create a standard ReLU function with the default threshold value 0. |
| Function | 1 | Calculates the ReLU function value: the value which is less than the threshold will be clipped as the threshold value. |
| ReLU | 3 | Apply the standard ReLU function on each element of the given vector. |
| CalculateDerivative | 1 | Calculates the derivative of this ReLU function: 1 when the input value is not less than the threshold, otherwise the threshold value. |
| Derivative | 1 | Calculates the derivative of this ReLU function: 1 when the input value is not less than the threshold, otherwise the threshold value. |
| ToString | 1 | Display this activation function as a text expression. |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| Store | 1 | Gets the XML serializable data model of this ReLU function. |
04 Fields
| Name | Overloads | Summary |
|---|---|---|
| threshold | 1 | The clip threshold value, all of the input values which are less than this threshold will be clipped as this threshold value. |
05 Members
#ctor
Create a standard ReLU function with the default threshold value 0.
#ctor(
Double)Create a ReLU function with a specific clipping threshold value.
Parameters
| Name | Type | Description |
|---|---|---|
threshold | Double | The clipping threshold value. |
Function(
Double)Calculates the ReLU function value: the value which is less than the threshold will be clipped as the threshold value.
Parameters
| Name | Type | Description |
|---|---|---|
x | Double | The function input value. |
Returns
The function output value; the output will be truncated by the IActivationFunction.Truncate limitation when the x value is greater than the threshold.
ReLU(
Double)The standard ReLU function: the negative value will be clipped as zero.
Parameters
| Name | Type | Description |
|---|---|---|
x | Double | The function input value. |
Returns
0 when x is a negative value, otherwise the x value itself.
ReLU(
Vector)Apply the standard ReLU function on each element of the given vector.
Parameters
| Name | Type | Description |
|---|---|---|
x | Vector | A |
Returns
The x vector itself, in which all of the negative elements have been clipped as zero (the vector is modified in place).
ReLU(
Double())ReLU activator function will clip the negative value as zero
Parameters
| Name | Type | Description |
|---|---|---|
x | Double() | An array of the function input values, this array will be modified in place. |
Returns
The x array itself, in which all of the negative elements have been clipped as zero.
CalculateDerivative(
Double)Calculates the derivative of this ReLU function: 1 when the input value is not less than the threshold, otherwise the threshold value.
Parameters
| Name | Type | Description |
|---|---|---|
x | Double | The function input value. |
Returns
The derivative value.
Derivative(
Double)Calculates the derivative of this ReLU function: 1 when the input value is not less than the threshold, otherwise the threshold value.
Parameters
| Name | Type | Description |
|---|---|---|
x | Double | The function input value. |
Returns
The derivative value.
ToString
Display this activation function as a text expression.
Returns
A text expression in format like ReLU().
Store
Gets the XML serializable data model of this ReLU function.
Returns
A ActiveFunction data model which its function name is ReLU, and the threshold value is stored as its only argument.
threshold
The clip threshold value, all of the input values which are less than this threshold will be clipped as this threshold value.