nuget
Use a learned embedding layer to reduce the size of the word embedding space.
Embedding
01 Syntax
Microsoft.VisualBasic.MachineLearning.Transformer.Embedding
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| .ctor | 1 | Constructor |
| Embed | 1 | Multiply the one-hot embeddings with the embedding layer to project onto a smaller space |
| CalculateLossFunction | 1 | Cross entropy loss function between the correct word in a sentence and the decoder output word. |
| GetWords | 1 | Get a word based on its index in the dictionary |
| GetWordIndex | 1 | Get the index of a specific word in a dictionary |
| OneHotEmbedding | 1 | Encode all words in a dictionary with one-hot embedding |
| AddPositionalEncoding | 1 | Add positional encoding to embedded words according to "Attention is all you need" |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| DictionarySize | 1 | |
| EmbeddingSize | 1 | |
| SequenceLength | 1 |
04 Fields
| Name | Overloads | Summary |
|---|---|---|
| embeddingLayer | 1 | Learned linear embedding layer |
05 Members
#ctor(
Int32, Int32, List(Of List(Of String)))Constructor
Parameters
| Name | Type | Description |
|---|---|---|
embeddingSize | Int32 | - |
sequenceLength | Int32 | - |
sentences | List(Of List(Of String)) | - |
Embed(
List(Of List(Of String)), Boolean)Multiply the one-hot embeddings with the embedding layer to project onto a smaller space
Parameters
| Name | Type | Description |
|---|---|---|
sentences | List(Of List(Of String)) | - |
isTraining | Boolean | - |
CalculateLossFunction(
Tensor, List(Of List(Of String)), Int32, Rev)Cross entropy loss function between the correct word in a sentence and the decoder output word. For a batch of several sentences the loss is accumulated.
Parameters
| Name | Type | Description |
|---|---|---|
filteredOutout | Tensor | - |
correctSpanishSentences | List(Of List(Of String)) | - |
w | Int32 | - |
loss | Rev | - |
GetWords(
Int32())Get a word based on its index in the dictionary
Parameters
| Name | Type | Description |
|---|---|---|
indexes | Int32() | - |
GetWordIndex(
String)Get the index of a specific word in a dictionary
Parameters
| Name | Type | Description |
|---|---|---|
word | String | - |
OneHotEmbedding(
List(Of List(Of String)))Encode all words in a dictionary with one-hot embedding
Parameters
| Name | Type | Description |
|---|---|---|
sentences | List(Of List(Of String)) | - |
AddPositionalEncoding(
Tensor, Int32, Int32)Add positional encoding to embedded words according to "Attention is all you need"
Parameters
| Name | Type | Description |
|---|---|---|
wordEmbeddings | Tensor | - |
s | Int32 | - |
sentenceLength | Int32 | - |
embeddingLayer
Learned linear embedding layer
DictionarySize
EmbeddingSize
SequenceLength