nuget
api documents
block-wise randomized SVD reducer for the streaming pipeline.
the COO sparse vectors are read one block at a time, accumulated into a SparseMatrix (row-dictionary layout, so only the current block is ever in memory) and reduced with TruncatedSVD.Reduce(). the resulting m x k dense embedding is streamed straight to disk row-by-row, so the full embedding never has to be materialized as one big array.
01 Syntax
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| Reduce | 1 | reduce the sparse vectors (provided as blocks of (rowIndex, cols, vals)) down to dims dimensions and stream the dense embeddings to disk. |
| ReadEmbeddings | 1 | stream the dense embeddings back row-by-row (rowIndex, embedding vector) so the KNN stage can consume them in blocks. |
| LoadMeta | 1 |
03 Fields
| Name | Overloads | Summary |
|---|---|---|
| SVD_FILE | 1 | |
| SVD_META_FILE | 1 |
04 Members
IEnumerable(Of ValueTuple(Of Int32, String, Int32(), Double())), Int32, Int32)reduce the sparse vectors (provided as blocks of (rowIndex, cols, vals)) down to dims dimensions and stream the dense embeddings to disk.
String)stream the dense embeddings back row-by-row (rowIndex, embedding vector) so the KNN stage can consume them in blocks.
String)