nuget
Extracts kmer features from a protein sequence database and selects a top-N vocabulary used as the fixed dictionary for the TF-IDF vectorization step.
KmerVocabulary
00 Remarks
For each distinct kmer two statistics are collected during a single pass over the sequence database:
- the per-document occurrence count (how many times the kmer appears inside a single protein sequence);
- the global occurrence count (the total number of times the kmer appears across the whole database).
The kmers are then ranked (by default in ascending order of the chosen metric) and the top-N kmers are kept as the fixed vocabulary.
01 Syntax
SMRUCC.genomics.Model.MotifGraph.ProteinStructure.FamilyCluster.KmerVocabulary
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| BuildVocabularyOnly | 1 | count the kmer occurrences over the whole database in a single streaming pass, keeping only the global total count and the per-document maximum (the two ranking keys). |
| SelectVocabulary | 1 | shared top-N vocabulary selection used by both the in-memory and the streaming build paths |
| Build | 1 | build the kmer vocabulary from a stream of protein sequences |
| Vectorize | 1 | compute the L2-normalized TF-IDF sparse vector of a single protein sequence over the fixed vocabulary, returning the non-zero (columnIndex, value) pairs. |
| Save | 1 | persist the vocabulary (words + global counts + ranking mode) so a streaming run can be resumed without re-counting kmers. |
| Load | 1 | load a vocabulary previously written by KmerVocabulary.Save()) |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| words | 1 | the fixed kmer dictionary ordered as it will be used as the TF-IDF feature columns |
| index | 1 | kmer -> column index inside the KmerVocabulary.words dictionary |
| globalCounts | 1 | global total occurrence count for each selected kmer (aligned with KmerVocabulary.words) |
| docCounts | 1 | per-document occurrence counts: sequence id -> (kmer -> count) for the selected vocabulary only |
| rankingMode | 1 | selection ranking mode |
| k | 1 | kmer length in residues (used when vectorizing a single sequence) |
| size | 1 | number of features in the vocabulary |
04 Members
count the kmer occurrences over the whole database in a single streaming pass, keeping only the global total count and the per-document maximum (the two ranking keys). per-document counts are NOT retained so the memory footprint stays bounded by the vocabulary size rather than the number of sequences - this is what makes the streaming pipeline able to handle databases that do not fit in memory.
SelectVocabulary(
Dictionary(Of String, Int64), Dictionary(Of String, Int32), Dictionary(Of String, Dictionary(Of String, Int32)), Int32, SortMode, Int32)shared top-N vocabulary selection used by both the in-memory and the streaming build paths
build the kmer vocabulary from a stream of protein sequences
Parameters
| Name | Type | Description |
|---|---|---|
sequences | IEnumerable(Of FastaSeq) | the protein sequence database (title, sequence data) |
k | Int32 | kmer length in residues (default 5) |
topN | Int32 | number of kmer features to keep (default 10000) |
mode | SortMode | ranking mode used before slicing the top-N |
Vectorize(
String)compute the L2-normalized TF-IDF sparse vector of a single protein sequence over the fixed vocabulary, returning the non-zero (columnIndex, value) pairs. this is the streaming building block that lets the second pass emit one COO row per sequence without holding the whole database in memory.
Save(
String)persist the vocabulary (words + global counts + ranking mode) so a streaming run can be resumed without re-counting kmers. per-document counts are not stored because the streaming path never keeps them in memory.
Load(
String)load a vocabulary previously written by KmerVocabulary.Save()
words
the fixed kmer dictionary ordered as it will be used as the TF-IDF feature columns
index
kmer -> column index inside the KmerVocabulary.words dictionary
globalCounts
global total occurrence count for each selected kmer (aligned with KmerVocabulary.words)
docCounts
per-document occurrence counts: sequence id -> (kmer -> count) for the selected vocabulary only
rankingMode
selection ranking mode
k
kmer length in residues (used when vectorizing a single sequence)
size
number of features in the vocabulary