nuget
unsupervised protein family clustering pipeline.
ProteinFamilyClustering
00 Remarks
pipeline (all parameters are configurable, defaults follow the described algorithm):
- extract kmer features (default length 5) from the protein database and select a top-N vocabulary;
- vectorize each protein as a TF-IDF document over the vocabulary;
- compress the TF-IDF matrix with TruncatedSVD (default 9 dims);
- build a KNN similarity network from the embedding;
- split the network into communities with the Louvain algorithm (the protein families);
- for every family run an MSA and pick the sequence with the fewest edits as the reference.
01 Syntax
SMRUCC.genomics.Model.MotifGraph.ProteinStructure.FamilyCluster.ProteinFamilyClustering
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| Run | 1 | run the full clustering pipeline over a FASTA protein database (file or directory) |
| RunStreaming | 1 | run the clustering pipeline in streaming mode, designed for databases that do not fit in memory (e.g. |
| BuildTfidf | 1 | build the TF-IDF dataframe limited to the selected vocabulary, with L2 normalization |
| ToSparseMatrix | 1 | convert the TF-IDF dataframe into a SparseMatrix (COO format) for the randomized SVD |
| BuildEdgeList | 1 | turn the KNN neighbour lists into a symmetric, deduplicated, undirected edge list |
| LouvainPartition | 1 | load the edge list into a NetworkGraph and run Louvain community detection |
| BuildFamilies | 1 | group sequences by family, run MSA per family and pick the lowest-edit reference sequence |
03 Properties
| Name | Overloads | Summary |
|---|---|---|
| k | 1 | kmer length in residues |
| topN | 1 | number of kmer features to keep as the dictionary |
| svdDims | 1 | target dimension after TruncatedSVD compression |
| knnK | 1 | number of nearest neighbours used when building the similarity graph |
| similarityCutoff | 1 | similarity cutoff: KNN edges whose weight is below this value are dropped |
| kmerSortMode | 1 | ranking mode used when selecting the top-N kmer vocabulary |
04 Members
Run(
String)run the full clustering pipeline over a FASTA protein database (file or directory)
Parameters
| Name | Type | Description |
|---|---|---|
fastaHandle | String | path to a FASTA file or a directory of FASTA files |
RunStreaming(
String, String)run the clustering pipeline in streaming mode, designed for databases that do not fit in memory (e.g. a 20GB FASTA with tens of millions of sequences). the FASTA file is scanned in two passes and every intermediate product is written to workDir so that memory stays bounded by the configured block size rather than the database size. a failed run can be resumed (skipping already-produced artifacts) when StreamingClustering.resumeIfExists is left at its default of true.
Parameters
| Name | Type | Description |
|---|---|---|
fastaHandle | String | path to a FASTA file or a directory of FASTA files |
workDir | String | directory that will hold the on-disk intermediate products |
build the TF-IDF dataframe limited to the selected vocabulary, with L2 normalization
ToSparseMatrix(
DataFrame, Int32, Int32)convert the TF-IDF dataframe into a SparseMatrix (COO format) for the randomized SVD
BuildEdgeList(
KNeighbors(), Int32, Double)turn the KNN neighbour lists into a symmetric, deduplicated, undirected edge list
LouvainPartition(
ValueTuple(Of Int32, Int32, Double)(), Int32)load the edge list into a NetworkGraph and run Louvain community detection
BuildFamilies(
Int32(), String(), FastaSeq())group sequences by family, run MSA per family and pick the lowest-edit reference sequence
k
kmer length in residues
topN
number of kmer features to keep as the dictionary
svdDims
target dimension after TruncatedSVD compression
knnK
number of nearest neighbours used when building the similarity graph
similarityCutoff
similarity cutoff: KNN edges whose weight is below this value are dropped
kmerSortMode
ranking mode used when selecting the top-N kmer vocabulary