Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exclude multiple taxonomy from a database #924

Open
Xillanne opened this issue Jan 3, 2025 · 1 comment
Open

Exclude multiple taxonomy from a database #924

Xillanne opened this issue Jan 3, 2025 · 1 comment

Comments

@Xillanne
Copy link

Xillanne commented Jan 3, 2025

Expected Behavior

I'm trying to exclude multiple taxid from my UniRef100 database download from mmseqs databases.

Current Behavior

When I search against this database, I still find some of taxid I removed, but indeed they are less than if I run search without excluding any taxid.

Steps to Reproduce (for bugs)

mmseqs databases UniRef100 UniRef100DB tmp
I've tried different ways to exclude my taxid.
mmseqs filtertaxseqdb UniRef100DB UniRef100RuBiSCO3_1 --taxon-list '!33154,!35493,!201174,!1239,!976,!1930617,!640293,!136419,!200795,!3041,!1117,!1297,!33682,!1134404,!2818505,!2806169,!1224,!1154676'
mmseqs filtertaxseqdb UniRef100DB UniRef100RuBiSCO3_2 --taxon-list '!33154||!35493||!201174||!1239||!976||!1930617||!640293||!136419||!200795||!3041||!1117||!1297||!33682||!1134404||!2818505||!2806169||!1224||!1154676'
mmseqs filtertaxseqdb UniRef100DB UniRef100RuBiSCO3_3 --taxon-list '!33154&&!35493&&!201174&&!1239&&!976&&!1930617&&!640293&&!136419&&!200795&&!3041&&!1117&&!1297&&!33682&&!1134404&&!2818505&&!2806169&&!1224&&!1154676'
mmseqs search "$QUERY_DB" "$TARGET_DB" "$RESULT_DB" "$TMP_DIR" --cov-mode 0 -c 0.8 --max-seqs 5000

MMseqs Output (for bugs)

search /data/desmarais/CalvinReboot/RuBiSCO/Results/Iteration/Profils/profile_Deinococcota3 /data/desmarais/CalvinReboot/RuBiSCO/UniRef100RuBiSCO3_2 Deinococcota_Result3 tmp --cov-mode 0 -c 0.8 --max-seqs 5000

MMseqs Version: 15-6f452
Substitution matrix aa:blosum62.out,nucl:nucleotide.out
Add backtrace false
Alignment mode 2
Alignment mode 0
Allow wrapped scoring false
E-value threshold 0.001
Seq. id. threshold 0
Min alignment length 0
Seq. id. mode 0
Alternative alignments 0
Coverage threshold 0.8
Coverage mode 0
Max sequence length 65535
Compositional bias 1
Compositional bias 1
Max reject 2147483647
Max accept 2147483647
Include identical seq. id. false
Preload mode 0
Pseudo count a substitution:1.100,context:1.400
Pseudo count b substitution:4.100,context:5.800
Score bias 0
Realign hits false
Realign score bias -0.2
Realign max seqs 2147483647
Correlation score weight 0
Gap open cost aa:11,nucl:5
Gap extension cost aa:1,nucl:2
Zdrop 40
Threads 20
Compressed 0
Verbosity 3
Seed substitution matrix aa:VTML80.out,nucl:nucleotide.out
Sensitivity 5.7
k-mer length 0
Target search mode 0
k-score seq:2147483647,prof:2147483647
Alphabet size aa:21,nucl:5
Max results per query 5000
Split database 0
Split mode 2
Split memory limit 0
Diagonal scoring true
Exact k-mer matching 0
Mask residues 1
Mask residues probability 0.9
Mask lower case residues 0
Minimum diagonal score 15
Selected taxa
Spaced k-mers 1
Spaced k-mer pattern
Local temporary path
Rescore mode 0
Remove hits by seq. id. and coverage false
Sort results 0
Mask profile 1
Profile E-value threshold 0.1
Global sequence weighting false
Allow deletions false
Filter MSA 1
Use filter only at N seqs 0
Maximum seq. id. threshold 0.9
Minimum seq. id. 0.0
Minimum score per column -20
Minimum coverage 0
Select N most diverse seqs 1000
Pseudo count mode 0
Min codons in orf 30
Max codons in length 32734
Max orf gaps 2147483647
Contig start mode 2
Contig end mode 2
Orf start mode 1
Forward frames 1,2,3
Reverse frames 1,2,3
Translation table 1
Translate orf 0
Use all table starts false
Offset of numeric ids 0
Create lookup 0
Add orf stop false
Overlap between sequences 0
Sequence split mode 1
Header split mode 0
Chain overlapping alignments 0
Merge query 1
Search type 0
Search iterations 1
Start sensitivity 4
Search steps 1
Prefilter mode 0
Exhaustive search mode false
Filter results during exhaustive search 0
Strand selection 1
LCA search mode false
Disk space limit 0
MPI runner
Force restart with latest tmp false
Remove temporary files false

prefilter /data/desmarais/CalvinReboot/RuBiSCO/Results/Iteration/Profils/profile_Deinococcota3 /data/desmarais/CalvinReboot/RuBiSCO/UniRef100RuBiSCO3_2 tmp/4683709013258240388/pref_0 --sub-mat 'aa:blosum62.out,nucl:nucleotide.out' --seed-sub-mat 'aa:VTML80.out,nucl:nucleotide.out' -k 0 --target-search-mode 0 --k-score seq:2147483647,prof:2147483647 --alph-size aa:21,nucl:5 --max-seq-len 65535 --max-seqs 5000 --split 0 --split-mode 2 --split-memory-limit 0 -c 0.8 --cov-mode 0 --comp-bias-corr 1 --comp-bias-corr-scale 1 --diag-score 1 --exact-kmer-matching 0 --mask 1 --mask-prob 0.9 --mask-lower-case 0 --min-ungapped-score 15 --add-self-matches 0 --spaced-kmer-mode 1 --db-load-mode 0 --pca substitution:1.100,context:1.400 --pcb substitution:4.100,context:5.800 --threads 20 --compressed 0 -v 3 -s 5.7

Query database size: 1 type: Profile
Target split mode. Searching through 24 splits
Estimated memory consumption: 89G
Target database size: 420305229 type: Aminoacid
Process prefiltering step 1 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 24s 430ms
Index table: Masked residues: 163802136
Index table: fill
[=================================================================] 17.76M 2m 0s 978ms
Index statistics
Entries: 6423926956
DB size: 46523 MB
Avg k-mer size: 5.018693
Top 10 k-mers
SGQQRIA 55681
GPGGKLL 47667
YTGTGKG 32643
GGQRVAR 27189
FSHAGSI 20363
GRFVVEV 20209
AFRNNFW 19497
ALGSGKS 17622
RAEGRAV 17093
IFLLASS 16950
Time for index table init: 0h 3m 42s 389ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 1 of 24)
Query db start 1 to 1
Target db start 1 to 17762878
[=================================================================] 1 0s 3ms

4457.577963 k-mers per position
7393247 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_0: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_0_tmp: 0h 0m 0s 0ms
Process prefiltering step 2 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.45M 1m 23s 125ms
Index table: Masked residues: 162373885
Index table: fill
[=================================================================] 17.45M 1m 50s 708ms
Index statistics
Entries: 6322932372
DB size: 45945 MB
Avg k-mer size: 4.939791
Top 10 k-mers
SGQQRIA 55329
GPGGKLL 47142
YTGTGKG 31749
GGQRVAR 26837
GKTLRAG 20710
AFRNNFW 20672
GRFVVEV 20055
RYYVLGW 19009
APMFPNN 18556
TVDGDFS 18485
Time for index table init: 0h 3m 30s 968ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 2 of 24)
Query db start 1 to 1
Target db start 17762879 to 35213031
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7214446 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_1: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_1_tmp: 0h 0m 0s 0ms
Process prefiltering step 3 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.32M 1m 20s 333ms
Index table: Masked residues: 157584350
Index table: fill
[=================================================================] 17.32M 1m 48s 121ms
Index statistics
Entries: 6243318896
DB size: 45490 MB
Avg k-mer size: 4.877593
Top 10 k-mers
SGQQRIA 55146
GPGGKLL 47220
YTGTGKG 31964
GGQRVAR 27023
GRFVVEV 20229
AFRNNFW 19395
LLGPGKT 18338
ALGSGKS 17633
RAEGRAV 17047
IFLLASS 16666
Time for index table init: 0h 3m 24s 731ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 3 of 24)
Query db start 1 to 1
Target db start 35213032 to 52532314
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7168984 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_2: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_2_tmp: 0h 0m 0s 0ms
Process prefiltering step 4 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 22s 880ms
Index table: Masked residues: 164759024
Index table: fill
[=================================================================] 17.76M 1m 51s 813ms
Index statistics
Entries: 6425265922
DB size: 46531 MB
Avg k-mer size: 5.019739
Top 10 k-mers
SGQQRIA 56165
GPGGKLL 47900
YTGTGKG 32436
GGQRVAR 27343
GKTLRAG 21161
GRFVVEV 20578
LLGPGKT 18829
AFRNNFW 18732
ALGSGKS 17718
LSPLAIT 17549
Time for index table init: 0h 3m 31s 119ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 4 of 24)
Query db start 1 to 1
Target db start 52532315 to 70290296
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7396444 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_3: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_3_tmp: 0h 0m 0s 0ms
Process prefiltering step 5 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.45M 1m 21s 306ms
Index table: Masked residues: 161643642
Index table: fill
[=================================================================] 17.45M 1m 50s 377ms
Index statistics
Entries: 6306722701
DB size: 45852 MB
Avg k-mer size: 4.927127
Top 10 k-mers
SGQQRIA 54670
GPGGKLL 46954
YTGTGKG 31639
GGQRVAR 26960
AFRNNFW 21124
GRFVVEV 20141
LLGPGKT 18652
IFLLASS 18424
TMLDRNT 18197
RGAVAVR 17740
Time for index table init: 0h 3m 28s 109ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 5 of 24)
Query db start 1 to 1
Target db start 70290297 to 87743379
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7234873 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_4: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_4_tmp: 0h 0m 0s 0ms
Process prefiltering step 6 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.33M 1m 20s 515ms
Index table: Masked residues: 159129868
Index table: fill
[=================================================================] 17.33M 1m 49s 507ms
Index statistics
Entries: 6258430902
DB size: 45576 MB
Avg k-mer size: 4.889399
Top 10 k-mers
SGQQRIA 55380
GPGGKLL 47151
YTGTGKG 31926
GGQRVAR 27167
GRFVVEV 20231
AFRNNFW 18176
RGAVAVR 17770
ALGSGKS 17551
RAEGRAV 17009
IFLLASS 15740
Time for index table init: 0h 3m 26s 156ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 6 of 24)
Query db start 1 to 1
Target db start 87743380 to 105068912
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7203382 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_5: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_5_tmp: 0h 0m 0s 0ms
Process prefiltering step 7 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 22s 484ms
Index table: Masked residues: 163792410
Index table: fill
[=================================================================] 17.76M 1m 51s 720ms
Index statistics
Entries: 6415335463
DB size: 46474 MB
Avg k-mer size: 5.011981
Top 10 k-mers
SGQQRIA 56484
GPGGKLL 48486
YTGTGKG 32061
GGQRVAR 27573
GRFVVEV 20469
LLGPGKT 18937
AFRNNFW 18387
ALGNGKS 17289
RAEGRAV 17226
NNSWLPS 15994
Time for index table init: 0h 3m 30s 817ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 7 of 24)
Query db start 1 to 1
Target db start 105068913 to 122831425
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7411234 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_6: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_6_tmp: 0h 0m 0s 0ms
Process prefiltering step 8 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.46M 1m 21s 305ms
Index table: Masked residues: 160434897
Index table: fill
[=================================================================] 17.46M 1m 49s 673ms
Index statistics
Entries: 6308744841
DB size: 45864 MB
Avg k-mer size: 4.928707
Top 10 k-mers
SGQQRIA 55742
GPGGKLL 47297
YTGTGKG 32307
GGQRVAR 27260
GRFVVEV 20306
AFRNNFW 18190
ALGSGKS 17596
RAEGRAV 17269
NNSWLPS 15919
IFLLASS 15714
Time for index table init: 0h 3m 27s 139ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 8 of 24)
Query db start 1 to 1
Target db start 122831426 to 140286532
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7265654 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_7: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_7_tmp: 0h 0m 0s 0ms
Process prefiltering step 9 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.32M 1m 20s 774ms
Index table: Masked residues: 161457062
Index table: fill
[=================================================================] 17.32M 1m 50s 127ms
Index statistics
Entries: 6270220282
DB size: 45644 MB
Avg k-mer size: 4.898610
Top 10 k-mers
SGQQRIA 54823
GPGGKLL 46665
YTGTGKG 31899
GGQRVAR 26643
FSHAGSI 20656
GRFVVEV 19876
AFRNNFW 19791
LLGPGKT 18145
IFLLASS 17206
TMLDRNT 17099
Time for index table init: 0h 3m 26s 975ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 9 of 24)
Query db start 1 to 1
Target db start 140286533 to 157605962
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7207823 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_8: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_8_tmp: 0h 0m 0s 0ms
Process prefiltering step 10 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 22s 814ms
Index table: Masked residues: 162414118
Index table: fill
[=================================================================] 17.76M 1m 51s 287ms
Index statistics
Entries: 6413637722
DB size: 46464 MB
Avg k-mer size: 5.010654
Top 10 k-mers
SGQQRIA 56777
GPGGKLL 48206
YTGTGKG 32422
GGQRVAR 27681
GRFVVEV 20339
AFRNNFW 19206
LLGPGKT 18510
ALGSGKS 17927
RAEGRAV 17104
RYYVLGW 16813
Time for index table init: 0h 3m 30s 664ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 10 of 24)
Query db start 1 to 1
Target db start 157605963 to 175366608
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7350018 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_9: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_9_tmp: 0h 0m 0s 0ms
Process prefiltering step 11 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.46M 1m 21s 674ms
Index table: Masked residues: 161856379
Index table: fill
[=================================================================] 17.46M 1m 50s 736ms
Index statistics
Entries: 6325278504
DB size: 45959 MB
Avg k-mer size: 4.941624
Top 10 k-mers
SGQQRIA 55232
GPGGKLL 47159
YTGTGKG 31791
GGQRVAR 27039
GRFVVEV 20325
LLGPGKT 18362
AFRNNFW 17947
RAEGRAV 17064
LSPLAIT 16824
NNSWLPS 15527
Time for index table init: 0h 3m 28s 699ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 11 of 24)
Query db start 1 to 1
Target db start 175366609 to 192827062
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7283395 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_10: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_10_tmp: 0h 0m 0s 0ms
Process prefiltering step 12 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.32M 1m 21s 419ms
Index table: Masked residues: 162234745
Index table: fill
[=================================================================] 17.32M 1m 49s 543ms
Index statistics
Entries: 6297830287
DB size: 45802 MB
Avg k-mer size: 4.920180
Top 10 k-mers
SGQQRIA 54375
GPGGKLL 46338
YTGTGKG 31738
GGQRVAR 26243
GKTLRAG 20513
GRFVVEV 19714
AFRNNFW 18343
RGAVAVR 17391
ALGSGKS 17303
LSPLAIT 17066
Time for index table init: 0h 3m 27s 94ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 12 of 24)
Query db start 1 to 1
Target db start 192827063 to 210144961
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7236326 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_11: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_11_tmp: 0h 0m 0s 0ms
Process prefiltering step 13 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 23s 158ms
Index table: Masked residues: 164577386
Index table: fill
[=================================================================] 17.76M 1m 52s 560ms
Index statistics
Entries: 6431146705
DB size: 46564 MB
Avg k-mer size: 5.024333
Top 10 k-mers
SGQQRIA 56216
GPGGKLL 48144
YTGTGKG 32789
GGQRVAR 27574
AFRNNFW 21031
GRFVVEV 20610
LLGPGKT 18703
IFLLASS 18223
TMLDRNT 17897
GGRRVAR 17673
Time for index table init: 0h 3m 32s 857ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 13 of 24)
Query db start 1 to 1
Target db start 210144962 to 227906846
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7376220 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_12: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_12_tmp: 0h 0m 0s 0ms
Process prefiltering step 14 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.46M 1m 21s 684ms
Index table: Masked residues: 161044591
Index table: fill
[=================================================================] 17.46M 1m 47s 350ms
Index statistics
Entries: 6311297219
DB size: 45879 MB
Avg k-mer size: 4.930701
Top 10 k-mers
SGQQRIA 55435
GPGGKLL 47374
YTGTGKG 32016
GGQRVAR 27076
FSHAGSI 20586
GRFVVEV 19965
AFRNNFW 19677
IFLLASS 17170
TMLDRNT 16780
RAEGRAV 16755
Time for index table init: 0h 3m 25s 237ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 14 of 24)
Query db start 1 to 1
Target db start 227906847 to 245363972
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7241250 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_13: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_13_tmp: 0h 0m 0s 0ms
Process prefiltering step 15 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.32M 1m 21s 371ms
Index table: Masked residues: 160442737
Index table: fill
[=================================================================] 17.32M 1m 47s 556ms
Index statistics
Entries: 6265821981
DB size: 45618 MB
Avg k-mer size: 4.895173
Top 10 k-mers
SGQQRIA 55245
GPGGKLL 46733
YTGTGKG 31724
GGQRVAR 26854
GKTLRAG 20674
AFRNNFW 20384
LSPLAIT 19154
LLGPGKT 18137
PDAPRNM 17598
TMLDRNT 17555
Time for index table init: 0h 3m 25s 25ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 15 of 24)
Query db start 1 to 1
Target db start 245363973 to 262688727
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7169162 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_14: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_14_tmp: 0h 0m 0s 0ms
Process prefiltering step 16 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 23s 438ms
Index table: Masked residues: 164206015
Index table: fill
[=================================================================] 17.76M 1m 51s 810ms
Index statistics
Entries: 6429667500
DB size: 46556 MB
Avg k-mer size: 5.023178
Top 10 k-mers
SGQQRIA 56603
GPGGKLL 48254
YTGTGKG 32709
GGQRVAR 27528
GKTLRAG 21285
GRFVVEV 20607
AFRNNFW 19185
LLGPGKT 18460
LSPLAIT 18036
RAEGRAV 17359
Time for index table init: 0h 3m 32s 122ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 16 of 24)
Query db start 1 to 1
Target db start 262688728 to 280445623
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7404280 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_15: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_15_tmp: 0h 0m 0s 0ms
Process prefiltering step 17 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.46M 1m 21s 339ms
Index table: Masked residues: 160628099
Index table: fill
[=================================================================] 17.46M 1m 50s 380ms
Index statistics
Entries: 6311457516
DB size: 45880 MB
Avg k-mer size: 4.930826
Top 10 k-mers
SGQQRIA 55655
GPGGKLL 47571
YTGTGKG 32307
GGQRVAR 27097
FSHAGSI 20239
GRFVVEV 20221
AFRNNFW 19393
ALGSGKS 17613
NNSWLPS 16974
RAEGRAV 16971
Time for index table init: 0h 3m 28s 620ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 17 of 24)
Query db start 1 to 1
Target db start 280445624 to 297900745
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7242680 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_16: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_16_tmp: 0h 0m 0s 0ms
Process prefiltering step 18 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.33M 1m 25s 50ms
Index table: Masked residues: 159631311
Index table: fill
[=================================================================] 17.33M 1m 49s 159ms
Index statistics
Entries: 6270266886
DB size: 45644 MB
Avg k-mer size: 4.898646
Top 10 k-mers
SGQQRIA 55196
GPGGKLL 47126
YTGTGKG 31970
GGQRVAR 27002
FSHAGSI 20881
AFRNNFW 20071
LSPLAIT 18717
LLGPGKT 18226
NNSWLPS 17357
IFLLASS 17261
Time for index table init: 0h 3m 31s 130ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 18 of 24)
Query db start 1 to 1
Target db start 297900746 to 315227266
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7168793 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_17: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_17_tmp: 0h 0m 0s 0ms
Process prefiltering step 19 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.76M 1m 22s 782ms
Index table: Masked residues: 162144003
Index table: fill
[=================================================================] 17.76M 1m 51s 953ms
Index statistics
Entries: 6410771417
DB size: 46448 MB
Avg k-mer size: 5.008415
Top 10 k-mers
SGQQRIA 56444
GPGGKLL 47631
YTGTGKG 32504
GGQRVAR 27248
GKTLRAG 21206
FSHAGSI 20959
GRFVVEV 20538
AFRNNFW 20066
LLGPGKT 18566
IFLLASS 17461
Time for index table init: 0h 3m 31s 183ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 19 of 24)
Query db start 1 to 1
Target db start 315227267 to 332991338
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7393198 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_18: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_18_tmp: 0h 0m 0s 0ms
Process prefiltering step 20 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.46M 1m 21s 378ms
Index table: Masked residues: 160462700
Index table: fill
[=================================================================] 17.46M 1m 52s 504ms
Index statistics
Entries: 6311179553
DB size: 45878 MB
Avg k-mer size: 4.930609
Top 10 k-mers
SGQQRIA 56431
GPGGKLL 47588
YTGTGKG 32105
GGQRVAR 27257
GRFVVEV 20392
LLGPGKT 18458
AFRNNFW 18263
ALGSGKS 17644
RAEGRAV 17186
NNSWLPS 15843
Time for index table init: 0h 3m 30s 621ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 20 of 24)
Query db start 1 to 1
Target db start 332991339 to 350449810
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7260253 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_19: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_19_tmp: 0h 0m 0s 0ms
Process prefiltering step 21 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.32M 1m 20s 476ms
Index table: Masked residues: 159462408
Index table: fill
[=================================================================] 17.32M 1m 48s 868ms
Index statistics
Entries: 6258672268
DB size: 45578 MB
Avg k-mer size: 4.889588
Top 10 k-mers
SGQQRIA 54997
GPGGKLL 46860
YTGTGKG 31824
GGQRVAR 26888
AFRNNFW 20425
GRFVVEV 20066
LLGPGKT 18011
IFLLASS 17898
PDAPRNM 17721
TMLDRNT 17714
Time for index table init: 0h 3m 25s 325ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 21 of 24)
Query db start 1 to 1
Target db start 350449811 to 367772092
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7117994 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_20: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_20_tmp: 0h 0m 0s 0ms
Process prefiltering step 22 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.77M 1m 22s 488ms
Index table: Masked residues: 163016022
Index table: fill
[=================================================================] 17.77M 1m 51s 180ms
Index statistics
Entries: 6407429236
DB size: 46429 MB
Avg k-mer size: 5.005804
Top 10 k-mers
SGQQRIA 56604
GPGGKLL 48322
YTGTGKG 32875
GGQRVAR 28083
FSHAGSI 21116
GRFVVEV 20630
AFRNNFW 20209
LLGPGKT 18442
RAEGRAV 17510
IFLLASS 17496
Time for index table init: 0h 3m 29s 995ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 22 of 24)
Query db start 1 to 1
Target db start 367772093 to 385539670
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7377883 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_21: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_21_tmp: 0h 0m 0s 0ms
Process prefiltering step 23 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.45M 1m 22s 821ms
Index table: Masked residues: 158605055
Index table: fill
[=================================================================] 17.45M 1m 50s 982ms
Index statistics
Entries: 6279884259
DB size: 45699 MB
Avg k-mer size: 4.906160
Top 10 k-mers
SGQQRIA 55532
GPGGKLL 47593
YTGTGKG 32084
GGQRVAR 27172
FSHAGSI 20509
GRFVVEV 20237
AFRNNFW 19669
LLGPGKT 18251
ALGNGKS 17133
NNSWLPS 17102
Time for index table init: 0h 3m 30s 379ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 23 of 24)
Query db start 1 to 1
Target db start 385539671 to 402991018
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7210749 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_22: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_22_tmp: 0h 0m 0s 0ms
Process prefiltering step 24 of 24

Index table k-mer threshold: 0 at k-mer size 7
Index table: counting k-mers
[=================================================================] 17.31M 1m 20s 845ms
Index table: Masked residues: 160107551
Index table: fill
[=================================================================] 17.31M 1m 49s 338ms
Index statistics
Entries: 6269790062
DB size: 45641 MB
Avg k-mer size: 4.898273
Top 10 k-mers
SGQQRIA 55349
GPGGKLL 47052
YTGTGKG 31966
GGQRVAR 26595
FSHAGSI 20261
GRFVVEV 20083
AFRNNFW 19342
LLGPGKT 18094
RYYVLGW 18076
ALGSGKS 17534
Time for index table init: 0h 3m 26s 577ms
k-mer similarity threshold: 110
Starting prefiltering scores calculation (step 24 of 24)
Query db start 1 to 1
Target db start 402991019 to 420305229
[=================================================================] 1 0s 1ms

4457.577963 k-mers per position
7181818 DB matches per sequence
0 overflows
265 sequences passed prefiltering per query sequence
265 median result list length
0 sequences with 0 size result lists
Time for merging to pref_0_tmp_23: 0h 0m 0s 0ms
Time for merging to pref_0_tmp_23_tmp: 0h 0m 0s 0ms
Merging 24 target splits to pref_0
Preparing offsets for merging: 0h 0m 0s 88ms
[=================================================================] 1 0s 0ms
Time for merging to pref_0: 0h 0m 0s 1ms
Time for merging target splits: 0h 0m 0s 134ms
Time for merging to pref_0_tmp: 0h 0m 0s 1ms
Time for processing: 1h 25m 28s 946ms
align /data/desmarais/CalvinReboot/RuBiSCO/Results/Iteration/Profils/profile_Deinococcota3 /data/desmarais/CalvinReboot/RuBiSCO/UniRef100RuBiSCO3_2 tmp/4683709013258240388/pref_0 Deinococcota_Result3 --sub-mat 'aa:blosum62.out,nucl:nucleotide.out' -a 0 --alignment-mode 2 --alignment-output-mode 0 --wrapped-scoring 0 -e 0.001 --min-seq-id 0 --min-aln-len 0 --seq-id-mode 0 --alt-ali 0 -c 0.8 --cov-mode 0 --max-seq-len 65535 --comp-bias-corr 1 --comp-bias-corr-scale 1 --max-rejected 2147483647 --max-accept 2147483647 --add-self-matches 0 --db-load-mode 0 --pca substitution:1.100,context:1.400 --pcb substitution:4.100,context:5.800 --score-bias 0 --realign 0 --realign-score-bias -0.2 --realign-max-seqs 2147483647 --corr-score-weight 0 --gap-open aa:11,nucl:5 --gap-extend aa:1,nucl:2 --zdrop 40 --threads 20 --compressed 0 -v 3

Can not touch 162730650589 into main memory
Compute score and coverage
Query database size: 1 type: Profile
Target database size: 420305229 type: Aminoacid
Calculation of alignments
[=================================================================] 1 0s 17ms
Time for merging to Deinococcota_Result3: 0h 0m 0s 0ms
6359 alignments calculated
6357 sequence pairs passed the thresholds (0.999685 of overall calculated)
6357.000000 hits per query sequence
Time for processing: 0h 1m 42s 61ms

Can you please explain my the meaning of each different joinction (, && ||) and help me to exclude all my taxID from my database please ?
@milot-mirdita
Copy link
Member

The && version is the correct one, that also matches whats written in the help text of mmseqs filtertaxseqdb -h.

I tried to reproduce if any tax id is still present anywhere in any entry lineage, but based on swissprot, not the full uniprot:

mmseqs databases UniProtKB/Swiss-Prot sp tmp
mmseqs filtertaxseqdb sp sp_filt --taxon-list '!33154&&!35493&&!201174&&!1239&&!976&&!1930617&&!640293&&!136419&&!200795&&!3041&&!1117&&!1297&&!33682&&!1134404&&!2818505&&!2806169&&!1224&&!1154676'
awk '{ print $1"\t"$1; }' sp_filt.index > sp_filt.fake.tsv
mmseqs tsv2db sp_filt.fake.tsv sp_fake --output-dbtype 5
mmseqs addtaxonomy sp_filt sp_fake res_add --threads 1 --tax-lineage 2
tr -d '\000' < res_add > res_add.tsv

cat << 'EOF' > check.py
import sys
file_path = sys.argv[1]
numbers = set(sys.argv[2].split(','))
with open(file_path, 'r') as f:
    for line in f:
        column = line.split('\t')[3]
        if any(num in column.split(';') for num in numbers):
            print(f"Match found: {line.strip()}")
EOF

python check.py res_add.tsv 33154,35493,201174,1239,976,1930617,640293,136419,200795,3041,1117,1297,33682,1134404,2818505,2806169,1224,1154676

Seems like the filtering is correct with current latest MMseqs2. I did not try to reproduce your issue with the older version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@milot-mirdita @Xillanne