Error while running SPAdes in WSL2

[joseailtoncruz]

JOSÉ SANTOS ailton.cruz@ufrpe.br	JOSÉ SANTOS ailton.cruz@ufrpe.br	sex., 14 de jul., 15:05 (há 11 dias)
JOSÉ SANTOS ailton.cruz@ufrpe.br
para spades.support	para spades.support
para spades.support

I am encountering an error while running SPAdes in the Windows Subsystem for Linux 2 (WSL2) environment and would appreciate your assistance in resolving the issue.

Error description:

I am attempting to run SPAdes version 3.13.1 in WSL2 with the following system information:

- SPAdes version: 3.13.1

- Python version: 3.10.6

- OS: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

I am using the following command to run SPAdes:

/usr/lib/spades/bin/spades.py -1 P27_forward_paired.fq.gz -2 P27_reverse_paired.fq.gz -o spades_assembly_all_illumina

However, I am receiving the following error message during the read error correction step:

0:32:20.448   240M / 3G    ERROR  K-mer Counting           (kmer_data.cpp             : 351)   The reads contain too many k-mers to fit into available memory. You need approx. 11.987GB of free RAM to assemble your dataset

== Error ==  system call for: "['/usr/lib/spades/bin/spades-hammer', '/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/corrected/configs/config.info']" finished abnormally, err code: 255

In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru

or report an issue on our GitHub repository github.com/ablab/spades

Please provide us with params.txt and spades.log files from the output directory.

I have attached the following files for your reference:

- params.txt

- spades.log

I would appreciate any guidance or suggestions you can provide to help resolve this issue. Please let me know if there are any additional logs or files that would be helpful for troubleshooting.

Thank you for your attention and support.

Best regards,

Atenciosamente,

José Ailton Cruz Macêdo dos Santos
Estudante de Pós-graduação em fitopatologia
Universidade Federal Rural de Pernambuco

Departamento de Agronomia UFRPE

Pernambuco-PE
Fone: (81) 986837383 OI

E-mail: santos.j.a.c.m@gmail.com
Currículo Lattes: http://lattes.cnpq.br/1553284695502900

JOSÉ SANTOS Anexos sex., 14 de jul., 15:05 (há 11 dias) para spades.support

I am encountering an error while running SPAdes in the Windows Subsystem for Linux 2 (WSL2) environment and would appreciate your assistance in resolving the issue.

Error description:
I am attempting to run SPAdes version 3.13.1 in WSL2 with the following system information:

• SPAdes version: 3.13.1
• Python version: 3.10.6
• OS: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

I am using the following command to run SPAdes:
/usr/lib/spades/bin/spades.py -1 P27_forward_paired.fq.gz -2 P27_reverse_paired.fq.gz -o spades_assembly_all_illumina

However, I am receiving the following error message during the read error correction step:
0:32:20.448 240M / 3G ERROR K-mer Counting (kmer_data.cpp : 351) The reads contain too many k-mers to fit into available memory. You need approx. 11.987GB of free RAM to assemble your dataset
== Error == system call for: "['/usr/lib/spades/bin/spades-hammer', '/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/corrected/configs/config.info']" finished abnormally, err code: 255
In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru
or report an issue on our GitHub repository github.com/ablab/spades
Please provide us with params.txt and spades.log files from the output directory.

I have attached the following files for your reference:

• params.txt
• spades.log

I would appreciate any guidance or suggestions you can provide to help resolve this issue. Please let me know if there are any additional logs or files that would be helpful for troubleshooting.

Thank you for your attention and support.

Best regards,

Atenciosamente,

José Ailton Cruz Macêdo dos Santos
Estudante de Pós-graduação em fitopatologia
Universidade Federal Rural de Pernambuco
Departamento de Agronomia UFRPE
Pernambuco-PE
Fone: (81) 986837383 OI
E-mail: santos.j.a.c.m@gmail.com
Currículo Lattes: http://lattes.cnpq.br/1553284695502900

WSL: # Settings apply across all Linux distros running on WSL 2
[wsl2]

Limits VM memory to use no more than 4 GB, this can be set as whole numbers using GB or MB

memory=12GB

Sets amount of swap storage space to 8GB, default is 25% of available RAM

swap=10GB

SPADE.LOG: Command line: /usr/lib/spades/bin/spades.py -1 /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_forward_paired.fq.gz -2 /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_reverse_paired.fq.gz -o /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina

System information:
SPAdes version: 3.13.1
Python version: 3.10.6
OS: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Output dir: /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_forward_paired.fq.gz']
right reads: ['/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_reverse_paired.fq.gz']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: automatic selection based on read length
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/tmp
Threads: 16
Memory limit (in Gb): 9

======= SPAdes pipeline started. Log can be found here: /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/spades.log

===== Read error correction started.

== Running read error correction tool: /usr/lib/spades/bin/spades-hammer /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/corrected/configs/config.info

0:00:00.000 4M / 4M INFO General (main.cpp : 75) Starting BayesHammer, built from N/A, git revision N/A
0:00:00.000 4M / 4M INFO General (main.cpp : 76) Loading config from /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/corrected/configs/config.info
0:00:00.014 4M / 4M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 12
0:00:00.015 4M / 4M INFO General (memory_limit.cpp : 49) Memory limit set to 9 Gb
0:00:00.015 4M / 4M INFO General (main.cpp : 86) Trying to determine PHRED offset
0:00:00.015 4M / 4M INFO General (main.cpp : 92) Determined value is 33
0:00:00.015 4M / 4M INFO General (hammer_tools.cpp : 36) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
0:00:00.015 4M / 4M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes
=== ITERATION 0 begins ===
0:00:00.016 4M / 4M INFO K-mer Index Building (kmer_index_builder.hpp : 301) Building kmer index
0:00:00.016 4M / 4M INFO General (kmer_index_builder.hpp : 117) Splitting kmer instances into 192 files using 12 threads. This might take a while.
0:00:00.016 4M / 4M INFO General (file_limit.hpp : 32) Open file limit set to 1024
0:00:00.016 4M / 4M INFO General (kmer_splitters.hpp : 89) Memory available for splitting buffers: 0.249891 Gb
0:00:00.016 4M / 4M INFO General (kmer_splitters.hpp : 97) Using cell size of 174686
0:00:00.363 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 97) Processing /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_forward_paired.fq.gz
0:00:16.308 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 1277388 reads
0:00:27.416 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 2574638 reads
0:00:45.084 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 3894827 reads
0:00:55.600 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 5194453 reads
0:01:06.433 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 6504286 reads
0:01:17.793 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 7834471 reads
0:01:34.915 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 9136749 reads
0:01:45.419 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 10431629 reads
0:01:56.158 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 11733306 reads
0:02:56.505 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 16954423 reads
0:05:36.467 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 97) Processing /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_reverse_paired.fq.gz
0:06:40.406 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 34323060 reads
0:11:42.636 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 112) Total 58197810 reads processed
0:11:42.791 48M / 3G INFO General (kmer_index_builder.hpp : 120) Starting k-mer counting.
0:30:20.880 48M / 3G INFO General (kmer_index_builder.hpp : 127) K-mer counting done. There are 321772420 kmers in total.
0:30:20.880 48M / 3G INFO General (kmer_index_builder.hpp : 133) Merging temporary buckets.
0:30:43.278 48M / 3G INFO K-mer Index Building (kmer_index_builder.hpp : 314) Building perfect hash indices
0:31:09.590 240M / 3G INFO General (kmer_index_builder.hpp : 150) Merging final buckets.
0:31:52.099 240M / 3G INFO K-mer Index Building (kmer_index_builder.hpp : 336) Index built. Total 149214408 bytes occupied (3.70981 bits per kmer).
0:32:20.448 240M / 3G ERROR K-mer Counting (kmer_data.cpp : 351) The reads contain too many k-mers to fit into available memory. You need approx. 11.987GB of free RAM to assemble your dataset

== Error == system call for: "['/usr/lib/spades/bin/spades-hammer', '/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/corrected/configs/config.info']" finished abnormally, err code: 255

In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru
or report an issue on our GitHub repository github.com/ablab/spades
Please provide us with params.txt and spades.log files from the output directory.

PARAMS: Command line: /usr/lib/spades/bin/spades.py -1 /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_forward_paired.fq.gz -2 /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_reverse_paired.fq.gz -o /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina

System information:
SPAdes version: 3.13.1
Python version: 3.10.6
OS: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Output dir: /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_forward_paired.fq.gz']
right reads: ['/home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/P27_reverse_paired.fq.gz']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: automatic selection based on read length
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /home/ailton_ubuntu/bioinformatica/Trimmomatic-0.39/spades_assembly_all_illumina/tmp
Threads: 16
Memory limit (in Gb): 9

[asl]

The log reads:

0:32:20.448   240M / 3G    ERROR  K-mer Counting           (kmer_data.cpp             : 351)   The reads contain too many k-mers to fit into available memory. You need approx. 11.987GB of free RAM to assemble your dataset

So you need more RAM to assemble your dataset. You machine has only 9 that is not enough.