HybridSPAdes - different graph outputs

[nettanetta]

Hello!
I'm using HybridSPAdes and I would like to make some analysis on the assembly graph. I have a few questions:

1. Regarding the different graphs outputted by the tool - is assembly_graph_after_simplification.gfa or assembly_graph_with_scaffold.gfa equivalent to assembly_graph.fastg? Can you share some more details about the diffrences between the 3 outputs? are the long read used for generating any of these outputs or they are based exclusively on short reads?
2. Is there a way for me to see the mapping of the short reads to edges in the assembly graph?

Thank you very much,
Netta

[asl]

Hello

1. assembly_graph.fastg is an assembly graph in FASTG format while assembly_graph_after_simplification.gfa and assembly_graph_with_scaffolds.gfa are assembly graphs in GFA format. The last file contains in addition scaffold paths. See SPAdes manual https://cab.spbu.ru/files/release3.15.4/manual.html#spadesoutput for more information about SPAdes output. Assembly graphs are constructed from Illumina reads and long noisy reads are used for repeat resolution and coverage gap closure (see hybridSPAdes paper for more information).
2. We are unaware of any such user-friendly tool. However, you can align short reads down to edges of the graph using your favourite short read aligner tool. There is spades-gmapper tool though which you might modify for this purpose.

[nettanetta]

Thanks for the incredibly quick and detailed response!
One last thing that still isn't clear to me - can I assume that assembly_graph_with_scaffolds.gfa and assembly_graph.fastg is the same graph but in different formats? or are they from different stages of the pipeline?

I'm asking because I want to map long reads to the graph using spades-gmapper that works with the GFA format, but aso parse the graph and run my own python code, and I could only find packages that parse fastg in python.

Thanks again :)
Netta

[asl]

There are certainly lots of packages that enables use of GFA from Python including e.g. https://github.com/ggonnella/gfapy The list at https://github.com/GFA-spec/GFA-spec might certainly help

These are the same graphs.

For mapping long reads to graph we'd suggest using more user-oriented tools like SPAligner.

[nettanetta]

Thanks! will do (regarding all your suggestions)

[nettanetta]

Hi again Anton (or anyone else that can help),
I have a few more questions regarding this subject:

1. From what's written in the documentation I understand that assembly_graph.fastg should represent the same graph as assembly_graph_after_simplification.gfa, but from your answer above I understand that assembly_graph.fastg should represent the same graph as assembly_graph_with_scaffolds.gfa. Is one of the statements correct?
2. reading the HybridSpades paper and documentation I understand that assembly_graph.fastg is generated only based on short reads because it's generated before the stages involving long reads (gap filling, repeats resolution) are executed. Am I correct? if not, can please explain what I'm missing?

[nettanetta]

When I try checking Q1 myself, the numbers don't align in 100%:

• I parsed assembly_graph.fastg using pyfastg.parse_fastgand got a graph with 91123 nodes and 73096 edges (As far as I understand, it parses the sequences as nodes)
• I counted the number of lines of each type in the gfa files and got:
  assembly_graph_with_scaffolds.gfa - {'S': 45593, 'L': 36548, 'P': 36830}
  assembly_graph_after_simplification.gfa - {'S': 67901, 'L': 36548}

45593 * 2 = 91186, that is close to 91123 but not identical.
36548 * 2 = 73096.
So from what I see here, it seems that in the fastg files (almost) all the nodes and edges are stated twice. is this because of reverse complement?

Thanks again
Netta

[asl]

Hi again Anton (or anyone else that can help), I have a few more questions regarding this subject:

1. From what's written in the documentation I understand that assembly_graph.fastg should represent the same graph as assembly_graph_after_simplification.gfa, but from your answer above I understand that assembly_graph.fastg should represent the same graph as assembly_graph_with_scaffolds.gfa. Is one of the statements correct?

These are different representations of the very same graph. The only difference is assembly_graph_with_scaffolds.gfa contains also scaffolds paths.

2. reading the HybridSpades paper and documentation I understand that assembly_graph.fastg is generated only based on short reads because it's generated before the stages involving long reads (gap filling, repeats resolution) are executed. Am I correct? if not, can please explain what I'm missing?

No. See above.

[asl]

So from what I see here, it seems that in the fastg files (almost) all the nodes and edges are stated twice. is this because of reverse complement?

Sure. Also note that there are self reverse-complementary edges (the ones that are complementary to themselves)

[nettanetta]

Thank you very much Anton!
One last question regarding the numbers -
assembly_graph_with_scaffolds.gfa - {'S': 45593, 'L': 36548, 'P': 36830}
assembly_graph_after_simplification.gfa - {'S': 67901, 'L': 36548}

Now that I understand that assembly_graph_with_scaffolds.gfa and assembly_graph_after_simplification.gfa are the same graph except for the addition of scaffolds - why does the first has 45593 segments and the second has 67901?
I'm trying to understand which one will be best for me to use
Thanks again

[asl]

Will you please provide spades.log file from that run?

[nettanetta]

Of course!

spades.log

[asl]

Ok, to close this discussion. We changed the behavior recently:

• Without long reads the graphs are the same, only scaffolds paths are added
• With long reads, the assembly_graph_with_scaffolds.gfa is different as it contains short-read-graph gaps closed using long read data in addition.

[padpadpadpad]

Hi @nettanetta @asl (or anyone else). Did you ever work out whether HybridSpades outputs an assembly graph after the long-reads have been used?

I have some assemblies from a sequencing centre and want to see whether two contigs are separate or attached (I think one might be a plasmid/phage), but the assembly graphs have many many more contigs than the final contig.fasta.

[asl]

@padpadpadpad What do you mean as "many more contigs"? GFA includes exactly the same contig paths as scaffolds.fasta.

[padpadpadpad]

@asl I must be misunderstanding something somewhere...

The longest node in scaffolds.fasta is 6,549,044 bp and there are 115 Nodes, but in assembly_graph_with_scaffolds.gfa the longest Node is 336,830 bp and there are 785 nodes.

Essentially I was expecting a one really long Node in the assembly graph after using HybridSpades but I do not know where it is if it exists.

Sorry for being an idiot if it is an easy fix/something i am missing.

Attached are a few files from the sequencing centre.

Archive.zip

[asl]

Essentially I was expecting a one really long Node in the assembly graph after using HybridSpades but I do not know where it is if it exists.

SPAdes emits assembly graph, not so-called scaffold graph. In assembly graph every scaffold is essentially (a set of) paths spanning multiple nodes of a graph. So, your NODE_1 traverses the graph. It is emitted as a P (path) record in GFA:

P	NODE_1_length_6549044_cov_107.531167_1	1547615-,4614395-,4618749-,4621626+,4188826+,4617435-,4188826+,4620784-,4538858-,4597856+,4621562-,4620534+,4591970+,4621972+,4605976-,4560632-,4605976-,4620762+,3873976-,4620446-,3873976-,4620736-,4520022-,4619328-,4520022-,4621714+,4515528+,4616257+,4515528+,4618279-,4538818+,4618277-,4538818+,4616239-,4557388+,4557394+,4621188-,4445619+,4620780-,4580182-,4620572+,4580182+,4580188+,4587976+,4621648+,4290096-,4613267-,4471212-,4620752+,4594350-,4577848+,4577850+,4161336-,4585964-,3698491+,4544956-,4619494+,4594350-,4516518+,4577850+,4110319-,4585964-,3389553-,4544956-,4621162-,4519086+,4436623-,4610668-,4620018-,4619116+,4461005+,4617837-,4593206-,4488228-,4478681-,4617787+,4621550+,4617999+,3272846+,4621970-,3850286-,4620774-,3630309+,4621504+,4547700+,4620148+,4514712+,4514714+,4427995-,4279484+,4507810-,4492420+,4422952+,4365835+,4574390-,4547730+,4587060+,4482210-,3902066+,4281994+,4281996+,4280548+,4280550+,4182402-,4380909+,4621588+,4614635-,4621586+,4614635-,4620726+,4607918+,4612987-,4476204-,4616127+,4587676+,4587682+,4587686+,4621776-,4621774-,4621778-,4493350+,3882332-,4502910-,4502908-,4587686+,4621776-,4621638-,4493350+,4323566-,4502910-,4535584+,4618635+,4618637+,4618635+,3212900+,4523798+,4621194-,4602092+,4617987+,4607918+,4607892+,4607898+,4619358+,4476204-,4618393+,4621776-,4621774-,4613509-,4523798+,4620556+,4602092+,4612985-,4607898+,4613261+,4587676+,4614913+,4568882+,4621976-,4445419-,4348129-,4610060+,4598700+,4600736+,3612963-,4609510+,4620584+,4609510-,3367165-,4600736-,4052400+,4610060-,4610058-,4445419+,4620750-,4583332-,4583326-,4583320-,2957187-,4543818+,4543824+,4620014-,4621568-,4549362-,4619352-,4620550+,4620798-,4614291+,4537600-,4595114+,3985847-,4604046-,4618570+,4621576+,4543734-,4603236+,4620174+,4620342+,4601882+,4478580+,4618598-,4592748-,4519394-,4519392-,4485912-,4485930-,4617281+,4583332-,4583326-,4583320-,4617279+,2925683+,4543822+,4543812+,4543818+,4543824+,4576336+,4586884+,4621568-,4549362-,4619352-,4620550+,4620798-,4620362+,4556874+,4556868+,4615657+,4421823-,4263414+,4595118+,4595114+,4604054-,4604046-,4604040-,4612451+,4621576+,4591944-,4603236+,4613825+,4620342+,4601882+,4478580+,4618598-,4592748-,4592740-,4620340-,4619910+,4547700-,4621702-,2412560+,4613035+,4608224+,4560150-,1635256+,2954757+,2412560+,4618606+,4608224+,4608230+,1635256+,4620714-,4619750-,4534576-,4618917-,4051578-,4619750-,4619224+,4618917-,4619232+,3734413-,3911640-,4620090+,3734413-,3911640-,3747497-,4605738+,3911640-,3747497-,4620720-,4445619+,4621652-,4571958-,4621654-,4571958-,4620670-,4568882+,4621676-,4621890-,4621582+,4620228-,4621896-,4621890-,4621884-,4621878-,4621894-,4620228-,4621896-,4621890-,4621884-,4621878-,4615345-,4621896-,4621890-,4621884-,4621658-,4123753+,4619770+,4538858-,4597856+,4621562-,4620676+,4117619-,4620544-,4380909-,2832866-,4280550-,4325210-,4281996-,3902068-,3902066-,4160722-,4587060-,4587056-,4574390+,4422954-,4422952-,4507814+,4507810+,4507816+,4427995+,4428001+,4514712-,4620776-,3850286+,4613469+,4215964-,4620764+,4215964+,4598266+,4327524+,4620086+,4468554+,4468552+,4468554+,4620476-,4591970-,4610218-,4587976-,4621330-,4488702-,4615461+,4327524+,4619796-,4448337-,4305186-,4305178-,4417959+,4488702-,4617261-,3919816+,4620558+,3919816+,4620646+,4569274-,4569272-,4569264-,4620384+,3935682-,3935686-,4490492+,4537468+,4593512+,4593514+,4485912-,4480262+,4620020+,4617281+,4583332-,4583326-,4583320-,4617279+,4614813+,4543812+,4543818+,4612587-,4586884+,4621568-,3103616+,4619352-,4619308+,4620798-,4614291+,4618019+,4556874+,4556868+,4556876+,4421829-,4421823-,4421833-,4207376-,4595118+,4595114+,3985847-,4604046-,4618570+,4621576+,4543734-,4603236+,4620174+,4620342+,4612184+,3491827+,4478580+,4618598-,4620716+,4512744-,4557394-,4620640+,4448337-,3146102+,4305178-,4620796+,4621680+,4618255+,4619071+,4579420+,2742113+,4621710+,4370273+,4617977+,4618953+,4620134+,2447037-,2924830+,2561749-,2935129+,2742113+,4618401+,3916884+,4620450+,4515956-,4515950-,4619778+,4621680+,4618255+,4619071+,4618253+,2447037-,2924830+,3193543+,2223546+,4612717-,4287880-,3978265+,3850784+,3980169+,3980159+,4615003-,4507458-,2489367+,4621708+,4578646+,4578650+,4618953+,4620134+,2447037-,2924830+,2561749-,4615283-,1496553+,4615285-,4612074+,4612080+,4616811-,3814592-,3814590-,4507458-,4507452-,4507446-,4617975+,3980159+,3858674+,4515950-,4617759+,2915017+,4481782-,4578646+,4618562+,1496553+,4621682+,4619071+,4618253+,2447037-,2924830+,3193543+,2223546+,3885764+,4507446-,4618399+,3916884+,4620450+,4515956-,4515950-,4619778+,4621680+,4515966-,4612080+,4619069+,2223546+,4612717-,4287880-,4621950-,4621952-,4621910-,4621952-,4621756-,4569272-,4613833+,4471212+,4620232+,4290096+,4620324+,4595096+,4282624-,4585194-,3696501+,4512280+,4096953-,3475685-,4040978-,4108487-,3258144-,4001231-,4616593-,4548152+,4620718+,4621562+,4099961-,4538858+,4620978+,4548152-,4616595+,4001231+,3535768+,4108487+,1835339-,3475685+,4285594-,4512280-,4474314+,4585194+,4585196+,4595096-,4619868-,4620960+,4620662+,4613415+,4620964+,4620960+,4620662+,4591574+,4587602-,4615481+,4612685+,4587602-,4613417+,4601446+,4620964+,4620960+,4620966+,4612685+,4587602-,4613417+,4620038-,4123753+,4621670-,4495012-,4618763-,4621668-,4618763-,4588248+,3381633+,4495012-,4618763-,4588248+,4618325-,4560774-,4617095-,4560774-,4619226-,4351363+,4284734-,4351363+,4544784-,4449695+,4618919-,4589246-,4589238-,4589232-,4617849+,4618921-,4531560-,3673901-,4290572+,4532508-,4290562+,4562650-,4562644-,4562648-,4612196-,4510600-,4607402+,4617781+,3041886+,3041866+,4616655-,3691789-,4619292-,4589246-,4589238-,4589232-,4617849+,4618921-,1954051-,4389935-,4576262+,4619594+,4336231+,4562650-,4562644-,4562648-,4510176+,4615145+,4607402+,4613481+,4225112+,4225108+,4528152+,4612475+,4589232-,4617849+,4614943-,4389935-,3673875-,3673901-,3568874-,4617855+,4562644-,4562648-,4510176+,4617853+,4510606-,4510600-,4607402+,4617781+,1923541-,2908318-,4225112+,4225108+,4618331+,4449695+,4617851+,4589238-,4589232-,4620644-,4619594+,4471086+,4617855+,4562644-,4510612-,4510606-,4510600-,4607402+,4617781+,1923541-,2606163+,3041866+,106035-,4225108+,4528152+,4615281-,4176410+,4617153-,4176410+,4621946+,4618295+,4621958+,4618295+,11-,4619592-,4476748-,3198654-,2059610+,4619592-,4476748-,3198654-,4612523-,4476748-,3854766-,3499588+,4405675-,4405669-,4621616+,4617999-,4621550-,4617787-,4609146+,4589524-,4593206+,4593202+,4593208+,4619116-,4620294+,4610668+,4506184+,4398897+,4519088-,4519086-,4227520-,3684325+,3477621+,4620532-,2934479+,4207604-,4621536+,4618673+,4602714-,3873924+,4542916+,3873926+,4461935-,3595798+,4620934-,4619140-,4511140+,4616761-,4613679-,4443325+,4619426-,4443325+,4486966-,4619626-,4532508+,4620642-,4498698+,4547178-,4498698+,4620854+,4405669+,4572738-,3499588-,4621650+,4298160-,4361557-,4298160-,4620396-,3684325-,4227520+,4519086+,4519088+,4450299+,4506184-,4610668-,4620018-,4619116+,4461005+,4461011+,4593202-,4593206-,4589524+,4143806-,4478681-,4617787+,4621550+,4619946+,4227520+,4519086+,4519088+,4450299+,4506184-,4610668-,4620018-,4619116+,4593208-,4593202-,4593206-,4589524+,4609146-,4617787+,4620658+,3272846-,4617999-,4621550-,4617787-,4609146+,4589524-,4593206+,4593202+,4593208+,4619116-,4620294+,4610668+,4506184+,4398897+,4519088-,4519086-,4227520-,3684325+,3477621+,4621544-,4530180-,4616863+,4049184-,4530180-,3803374+,4618043-,3803406+,4049184-,4530180-,3803374+,4620424-,4029970-,4029964-,4029974-,4566542+,4619776-,4029970-,4029964-,805248-,4612725-,4029964-,805248-,2788647+,4620738+,2876152-,4614491+,2876152-,4610686-,4610684-,4619976+,4610684-,4620782-,3630309+,4621962+,3489237-,4619103-,4512238-,4616221+,3489237-,4618305-,4512238-,4616221+,3489237-,4618305-,4512238-,4620380+,4076533-,4612641+,4076537-,4517066-,4517076-,4076539-,4076533-,4620016+,4517072-,4517066-,4349585-,4621942+,4621974-,2173120+,4472480+,4615815-,4472480+,4591592+,2173120+,4472480+,4620304-,4569264+,4569272+,4569274+,4620606-,4266736+,2009229+,4081221+,4619752+,4266736+,4198460+,4619752+,4266736+,2009229+,4620788+,4592740+,4592748+,4618598+,4590314+,4612184-,4620342-,4613825-,4603236-,4591944+,4621576-,4612451-,4391401+,4421833+,4421823+,4421829+,4620050-,4556874-,4618019-,4614291-,4620798+,4619308-,4619352+,3103616-,4621568+,4586884-,4612587+,4543818-,4543812-,4543822-,3928540-,4583326+,4615177-,4480262-,4485912+,4519392+,4469052+,4593512-,4588492+,4490492-,4490496-,3935682+,4620704+,4621966+,4498724-,4616871-,4498724-,4620344-,4460067+,4617573-,4091731-,4617965+,4570082-,3224042+,4245224-,4618151-,4563248-,4618153-,3408471+,4497314+,4327685-,4612609-,4538610-,4538608-,4235500-,4235504-,3759150+,4619980+,4433541-,3970369+,4582570-,4228054+,4618385+,4615671+,3821874+,4205734+,4458719+,2962935+,2868420-,4133830+,4133832+,4621182+,4601544+,4619404+,4617675+,4615341+,4540506+,4588510+,4601544+,4601546+,4617675+,4620456+,4540506+,4621324-,4619684-,4460067+,4618857+,4091731-,4618855+,4570082-,4570080-,4245224-,4613591-,4563248-,4619596-,3408471+,4617957-,4327685-,4617963-,4538610-,3850984+,4235500-,4164794+,3759150+,4613335+,4433541-,4582572-,4582570-,4071749-,4618385+,4618383+,3821874+,2719690+,4117619-,2201464+,4458719+,4314678+,2868420-,4069899-,4133832+,4620182-,4617703-,4605252+,4619876-,4605252+,4620568-,4605254+,4617703-,4605252+,4620568-,4621514-,4621930-,4512744-,4557394-,4557388-,4620684+,4614685-,4620586+,2934479+,2401985-,4621536+,4621538+,4602714-,4542914+,4542916+,4461943-,4461935-,4461941-,4620934-,4620932-,4511140+,4612677+,4298484+,4618747-,126429+	*

Assembly graphs has at least one huge advantage – they preserve the genome contiguity, while scaffold graphs are not (in the presence of e.g. unresolved repeats).