Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doublet detection errors with troublet: non-zero exit status and index out of bounds #231

Open
shirshuj opened this issue Apr 27, 2024 · 22 comments

Comments

@shirshuj
Copy link

I completed the tutorial and two samples of my own without error but now I'm consistently getting this problem with subsequent samples:

running souporcell doublet detection
Traceback (most recent call last):
  File "/opt/souporcell/souporcell_pipeline.py", line 596, in <module>
    doublets(args, ref_mtx, alt_mtx, cluster_file)
  File "/opt/souporcell/souporcell_pipeline.py", line 541, in doublets
    subprocess.check_call([directory+"/troublet/target/release/troublet", "--alts", alt_mtx, "--refs", ref_mtx, "--clusters", cluster_file], stdout = dub, stderr = err)
  File "/usr/local/envs/py36/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/souporcell/troublet/target/release/troublet', '--alts', 'S8C_souporcell_2/alt.mtx', '--refs', 'S8C_souporcell_2/ref.mtx', '--clusters', 'S8C_souporcell_2/clusters_tmp.tsv']' returned non-zero exit status 101.

I checked the doublets.err file which says:

thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 95256', src/main.rs:328:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

But I'm not sure how to correct this issue. Any advice or things to try is appreciated. Thank you!

@wheaton5
Copy link
Owner

Probably upstream of doublet detection if len is 0. Can you give me the first few lines of alt.mtx and the cluster_tmp.tsv?

@shirshuj
Copy link
Author

First lines of alt.mtx

%%MatrixMarket matrix coordinate real general
% written by sprs
14492 6794880 457068
1 95257 1
1 337008 1
1 1023389 1
1 1158281 1
1 2192702 1
1 3038255 1
1 3261750 1
1 3548729 1
1 3729811 1
1 3736516 1
1 3893785 1
1 4267166 1
1 4336965 1

But cluster_tmp.tsv is empty

@wheaton5
Copy link
Owner

Right so u are using the unfiltered barcodes file. Only use barcodes that are cells according to cellranger. This file should be in outs/filtered/ or something like that

@shirshuj
Copy link
Author

The filepath is going to the filtered ones for each sample but maybe something got corrupted in them. I'll try some different samples to see if they will run and I will try taking a closer look at the filtered barcodes files that aren't working.

@wheaton5
Copy link
Owner

Well that alt.mtx says you have 6.79 million cells

@shirshuj
Copy link
Author

That doesn't sound right at all but I'm looking into it. Thanks for your help in figuring out where I need to look for troubleshooting these runs!

@megannguyen1009
Copy link

I'm having this same exact error. Did you ever figure out how to fix it?

@wheaton5
Copy link
Owner

Use the filtered barcodes not the raw barcodes

@wheaton5
Copy link
Owner

In cellranger output there is a filtered folder and raw folder. Get it from the filtered folder

@megannguyen1009
Copy link

Hello thank you for your quick reply. My barcodes.tsv file is from the filtered folder, and I still got those errors. Do you know any other possibilities that could be causing this error?

@wheaton5
Copy link
Owner

Give me the first 3 lines of alt.mtx

@megannguyen1009
Copy link

My first few lines of alt.mtx

%%MatrixMarket matrix coordinate real general
% written by sprs
2112270 2035673 50014710
1 64536 1
1 140738 0
1 239400 0
1 251279 1
1 254260 0
1 279530 0
1 518165 0

@wheaton5
Copy link
Owner

wheaton5 commented Sep 13, 2024

Right. This is due to an unflitered barcodes file. This says u have 2 million cells

@Ajeet1699
Copy link

I am also having the same error. I am using the filtered barcode.
$ head alt.mtx
%%MatrixMarket matrix coordinate real general
% written by sprs
417445 20612 29276610
1 120 0
1 247 0
1 711 0
1 1707 0
1 2023 0

$ head ref.mtx
%%MatrixMarket matrix coordinate real general
% written by sprs
417445 20612 29276610
1 120 1
1 247 1
1 711 1
1 1707 1
1 2023 1

@wheaton5
Copy link
Owner

wheaton5 commented Dec 5, 2024

Okay, this looks like an actual bug. It seems to be caused when at least one cell has 0 genetic variants. Normally this is not the case, but its possible. Can I get a wc -l on the barcodes.tsv? Will help me pin down the issue to know if its 20612 as above or a diff number.

@Ajeet1699
Copy link

zcat -c ../outs/filtered_feature_bc_matrix/barcodes.tsv.gz |wc -l
20612

@wheaton5
Copy link
Owner

wheaton5 commented Dec 5, 2024

and is there any output in clusters.err? I assume that clusters_tmp.tsv is empty -- that is the cause of this error. But I'm not sure why it would be empty unless souporcell fails silently.

@Ajeet1699
Copy link

No they are not empty.
wc -l clusters.err clusters_tmp.tsv
3782 clusters.err
20612 clusters_tmp.tsv

head clusters.err
total loci used 84424
binomial 8 0 1 0 -5329254 inf
binomial 0 0 1 0 -5329254 inf
binomial 2 0 1 0 -5329254 inf
binomial 10 0 1 0 -5329254 inf
binomial 3 0 1 0 -5329254 inf
binomial 12 0 1 0 -5329254 inf
binomial 11 0 1 0 -5329254 inf
binomial 6 0 1 0 -5329254 inf
binomial 13 0 1 0 -5329254 inf

@wheaton5
Copy link
Owner

wheaton5 commented Dec 5, 2024

oh, interesting. and the first lines of clusters_tmp.tsv? maybe all unassigned?

@Ajeet1699
Copy link

I have 84 genotyped individuals. I think it is assigned.
head clusters_tmp.tsv
AAACCCAAGCAGGGAG-1 45 -494.61774 -470.01822 -494.61774 -494.61774 -500.36407 -494.61774 -494.61774 -498.5801 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -491.40506 -494.61774 -544.1983 -494.61774 -494.61774 -494.61774 -494.61774 -424.62784 -494.61774 -529.88464 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -404.41818 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -498.65744 -494.61774 -494.61774 -494.61774 -207.25584 -494.61774 -494.61774 -451.20953 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -494.61774 -490.46118 -494.61774 -520.2698 -494.61774 -494.61774 -488.23813 -494.61774 -494.61774
AAACCCAAGCGAAACC-1 20 -284.0141 -278.41904 -284.0141 -284.0141 -348.25916 -284.0141 -284.0141 -362.4212 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -281.7458 -284.0141 -133.80672 -284.0141 -284.0141 -284.0141 -284.0141 -276.55804 -284.0141 -327.51645 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -227.2315 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -315.99777 -284.0141 -284.0141 -284.0141 -276.1158 -284.0141 -284.0141 -272.93036 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -284.0141 -282.58875 -284.0141 -335.55588 -284.0141 -284.0141 -308.7206 -284.0141 -284.0141
AAACCCAAGGACCCAA-1 4 -697.7711 -653.1642 -697.7711 -697.7711 -279.85156 -697.7711 -697.7711 -761.7951 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -694.2766 -697.7711 -775.6767 -697.7711 -697.7711 -697.7711 -697.7711 -622.32385 -697.7711 -710.2363 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -504.30612 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -660.6365 -697.7711 -697.7711 -697.7711 -682.2099 -697.7711 -697.7711 -638.9098 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -697.7711 -691.37994 -697.7711 -748.58765 -697.7711 -697.7711 -726.13824 -697.7711 -697.7711
AAACCCAAGGCTTCCG-1 7 -549.47455 -519.98755 -549.47455 -549.47455 -580.8739 -549.47455 -549.47455 -191.96658 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -544.02 -549.47455 -536.6588 -549.47455 -549.47455 -549.47455 -549.47455 -462.9296 -549.47455 -511.59616 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -323.35846 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -513.1172 -549.47455 -549.47455 -549.47455 -497.1701 -549.47455 -549.47455 -488.69315 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -549.47455 -539.27313 -549.47455 -551.5304 -549.47455 -549.47455 -496.3492 -549.47455 -549.47455
AAACCCAAGTAAAGCT-1 25 -40.4805 -37.005783 -40.4805 -40.4805 -34.90482 -40.4805 -40.4805 -41.499336 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -36.462257 -40.4805 -37.288948 -40.4805 -40.4805 -40.4805 -40.4805 -23.512226 -40.4805 -34.45991 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -29.58398 -40.4805 -40.4805-40.4805 -40.4805 -40.4805 -40.4805 -29.504232 -40.4805 -40.4805 -40.4805 -30.6085 -40.4805-40.4805 -39.184963 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805-40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805-40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.4805 -40.670948 -40.4805 -35.388233 -40.4805 -40.4805 -46.162205 -40.4805 -40.4805
AAACCCAAGTCGTTAC-1 45 -246.25655 -237.54282 -246.25655 -246.25655 -266.24124 -246.25655 -246.25655 -280.84122 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -243.03888 -246.25655 -276.78778 -246.25655 -246.25655 -246.25655 -246.25655 -228.6645 -246.25655 -287.8237 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -220.83597 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -272.4318 -246.25655 -246.25655 -246.25655 -123.56724 -246.25655 -246.25655 -239.03622 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -246.25655 -244.79462 -246.25655 -259.64307 -246.25655 -246.25655 -241.01785 -246.25655 -246.25655
AAACCCACAACGGTAG-1 4 -714.15533 -664.28186 -714.15533 -714.15533 -267.86154 -714.15533 -714.15533 -869.08997 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -710.0085 -714.15533 -785.0162 -714.15533 -714.15533 -714.15533 -714.15533 -628.34705 -714.15533 -748.87024 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -539.6336 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -698.2026 -714.15533 -714.15533 -714.15533 -676.48737 -714.15533 -714.15533 -660.0904 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -714.15533 -705.69617 -714.15533 -786.8111 -714.15533 -714.15533 -719.52545 -714.15533 -714.15533
AAACCCACACGCAAAG-1 25 -31.44638 -25.99665 -31.44638 -31.44638 -25.145903 -31.44638 -31.44638 -25.52797 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -30.003725 -31.44638 -26.036934 -31.44638 -31.44638 -31.44638 -31.44638 -14.052211 -31.44638 -32.311222 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -21.551899 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -21.574358 -31.44638 -31.44638 -31.44638 -20.32691 -31.44638 -31.44638 -20.188126 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -31.44638 -30.673223 -31.44638 -17.676369 -31.44638 -31.44638 -21.101473 -31.44638 -31.44638
AAACCCACAGGACAGT-1 4 -386.59607 -373.56787 -386.59607 -386.59607 -178.48932 -386.59607 -386.59607 -442.8889 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -381.19537 -386.59607 -417.57404 -386.59607 -386.59607 -386.59607 -386.59607 -348.29416 -386.59607 -418.46332 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -293.35583 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -385.95648 -386.59607 -386.59607 -386.59607 -399.5887 -386.59607 -386.59607 -373.41452 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -386.59607 -384.8035 -386.59607 -396.4317 -386.59607 -386.59607 -377.74564 -386.59607 -386.59607
AAACCCACAGTAGTGG-1 81 -608.94727 -551.2512 -608.94727 -608.94727 -537.0687 -608.94727 -608.94727 -591.24744 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -597.4976 -608.94727 -720.0432 -608.94727 -608.94727 -608.94727 -608.94727 -590.35406 -608.94727 -607.5505 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -464.07288 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -586.5303 -608.94727 -608.94727 -608.94727 -568.83813 -608.94727 -608.94727 -534.96423 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -608.94727 -604.1079 -608.94727 -562.90076 -608.94727 -608.94727 -217.406 -608.94727 -608.94727

@wheaton5
Copy link
Owner

wheaton5 commented Dec 5, 2024

seems that many clusters are exactly the same. How that causes this error I'm still figuring out. I guess if some clusters are the same as other clusters, no cells will be assigned to them.

But even if I fix that, we will have another more difficult problem. One is a problem I'm actively researching -- clustering many individuals. Previously we have tested up to 21 individuals and that only works with many random restarts and a high number of umi/cell. 84 is definitely out of the scope of the current software. Clustering is an NP hard problem and this is additionally difficult because it is sparse data and binomial not euclidean. We should move this to an email thread if you want to continue this conversation. whheaton@gmail.com

@Ajeet1699
Copy link

I see. Thanks for very quick responses and clarification. Really appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants