[BUG] 'pattern' error during BUILDING JOB: slice timing correction #967
Comments
|
Ok defo a bug but this also means there is something wrong with your slice timing info. |
|
Sure, here it is (I'm not sure if it will work via email)
…On Mon, 27 Mar 2023 at 19:06, Remi Gau ***@***.***> wrote:
Ok defo a bug but this also means there is something wrong with your slice
timing info.
Can you send me the json of one of your bold file?
—
Reply to this email directly, view it on GitHub
<#967 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AWQTRS3KWNTKV5IGSF7QDXLW6HCJTANCNFSM6AAAAAAWJMNHDI>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
Ph. D.
Laboratory of Brain Imaging (LOBI)
Nencki Institute of Experimental Biology
Polish Academy of Sciences
Pasteur 3, 02-093 Warsaw, Poland
Tel: +48 22 5892 551
mail: ***@***.*** ***@***.***>
website: http://lobi.nencki.gov.pl/
Google Scholar profile
<https://scholar.google.com/citations?user=QI1KL4wAAAAJ&hl=pl&oi=ao>
|
|
@all-contributors add @JacMatu for bug, userTesting |
|
I've put up a pull request to add @JacMatu! 🎉 |
|
Hi Remi,
I tried to run the full preprocessing pipeline with fixed issue #967
<#967> (slice timing errors),
and I've encountered new types of errors. I have no idea why it happens,
but it looks like some runs of some subjects have different slice timings
(e.g. 1 of of 6 runs within subject is different).
Judging by the content of test_getAcquisitionTime.m
(bidspm/tests/tests_preproc/utils/) it looks like the slice timings were
hard-coded for my sequence and work usually. with values of
[0
0.0575
0.115
0.1725
0.23
0.2875
0.345
0.4025
0.46
0.5175
0.575
0.6325
0.69
0.745
0.8025
0.86
0.9175
0.975
1.0325
1.09
1.1475
1.205
1.2625
1.32
1.3775
1.435
1.4925
1.55
1.6075
1.665
1.7225
1.7775
1.835
1.8925
1.95]
However, sometimes I have a run with slice timing of
"SliceTiming": [
0,
0.0575,
0.115,
0.1725,
0.23,
0.2875,
0.345,
0.4025,
0.46,
0.5175,
0.575,
0.6325,
0.69,
0.7475,
0.805,
0.8625,
0.92,
0.9775,
1.0325,
1.09,
1.1475,
1.205,
1.2625,
1.32,
1.3775,
1.435,
1.4925,
1.55,
1.6075,
1.665,
1.7225,
1.78,
1.8375,
1.895,
1.9525
],
Which is slightly off and causes errors. I have no idea why it happened
since everything was run with an identical protocol...
See error log and example jsons attached.
I'm tempted to skip slice timing for now if it's something complicated to
fix. If I were to do this, I assumed that it could be done via
opt.stc.skip = 0, ;
changed to 1, but I'm not sure how to access opt from the script level.
Best,
Jacek
…On Wed, 29 Mar 2023 at 10:27, Remi Gau ***@***.***> wrote:
Closed #967 <#967> as
completed via #969 <#969>.
—
Reply to this email directly, view it on GitHub
<#967 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AWQTRS6L7FNNYGRNGSEGZUDW6PW5NANCNFSM6AAAAAAWJMNHDI>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
Ph. D.
Laboratory of Brain Imaging (LOBI)
Nencki Institute of Experimental Biology
Polish Academy of Sciences
Pasteur 3, 02-093 Warsaw, Poland
Tel: +48 22 5892 551
mail: ***@***.*** ***@***.***>
website: http://lobi.nencki.gov.pl/
Google Scholar profile
<https://scholar.google.com/citations?user=QI1KL4wAAAAJ&hl=pl&oi=ao>
|
|
Sorry forgot to attach the files
On Tue, 18 Apr 2023 at 16:01, Jacek Matuszewski ***@***.***>
wrote:
… Hi Remi,
I tried to run the full preprocessing pipeline with fixed issue #967
<#967> (slice timing errors),
and I've encountered new types of errors. I have no idea why it happens,
but it looks like some runs of some subjects have different slice timings
(e.g. 1 of of 6 runs within subject is different).
Judging by the content of test_getAcquisitionTime.m
(bidspm/tests/tests_preproc/utils/) it looks like the slice timings were
hard-coded for my sequence and work usually. with values of
[0
0.0575
0.115
0.1725
0.23
0.2875
0.345
0.4025
0.46
0.5175
0.575
0.6325
0.69
0.745
0.8025
0.86
0.9175
0.975
1.0325
1.09
1.1475
1.205
1.2625
1.32
1.3775
1.435
1.4925
1.55
1.6075
1.665
1.7225
1.7775
1.835
1.8925
1.95]
However, sometimes I have a run with slice timing of
"SliceTiming": [
0,
0.0575,
0.115,
0.1725,
0.23,
0.2875,
0.345,
0.4025,
0.46,
0.5175,
0.575,
0.6325,
0.69,
0.7475,
0.805,
0.8625,
0.92,
0.9775,
1.0325,
1.09,
1.1475,
1.205,
1.2625,
1.32,
1.3775,
1.435,
1.4925,
1.55,
1.6075,
1.665,
1.7225,
1.78,
1.8375,
1.895,
1.9525
],
Which is slightly off and causes errors. I have no idea why it happened
since everything was run with an identical protocol...
See error log and example jsons attached.
I'm tempted to skip slice timing for now if it's something complicated to
fix. If I were to do this, I assumed that it could be done via
opt.stc.skip = 0, ;
changed to 1, but I'm not sure how to access opt from the script level.
Best,
Jacek
On Wed, 29 Mar 2023 at 10:27, Remi Gau ***@***.***> wrote:
> Closed #967 <#967> as
> completed via #969 <#969>.
>
> —
> Reply to this email directly, view it on GitHub
> <#967 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AWQTRS6L7FNNYGRNGSEGZUDW6PW5NANCNFSM6AAAAAAWJMNHDI>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
Ph. D.
Laboratory of Brain Imaging (LOBI)
Nencki Institute of Experimental Biology
Polish Academy of Sciences
Pasteur 3, 02-093 Warsaw, Poland
Tel: +48 22 5892 551
mail: ***@***.*** ***@***.***>
website: http://lobi.nencki.gov.pl/
Google Scholar profile
<https://scholar.google.com/citations?user=QI1KL4wAAAAJ&hl=pl&oi=ao>
--
Ph. D.
Laboratory of Brain Imaging (LOBI)
Nencki Institute of Experimental Biology
Polish Academy of Sciences
Pasteur 3, 02-093 Warsaw, Poland
Tel: +48 22 5892 551
mail: ***@***.*** ***@***.***>
website: http://lobi.nencki.gov.pl/
Google Scholar profile
<https://scholar.google.com/citations?user=QI1KL4wAAAAJ&hl=pl&oi=ao>
|
|
I think you will need to drop the log manually in the github windows and not just by email. Also if you have several runs with different slice timing values:
|
|
send me the log in any case |
|
bidspm should throw a warning and I wonder if SPM won't crash after that: bidspm/src/bids/getAndCheckSliceOrder.m Line 82 in b5d8087 |
|
thanks |
|
Ok, I've found even a third set of values, which is extremely puzzling since it was all done with the same protocol. Run01 = rare, weird case of "abnormality" All in the same subject... I used dcm2bids for conversion, I'm not sure if that can cause some issues. |
|
Thanks for those. |
|
Absolutely, I see no reason why copied protocol would have different timings between runs and subjects in a random manner. If that would always be the same run# or group then maybe that would be experimentor's / technician's error in setting up the protocol... But this? I have no idea what's going on... |
|
These differences are literally in 0.0025 s, I doubt it will affect the BOLD signal, but it's annoying in the BIDS perfect world... |
|
at this stage I am more into: if even the scanner sequence are adding some jitter just for fun what hope do we have at reproducible results... 🤷🏾 |
|
Yeah. I kind of see two solutions for now (to quickly move with the analyses since I want to learn models and mvpa too)
|
|
So the weird thing is that |
|
closing for now reopen or open an new issue if something related reappears |
|
Follow up on the issue: Only 4 are left with identical error of Acquisition time cannot be < to any slice timing value. If I'm not mistaken (I have checked other random "deviant" subjects), this issue might be related to the fact that these subjects have "deviant" values in the run01. I.e., there are other subjects with identical slice timings e.g. in run05 (but their run01 is "typical") and there were no errors during preprocessing. Is it possible that run01 servers as some sort of reference or value check and then everything is assumed to be constant across runs? |
|
there is kind of something like this: bidspm/src/bids/getAndCheckSliceOrder.m Lines 70 to 88 in e684f7a this checks that all runs have the same slice timing values: so it checks each run against the first one but what puzzles me is that the error you report happens here: bidspm/src/batches/preproc/setBatchSTC.m Line 66 in e684f7a where as the check that all runs have the same values happens before: bidspm/src/batches/preproc/setBatchSTC.m Line 56 in e684f7a so if some of your runs have different slice timing, this should have been caught earlier are you using the lastest version that is on the main branch? |
|
No, I'm using the fix-acq_time_error branch from Your remote that you created when I first opened that issue |
|
Hi Remi, just to follow up: I've tried to run this analyses with up-to-date main branch bidspm and the error is the same. |
|
Do you have this data somewhere I can check it? On GIN? |
|
I have a toy repo which served as a datalad 101 on gin, what do you need? Raw data from 1 subject that is problematic and one that is not? Or something else too? |
|
Gin repo should be here Sorry it's not a perfect yoda structure, let me know if you need anything else |
|
that's perfect |
|
@JacMatu |
|
at least add a dataset_description.json: bidspm cannot even start for me even if I use |
|
In case you are lazy like me, there is a function in bids matlab to create an empty dataset_description.json you can then fill in: ds_desc = bids.Description();
ds_desc.write(fullfile(pwd, 'GIN_TOY_REPO_JM'); |
|
doing some basic QA indeed the slice timing values are different for many runs bids_dir = fullfile(this_dir, '..', 'GIN_TOY_REPO_JM');
BIDS = bids.layout(bids_dir);
%%
subjects = bids.query(BIDS, 'subjects');
%%
clc
for i_sub = 1:numel(subjects)
meta = bids.query(BIDS, 'metadata', ...
'sub', subjects{i_sub}, ...
'suffix', 'bold', ...
'target', 'SliceTiming');
values = [];
equal_to_run_1 = [];
for i = 1:numel(meta)
values(:,i) = meta{i};
equal_to_run_1(:,i) = meta{i} == meta{1}; %#ok<*SAGROW>
end
fprintf('\nSliceTiming subject: %s\n', subjects{i_sub})
disp(values);
fprintf('\nEqual to run 1\n')
disp(equal_to_run_1);
endGives me: each columns are the values from one run the second output for each subject has 1 if the value on a given row is equal to that same value for run 1: in brief ideally it should all be 1 and bidspm should crash if any value is 0 apparently it does not crash for the first subject, so some fixing is required |
|
what I think I am going to do is to modify how the slice time correction is run: I will run one job per run if not all runs have the same slice timing, but it will defo throw a warning |
|
that means that slice timing will be run on your data but you should probably figure out why your data has different slice timing for different runs |
|
Thank you Remi, I'll try to get in touch with physicists from my previous lab on that. My main folder goes through the BIDS validator without an error, sorry I just copied 2 subjects to the gin repo to share with you. |
I suspect you will have to re-run your preprocessing for everyone indeed. Once again the differences are not big so I would be surprised if this made a difference in the end. But maybe just to make you can make sure that things were done right.
ha OK I wondering how you were getting bidspm started otherwise. 😮 |
|
Ok, should be easier when it works for every subject instead of trying to find a pattern or errors like a madman. Let me know if that's something that I should change in bidspm config locally or do you plan to have a dedicated branch with a different slice timing approach that you described. |
|
most likely I will open a PR to make bidspm smarter and do this kind of thing automatically, so the only thing you should have to do is update bidspm and run it |
|
Is there an existing issue for this?
Operating system
Operating system version
SPM 12 version
Platform
Platform version
bidspm version
v3.0.0
bidspm branch / commit number
Expected Behavior
bidspm basic preprocessing called with
Current Behavior & Error message
Anything else?
error_2023-03-27T18-10.log
The text was updated successfully, but these errors were encountered: