Skip to content

4. Preparing data for ephys GUI

Jump to bottom
mayofaulkner edited this page Apr 27, 2021 · 8 revisions

In order to use the alignment GUI specific datasets with given names and dimensions must be provided. An overview of expected input can be seen in the Overview of datasets section

Here we provide some information and example code snippets that we hope can help convert your data into the correct format for the GUI.

A word of caution. The alignment GUI and ibllib-repo codebase on which it heavily relies, have been developed and extensively tested with data collected using the Neuropixel 1.0 probe. While we have attempted to design the code to be generalisable to different probe types and channel layouts, we cannot guarantee that things will transform smoothly. If you find you are getting errors, please post an issue here and we will do our best to address the problems!

Preparing Ephys data

The following code snippet can be used to convert ephys data acquired using spikeglx and sorted using Kilosort into the correct format for the GUI

from pathlib import Path
from atlaselectrophysiology.extract_files import extract_data

# Path to KS2 output
ks_path = Path(r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/KS2')

# Path to raw ephys data
ephys_path = Path(r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/raw_ephys_data')

# Save path
out_path = Path(r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/alf')

extract_data(ks_path, ephys_path, out_path)

Preparing histology data

Coordinate systems

Coordinates in the ephys alignment GUI are given with respect to bregma with x == ML, y == AP and z == DV. Bregma is defined to be located at a distance of ML = 5739 um, AP = 5400 um and DV = 332 um from the front, top, left corner (from point of view of mouse) of the Allen CCF data volume.

If you have traced probe tracks that are in the Allen CCF coordinate framework, the following code snippets can be used to transform between Bregma coordinate space (origin bregma) and CCF coordinate space (origin front, top, left corner of Allen CCF)

import ibllib.atlas as atlas
import numpy as np

# resolution of Allen CCF atlas
res = 25
brain_atlas = atlas.AllenAtlas(res)

###################################################
# Transform coords from CCF origin (order mlapdv) to Bregma origin (order mlapdv)
# N.B. x == ML, y == AP, z == DV
# example coordinates in um with CCF origin 
ccf_mlapdv = np.array([[3000, 4000, 3000], [6000, 6000, 500] ], dtype=np.float)
bregma_mlapdv = brain_atlas.ccf2xyz(ccf_mlapdv, ccf_order='mlapdv')

# Transform coords from CCF origin (order apdvml) to Bregma origin (order mlapdv)
ccf_apdvml = np.array([[3000, 4000, 3000], [6000, 6000, 500] ], dtype=np.float)
bregma_mlapdv = brain_atlas.ccf2xyz(ccf_apdvml, ccf_order='apdvml')
                                  
###################################################                                  
# Transform coords from Bregma origin (order mlapdv) to CCF origin (order mlapdv)
# example coordinates in m with Bregma origin
bregma_mlapdv = np.array([[2000, 4000, 0], [4000, -1000, -4000]]) / 1e6 
ccf_mlapdv = brain_atlas.xyz2ccf(bregma_mlapdv, ccf_order='mlapdv')

# Transform coords from Bregma origin (order mlapdv) to CCF origin (order apdvml)
bregma_mlapdv = np.array([[2000, 4000, 0], [4000, -1000, -4000]]) / 1e6
ccf_apdvml = brain_atlas.xyz2ccf(bregma_mlapdv, ccf_order='apdvml')

Probe tracing using brainreg-segment

If you have used brainreg and brainreg-segment to trace probes in the brain, the following code snippet can be used to transform the traced track .npy file output to create the xyz_pick.json file needed for the GUI

import numpy as np
from pathlib import Path
import json
from ibllib.atlas import AllenAtlas

atlas = AllenAtlas(25)

brainreg_path = Path(r'C:\Users\Mayo\Downloads\brainreg\output\allen_mouse_25um\manual_segmentation\atlas_space\tracks\track_1.npy')
# Load in coordinates of track in CCF space (order - apdvml, origin - top, left, front voxel
xyz_apdvml = np.load(brainreg_path)
# Convert to IBL space (order - mlapdv, origin - bregma)
xyz_mlapdv = atlas.ccf2xyz(xyz_apdvml, ccf_order='apdvml') * 1e6
xyz_picks = {'xyz_picks': xyz_mlapdv.tolist()}

# Path to save the data (same folder as where you have all the data)
output_path = Path(r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/alf')
with open(Path(output_path, 'xyz_picks.json'), "w") as f:
    json.dump(xyz_picks, f, indent=2)

Probe tracing using Lasagna

Within the IBL, we use lasagna for probe tracing. For instructions of how to use lasagna to trace probes please see this section

A few important points for tracing

  • When tracing in lasagna do not apply any rotations, flips or mirrors to the image
  • Within the IBL, we always trace in the histology image that has been already been registered to the Allen atlas. If you trace in the original image space, make sure you apply the registration transform to the output from lasagna
  • Save the __pts line rather than the __fit line

Given the output of the user traced tracks xxx_pts.csv from lasagna. The following code snippet can be used to create the xyz_pick.json file needed for the GUI

from ibllib.pipes.histology import load_track_csv
from pathlib import Path
import json

# Path to tracing track from lasagna
file_track = r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/lasagna/SWC_023_tracing_pts.csv' 
xyz = load_track_csv(file_track) * 1e6
xyz_picks = {'xyz_picks': xyz.tolist()}

# Path to save the data (same folder as where you have all the data)
output_path = Path(r'C:/Users/Mayo/Downloads/FlatIron/SWC_023/alf')
with open(Path(output_path, 'xyz_picks.json'), "w") as f:
    json.dump(xyz_picks, f, indent=2)
Clone this wiki locally