Skip to content

Commit

Permalink
Update instructions in notebook for troubleshooting automatic hyperpa…
Browse files Browse the repository at this point in the history 
…rameter tuning
  • Loading branch information
@RichieHakim
RichieHakim committed Mar 4, 2024
1 parent a524040 commit 0485d68
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 1,232 deletions.
1,178 changes: 31 additions & 1,147 deletions notebooks/jupyter/classification/classify_by_drawingSelection.ipynb
View file

Large diffs are not rendered by default.

129 changes: 44 additions & 85 deletions notebooks/jupyter/tracking/tracking_interactive_notebook.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@
"metadata": {},
"outputs": [],
"source": [
"dir_allOuterFolders = r'/media/rich/bigSSD/analysis_data/face_rhythm/mouse_0403R/stat_and_ops/'\n",
"dir_allOuterFolders = r'/media/rich/bigSSD/analysis_data/face_rhythm/mouse_0322N/stat_and_ops/'\n",
"\n",
"pathSuffixToStat = 'stat.npy'\n",
"pathSuffixToOps = 'ops.npy'\n",
Expand Down Expand Up @@ -369,7 +369,9 @@
"cell_type": "code",
"execution_count": null,
"id": "11727873-69c6-4539-b15f-cf68e8216756",
"metadata": {},
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"print(f'Pre-alignment below')\n",
Expand Down Expand Up @@ -663,15 +665,27 @@
"id": "dd14315d",
"metadata": {},
"source": [
"##### 1. Mix the similarity matrices into a single distance matrix\n",
"#### 1. Mix the similarity matrices into a single distance matrix\n",
"\n",
"This step can be done either automatically, using the `clusterer.find_optimal_parameters_for_pruning` method, or manually by specifying the `kwargs_makeConjunctiveDistanceMatrix` dictionary.\n",
"\n",
"\n",
"##### Option A: Automatic Method\n",
"This step finds the optimal parameters to mix the similarity matrices by tuning mixing parameters to maximally separate the distributions of pairwise similarities for ROI pairs known to be different and ROI pairs that are likely matched.\n",
"\n",
"The details of how this is done:\n",
"1. Mix all the similarity matrices (`s_sf`, `s_NN_z`, `s_SWT_z`, `s_sesh`) into a single similarity matrix (`sConj`) by passing them through sigmoid functions and then taking the element-wise p-norm. This bounds the values between 0-1. \n",
"2. Convert `sConj` into a distance matrix `dConj`.\n",
"3. Define distribution of known 'different' ROIs as all the pairwise similarities of ROIs from the same sessions. Define the distribution of likely 'matched' ROIs as the 'different' distribution minus the distribution of 'all' pairwise similarities.\n",
"4. Define the objective function to minimize as the sum of the overlap between the 'same' and 'different' distributions, and optimize the mixing parameters (sigmoid function and p-norm parameters) over this objective."
"Some of the details of underlying algorithm:\n",
"1. For each step in the optimization process, all the similarity matrices (`s_sf`, `s_NN_z`, `s_SWT_z`, `s_sesh`) are each passed through a sigmoid activation function that is parameterized (e.g. `'power_SF'`, `'sig_SF_kwargs'`), then all the similarity matrices are combined using a p-norm, where the 'p' is parameterized with `'p_norm'`. This results in a single conjunctive similarity matrix called `sConj` bounded between 0-1. \n",
"2. `sConj` is converted into a distance matrix `dConj` (bounded from 1-0).\n",
"3. `dConj` is then passed through the objective function: The full distance matrix is separated into a few components. First, all pairs of ROIs that are known to be from 'different' sources because they are from the same session are separated out into a distribution of pairwise distances (`d_diff`). Second, we define pairs of ROIs that are likely to be from the same source as `d_same` = `d_all` - `d_diff`. The objective function is then the overlap between the `d_diff` and `d_same` distributions. \n",
"4. The objective function is minimized by tuning the mixing parameters in `kwargs_makeConjunctiveDistanceMatrix`.\n",
"5. The output of this step is the optimal `kwargs_makeConjunctiveDistanceMatrix` dictionary.\n",
"\n",
"##### Option B: Manual Method\n",
"You can also simply specify the `kwargs_makeConjunctiveDistanceMatrix` dictionary manually. This is useful if you have a good idea of what the optimal parameters are or if the automatic method is not working well. Uncomment the code block below to overwrite the `kwargs_mcdm_tmp` variable.\n",
"\n",
"#### TROUBLESHOOTING FOR THIS STEP\n",
"- If you have any issues, just email Rich Hakim or open an issue on the github issues page.\n",
"- If you see: `'No crossover found, not plotting'`: Your data may not be easily separable. For some people, this is because the number of matching ROIs is very low compared to the number of non-matching ROIs. I recommend trying out the manual method."
]
},
{
Expand Down Expand Up @@ -722,15 +736,15 @@
"\n",
"### Uncomment below to manually specify mixing parameters\n",
"# kwargs_mcdm_tmp = {\n",
"# 'power_SF': 1.0,\n",
"# 'power_NN': 1.0,\n",
"# 'power_SWT': 0.0,\n",
"# 'p_norm': -4.0,\n",
"# # 'sig_SF_kwargs': {'mu':0.5, 'b':1.0},\n",
"# 'power_SF': 1.0, ## s_sf**power_SF (Higher values means clustering is more sensitive to spatial overlap of ROIs)\n",
"# 'power_NN': 1.0, ## s_NN**power_NN (Higher values means clustering is more sensitive to visual similarity of ROIs)\n",
"# 'power_SWT': 0.0, ## s_SWT**power_SWT (Higher values means clustering is more sensitive to visual similarity of ROIs)\n",
"# 'p_norm': -4.0, ## norm([s_sf, s_NN, s_SWT], p=p_norm) (Higher values means clustering requires all similarity metrics to be high)\n",
"# # 'sig_SF_kwargs': {'mu':0.5, 'b':1.0}, ## Sigmoid parameters for s_sf (mu is the center, b is the slope)\n",
"# 'sig_SF_kwargs': None,\n",
"# 'sig_NN_kwargs': {'mu':1.0, 'b':0.5},\n",
"# 'sig_NN_kwargs': {'mu':1.0, 'b':0.5}, ## Sigmoid parameters for s_NN (mu is the center, b is the slope)\n",
"# # 'sig_NN_kwargs': None,\n",
"# # 'sig_SWT_kwargs': {'mu':0.5, 'b':1.0},\n",
"# # 'sig_SWT_kwargs': {'mu':0.5, 'b':1.0}, ## Sigmoid parameters for s_SWT (mu is the center, b is the slope)\n",
"# 'sig_SWT_kwargs': None,\n",
"# }\n",
"\n",
Expand Down Expand Up @@ -1020,7 +1034,7 @@
"source": [
"roicat.visualization.display_toggle_image_stack(\n",
" FOV_clusters, \n",
" image_size=(np.array(FOV_clusters[0].shape)*1.5).astype(int)[:2],\n",
" image_size=1.5,\n",
"# clim=[0,1.0],\n",
")"
]
Expand All @@ -1036,10 +1050,8 @@
{
"cell_type": "code",
"execution_count": null,
"id": "17211ebd",
"metadata": {
"scrolled": true
},
"id": "760237d0",
"metadata": {},
"outputs": [],
"source": [
"roicat.visualization.display_cropped_cluster_ims(\n",
Expand All @@ -1066,7 +1078,7 @@
"metadata": {},
"outputs": [],
"source": [
"dir_save = Path('/media/rich/bigSSD/analysis_data/face_rhythm/mouse_0403R').resolve()\n",
"dir_save = Path('/media/rich/bigSSD/analysis_data/face_rhythm/mouse_0322N').resolve()\n",
"name_save = Path(dir_allOuterFolders).resolve().name\n",
"\n",
"path_save = dir_save / (name_save + '.ROICaT.tracking.results' + '.pkl')\n",
Expand Down Expand Up @@ -1101,57 +1113,6 @@
"Optionally save the FOV_clusters images as a GIF"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f27b17a-d57c-4dcf-9bd6-d9565e0d147f",
"metadata": {},
"outputs": [],
"source": [
"FOV_clusters_text = roicat.helpers.add_text_to_images(\n",
" (np.stack(FOV_clusters, axis=0)*255).astype(np.uint8)[:,:,:,:], \n",
" [[f'{ii}'] for ii in range(len(FOV_clusters))], \n",
" position=(20, 60),\n",
" font_size=2,\n",
" color=(255, 255, 255),\n",
" line_width=4,\n",
" font=None,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e23ba56b-d63e-490f-aaee-1f55133f02ba",
"metadata": {},
"outputs": [],
"source": [
"# FOV_clusters_text_rs = roicat.helpers.resize_images(\n",
"# images=FOV_clusters_text, \n",
"# new_shape=np.array(FOV_clusters_text.shape[1:3])//2,\n",
"# interpolation='bilinear',\n",
"# antialias=False,\n",
"# align_corners=False,\n",
"# device='cpu',\n",
"# )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0b2e8caa-bf9f-4e68-9595-b812cc270250",
"metadata": {},
"outputs": [],
"source": [
"roicat.helpers.save_gif(\n",
" # array=FOV_clusters, \n",
" array=FOV_clusters_text, \n",
" path=str(Path(dir_save).resolve() / 'FOV_clusters_text.gif'),\n",
" frameRate=5.0,\n",
" loop=0,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -1160,7 +1121,13 @@
"outputs": [],
"source": [
"roicat.helpers.save_gif(\n",
" array=FOV_clusters, \n",
" array=roicat.helpers.add_text_to_images(\n",
" images=[(f * 255).astype(np.uint8) for f in FOV_clusters], \n",
" text=[[f\"{ii}\",] for ii in range(len(FOV_clusters))], \n",
" font_size=3,\n",
" line_width=10,\n",
" position=(30, 90),\n",
" ), \n",
" path=str(Path(dir_save).resolve() / 'FOV_clusters.gif'),\n",
" frameRate=10.0,\n",
" loop=0,\n",
Expand All @@ -1182,19 +1149,11 @@
"metadata": {},
"outputs": [],
"source": [
"# roicat.helpers.matlab_save(\n",
"# obj=results,\n",
"# filepath='/home/rich/Desktop/results.mat',\n",
"# )"
"roicat.helpers.matlab_save(\n",
" obj=results,\n",
" filepath='/home/rich/Desktop/results.mat',\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "112cfe73",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
1 change: 1 addition & 0 deletions roicat/helpers.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3446,6 +3446,7 @@ def remap_sparse_images(
) -> List[scipy.sparse.csr_matrix]:
"""
Remaps a list of sparse images using the given remap field.
RH 2023
Args:
ims_sparse (Union[scipy.sparse.spmatrix, List[scipy.sparse.spmatrix]]):
Expand Down

0 comments on commit 0485d68

Please sign in to comment.