v5.3.2

v5.3.2 (2025-01-17)

Fix

• fix: fixing jumprelu threshold when folding dec norm (#404) (06b6669)

v5.3.1

v5.3.1 (2025-01-14)

Chore

• chore: Fixed bug in the neuronpedia_integration cell in notebook (#402)

• corrected basic loading tutorial

• fix formatting

---

Co-authored-by: David Chanin <chanindav@gmail.com> (4935b83)

Fix

• fix: removing deleted saebench SAEs from pretrained_saes.yaml (#403) (65e1986)

v5.3.0

v5.3.0 (2024-12-29)

Feature

• feat: Replace assert statements with exception code (#400)

• replaces assert statements with exception code

• replaces assert statements with exception code in less obvious cases

• removes unnecessary if and else statements (324be25)

v5.2.1

v5.2.1 (2024-12-15)

Unknown

• Merge pull request #398 from jbloomAus/np_yaml

fix: width for llamascope 32x was incorrect (0617dba)

• Update Llama Scope NP ids (26899cd)

v5.2.0

v5.2.0 (2024-12-06)

Chore

• chore: fix tokenizer typing for bos_token_id (#399) (b3b67d6)

• chore: Replace isort black and flake8 with ruff (#393)

• replaces in cache_activations_runner.py

• replaces isort, black, adn flake8 with Ruff

• adds SIM lint rule

• fixes for CI check

• adds RET lint rule

• adds LOG lint rule

• fixes RET error

• resolves conflicts

• applies make format

• adds T20 rule

• replaces extend-select with select

• resolves conflicts

• fixes lint errors

• update .vscode/settings.json

• Revert "update .vscode/settings.json"

This reverts commit 1bb5497.

• updates .vscode/settings.json

• adds newline (52dbff9)

Feature

• feat: Save estimated norm scaling factor during checkpointing (#395)

• refactor saving

• save estimated_norm_scaling_factor

• use new constant names elsewhere

• estimate norm scaling factor in ActivationsStore init

• fix tests

• add test

• tweaks

• safetensors path

• remove scaling factor on fold

• test scaling factor value

• format

• format

• undo silly change

• format

• save fn protocol

• make save fn static

• test which checkpoints have estimated norm scaling factor

• fix test

• fmt (63a15a0)

Fix

• fix: force build (53180e0)

• fix: typo in pretrained yaml (9db9e36)

Unknown

• Merge pull request #397 from jbloomAus/np_yaml

fix: typo in pretrained yaml (19bcb2e)

v5.1.0

v5.1.0 (2024-11-30)

Feature

• feat: Replace print with controllable logging (#388)

• replaces in pretrained_sae_loaders.py

• replaces in load_model.py

• replaces in neuronpedia_integration.py

• replaces in tsea.py

• replaces in pretrained_saes.py

• replaces in cache_activations_runner.py

• replaces in activations_store.py

• replaces in training_sae.py

• replaces in upload_saes_to_huggingface.py

• replaces in sae_training_runner.py

• replaces in config.py

• fixes error for CI

---

Co-authored-by: David Chanin <chanindav@gmail.com> (2bcd646)

v5.0.0

v5.0.0 (2024-11-29)

Breaking

• feat: Cleaned up CacheActionsRunnerConfig (#389)

BREAKING CHANGE: Superfluous config options have been removed

• Cleaned up CacheActionsRunnerConfig

Before CacheActivationConfig had a inconsistent config file for some
interopability with LanguageModelSAERunnerConfig. It was kind of
unclear which parameters were necessary vs redundant, and just was
fairly unclear.

Simplified to the required arguments:

• dataset_path: Tokenized or untokenized dataset
• total_training_tokens
• model_name
• model_batch_size
• hook_name
• final_hook_layer
• d_in

I think this scheme captures everything you need when attempting to
cache activations and makes it a lot easier to reason about.

Optional:

activation_save_path # defaults to "activations/{dataset}/{model}/{hook_name}
shuffle=True
prepend_bos=True
streaming=True
seqpos_slice
buffer_size_gb=2 # Size of each buffer. Affects memory usage and saving freq
device="cuda" or "cpu"
dtype="float32"
autocast_lm=False
compile_llm=True
hf_repo_id # Push to hf
model_kwargs # `run_with_cache`
model_from_pretrained_kwargs

• Keep compatiability with old config

• Renamed to keep values same where possible
• Moved _from_saved_activations (private api for CachedActivationRunner)
  to cached_activation_runner.py
• Use properties instead of __post_init__ (d81e286)

v4.4.5

v4.4.5 (2024-11-24)

Fix

• fix: add missing np (9d26da4)

Unknown

• Merge pull request #387 from jbloomAus/np_yaml

fix: add missing neuronpedia yaml entries (deae2a7)

v4.4.4

v4.4.4 (2024-11-24)

Fix

• fix: add missing np (3192463)

Unknown

• Merge pull request #386 from jbloomAus/np_yaml

fix: add missing neuronpedia yaml entries (e35f998)

v4.4.3

v4.4.3 (2024-11-24)

Fix

• fix: add missing np (9b2a19c)

Unknown

• Merge pull request #385 from jbloomAus/np_yaml

fix: add missing neuronpedia yaml entry (7ac5253)