transformer-lens-interpretability
Category: ml-evaluation-and-safety · License: MIT · Version: 1.0.0
Tags: Mechanistic Interpretability, TransformerLens, Activation Patching, Circuit Analysis
When to use it
Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.Example requests
Ask your agent in plain language, for example:- “How do I cache all activations for gpt2-small on a prompt and look at layer 5 attention patterns?”
- “I’m working in ioi_patching.ipynb with gpt2-small on 200 IOI prompts (e.g. ‘When Mary and John went to the store, John gave a drink to’). I want to run path/activation patching over resid_pre at every layer and position and plot a heatmap of logit-difference recovery, but my corrupted and clean prompts have different token lengths and I get a shape mismatch error in the hook. What should I do?”
- “I trained a tiny 2-layer attention-only transformer on repeated random token sequences and want to figure out which heads copy the token that followed the previous occurrence of the current token, and prove they’re doing so causally. How should I go about it?”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole ml-evaluation-and-safety category with /plugin install ml-evaluation-and-safety@research-agent-skills.
Related
nnsight-remote-interpretabilitypyvene-interventionssparse-autoencoder-training- Skills for machine learning research