RepIt: Steering Language Models with Concept-Specific Refusal Vectors#

Step 1: Concept-Scoped Contrastive Dataset Construction#

RepIt begins by constructing carefully curated contrastive datasets for each target concept. For a concept $c$ (e.g., “violence”), the framework gathers:

• Harmful set $H_c$: Prompts that request harmful content specifically related to concept $c$.
• Benign set $B_c$: Prompts that are semantically similar but request benign, permissible content related to the same topical domain.

The key innovation is ensuring that the benign set is concept-matched—it covers the same general topic area but remains within acceptable bounds. For violence, this might mean prompts about historical battles or martial arts training versus prompts about building weapons or harming individuals. This matching is critical because it ensures the extracted direction captures the refusal-relevant distinction rather than superficial topic features.

The paper demonstrates that poorly matched contrastive pairs—using completely unrelated benign prompts—yield noisy directions that conflate topic and safety, whereas concept-scoped pairs produce clean, interpretable vectors.

Step 2: Activation Collection and Direction Extraction#

For each concept $c$, RepIt runs the harmful and benign prompt sets through the target model and collects intermediate activations at a specified layer $l$. The choice of layer matters: earlier layers encode more syntactic features, while later layers encode more semantic and safety-relevant features. RepIt typically targets middle-to-late residual stream layers where safety-related representations are most prominent.

The refusal vector for concept $c$ at layer $l$ is computed as:

$r_c^{(l)} = \mu(H_c^{(l)}) - \mu(B_c^{(l)})$

where $\mu(\cdot)$ denotes the mean activation vector. This difference-in-means approach yields a direction that points from benign behaviour toward refusal behaviour, specifically for concept $c$.

Importantly, RepIt also applies orthogonalisation to ensure that concept-specific vectors are not simply redundant projections of a shared global direction. By decorrelating vectors across concepts, the framework verifies that each vector captures genuinely distinct information.

Step 3: Concept Independence Analysis#

A crucial empirical question is whether these concept-specific vectors are truly distinct or merely noisy variants of a single underlying direction. RepIt provides rigorous analysis on this front:

• Cosine similarity analysis. The paper measures pairwise cosine similarity between refusal vectors for different concepts, finding consistently low similarity (often below 0.3) across categories like violence, sexual content, hate speech, and illegal activities.
• Cross-concept steering experiments. When the violence refusal vector is applied to sexual content prompts (and vice versa), the steering effect is dramatically weaker than when the matching concept vector is used. This confirms functional independence.
• Ablation selectivity. Ablating the violence vector degrades refusal on violence prompts but has minimal effect on the model’s willingness to refuse sexual content—evidence that the vectors occupy separable subspaces.

These findings collectively demonstrate that safety alignment produces a structured, multi-dimensional refusal representation rather than a single scalar refusal direction.

Refusal Induction (Adding the Vector)#

By adding a scaled concept-specific refusal vector to intermediate activations during inference, RepIt can induce refusal for queries related to that concept:

$\tilde{a}^{(l)} = a^{(l)} + \alpha \cdot r_c^{(l)}$

where $\alpha$ is a scaling coefficient that controls the strength of the steering effect. The paper shows that:

• Small values of $\alpha$ produce gentle nudges, making the model more cautious about the target concept without hard refusal.
• Larger values of $\alpha$ cause outright refusal, with the model generating standard refusal templates.
• The effect is concept-selective: adding the violence vector at appropriate strength causes the model to refuse violence-related queries while remaining fully compliant on unrelated topics.

Refusal Ablation (Subtracting the Vector)#

Conversely, subtracting the concept-specific vector removes the model’s refusal capability for that concept:

$\tilde{a}^{(l)} = a^{(l)} - \alpha \cdot r_c^{(l)}$

This is the more concerning operation from a safety perspective, as it effectively jailbreaks the model for a specific category. The paper demonstrates that concept-specific ablation:

• Successfully disables refusal for the target concept while preserving refusal for all other categories.
• Requires knowledge of the specific concept vector—an attacker cannot use, say, the hate speech vector to bypass violence refusals.
• Is more robust than global refusal ablation because it does not destabilise the model’s overall safety behaviour.

The Steering Coefficient $\alpha$#

The scaling coefficient $\alpha$ acts as a continuous dial for concept-specific safety. This parameterisation is powerful because it allows:

• Graduated safety tuning. Rather than binary safe/unsafe behaviour, models can be steered toward more or less cautious behaviour on a per-concept basis.
• Context-sensitive deployment. Different deployment contexts may require different safety profiles—a medical chatbot might want lower $\alpha$ for health-related discussions while maintaining high $\alpha$ for violence.
• Attack quantification. By measuring the minimum $\alpha$ required to break refusal for each concept, the framework provides a natural metric for per-concept safety robustness.

Concept Selectivity Is Real and Robust#

Across all tested models, concept-specific refusal vectors demonstrate strong selectivity. The paper reports that ablation of a single concept’s refusal vector reduces the refusal rate for that concept by 60-90% while affecting refusal rates for other concepts by less than 10%. This is a striking result: it means that safety alignment is not a single switch but a panel of independent switches, one per concept category.

Linear Separability of Safety Concepts#

The success of difference-in-means extraction implies that harmful vs. benign representations for each concept are approximately linearly separable within the model’s activation space. This has profound implications:

• It suggests that safety training (RLHF, DPO, etc.) organises harmful content representations along relatively simple geometric structures.
• It provides evidence that models develop linear feature representations for safety, consistent with the linear representation hypothesis advanced in recent interpretability work.
• It means that relatively simple probing methods can diagnose a model’s safety posture on a per-concept basis.

Transferability Within Model Families#

Concept-specific vectors extracted from one model in a family (e.g., Llama-3-8B) show moderate transferability to other models in the same family (e.g., Llama-3-70B), particularly for well-defined concepts like violence and sexual content. However, cross-family transfer is weaker, suggesting that different alignment procedures produce conceptually similar but geometrically distinct refusal representations.

Comparison with Global Refusal Vector Approaches#

The paper provides a detailed comparison between RepIt’s concept-specific steering and prior global refusal vector methods:

For Red Teaming and Adversarial Evaluation#

Concept-specific refusal vectors provide red teams with a powerful new tool for probing model vulnerabilities:

• Targeted safety auditing. Instead of testing a model’s overall safety, auditors can now probe individual safety concepts independently, identifying specific categories where the model’s refusal representations are weakest.
• Per-concept robustness metrics. The minimum $\alpha$ required to ablate refusal for each concept provides a natural robustness score, enabling fine-grained safety leaderboards.
• Adaptive attack potential. While concept-specific ablation requires more knowledge (separate vectors per concept), it also produces more surgical jailbreaks that are harder to detect because the model remains safety-compliant on non-targeted categories.

For Defence and Alignment#

On the defensive side, the discovery of concept-specific refusal vectors suggests several strategies:

• Redundant refusal representations. If safety can be encoded in a single vector per concept, it can also be encoded redundantly across multiple layers and subspaces. Training models to develop multiple, independent refusal representations per concept would make ablation attacks significantly harder.
• Concept coupling. Deliberately entangling refusal vectors for different concepts—so that ablation of one degrades others—would increase the cost of targeted attacks. This represents a deliberate move away from the independence that RepIt reveals.
• Detection of steering. Monitoring activation norms and directions during inference could detect when steering vectors are being applied, enabling runtime defences against activation manipulation attacks.
• Improved alignment training. Understanding that safety is represented per-concept allows alignment procedures to explicitly target under-represented categories, ensuring uniform robustness across all safety domains.

For Mechanistic Interpretability#

RepIt contributes to the broader mechanistic interpretability research programme by providing evidence that:

• Safety concepts are represented linearly in activation space, supporting the linear representation hypothesis.
• These representations are decomposable—they can be independently manipulated without catastrophic interference.
• The geometry of safety representations provides a useful predictor of model behaviour, enabling proactive safety evaluation without requiring extensive generation-based testing.