Decoding Compressed Trust: How LLM Compression Impacts Trustworthiness#

Why We Compress#

Modern LLMs are enormous. LLaMA-2 70B requires ~140 GB of memory in full precision (FP16). Even smaller variants like LLaMA-2 13B demand ~26 GB, putting them beyond the reach of consumer hardware and many edge deployments. Model compression techniques address this gap:

Quantization reduces the numerical precision of model weights and/or activations — from FP16 (16-bit floating point) to INT8, INT4, or even lower. This directly reduces memory footprint and can accelerate inference on hardware with integer arithmetic support. Techniques like GPTQ, AWQ, SmoothQuant, and LLM.int8() have made 4-bit quantization practical with minimal perplexity degradation.

Pruning removes redundant or less important model parameters — individual weights (unstructured pruning) or entire attention heads, neurons, or layers (structured pruning). SparseGPT, Wanda, and magnitude-based pruning can eliminate 50% or more of weights while maintaining competitive performance.

Knowledge Distillation transfers the capabilities of a large “teacher” model to a smaller “student” model by training the student to mimic the teacher’s output distribution. This produces inherently smaller models that retain much of the teacher’s knowledge, exemplified by models like Alpaca, Vicuna, and Orca.

The Safety Blind Spot#

The standard evaluation protocol for compression research focuses almost exclusively on task performance: perplexity on language modeling benchmarks, accuracy on MMLU, performance on GSM8K, and similar metrics. If the compressed model scores within a few percentage points of the original on these benchmarks, the compression is declared successful.

But task performance is only part of the story. As LLMs are deployed in increasingly sensitive domains — healthcare, finance, customer service, education — their trustworthiness becomes equally important. A model that answers trivia questions accurately but generates toxic content, leaks private information from its training data, or exhibits severe demographic bias is not suitable for production, regardless of its benchmark scores.

The critical question is: do compression techniques preserve the trustworthiness properties that were carefully instilled through alignment training (RLHF, SFT, constitutional AI)? Or does compression create a false sense of security — models that perform well on benchmarks but have silently degraded safety properties?

Multi-Dimensional Trustworthiness Evaluation#

The authors introduce a comprehensive evaluation framework that assesses compressed models across five key dimensions of trustworthiness, drawing on established safety and alignment benchmarks:

1. Toxicity: Does the compressed model generate harmful, offensive, or toxic content? Evaluated using benchmarks like ToxiGen and RealToxicityPrompts. This dimension tests whether compression preserves the model’s ability to refuse harmful requests and avoid generating toxic continuations.

2. Stereotype and Bias: Does the compressed model exhibit increased demographic bias or reinforce harmful stereotypes? Evaluated using BBQ (Bias Benchmark for QA) and similar benchmarks that test how model responses vary across demographic groups defined by gender, race, religion, and other protected attributes.

3. Out-of-Distribution (OOD) Robustness: How well does the compressed model handle inputs that differ from its training distribution — adversarial prompts, unusual formatting, or prompts from domains not well-represented in training data? Evaluated using datasets like OOD datasets that test generalization.

4. Fairness: Does the compressed model make equitable decisions across different demographic groups? This extends bias evaluation to decision-making contexts where the model’s outputs have direct consequences for users.

5. Privacy: Does the compressed model leak sensitive information from its training data? Evaluated using extraction attacks and memorization tests that probe whether the model can be induced to regurgitate personally identifiable information (PII) or other sensitive training data.

Compression Methods Studied#

The study covers a comprehensive set of compression techniques applied to popular open-source LLMs:

Models: LLaMA-2 (7B, 13B, 70B), Vicuna (7B, 13B), and other variants, providing coverage across different model scales and alignment approaches.

Quantization Methods:

• GPTQ: Post-training quantization using approximate second-order information to minimize layer-wise reconstruction error
• AWQ (Activation-Aware Weight Quantization): Protects salient weights identified by activation magnitudes during quantization
• SmoothQuant: Migrates quantization difficulty from activations to weights using a mathematically equivalent transformation
• Round-to-Nearest (RTN): Naive quantization baseline that simply rounds weights to the nearest quantization level

Quantization Levels: INT8, INT4, INT3, and in some cases INT2, providing a spectrum from aggressive to mild compression.

Pruning Methods:

• SparseGPT: Post-training pruning that solves a layer-wise sparse regression problem to minimize reconstruction error
• Wanda (Weights and Activations): Prunes weights based on the product of weight magnitude and input activation norms
• Magnitude Pruning: Classic baseline that removes the smallest-magnitude weights

Pruning Ratios: 20%, 50%, and in some cases up to 70% sparsity.

Knowledge Distillation: Various teacher-student configurations where smaller models are trained to mimic larger aligned models.

Evaluation Methodology#

The evaluation follows a rigorous protocol:

1. Baseline Establishment: The uncompressed, aligned model (e.g., LLaMA-2-Chat 7B) serves as the trustworthiness baseline.
2. Compression Application: Each compression technique is applied at various severity levels.
3. Multi-Dimensional Evaluation: The compressed model is evaluated across all five trustworthiness dimensions.
4. Comparison: Trustworthiness metrics of compressed models are compared against the uncompressed baseline to identify degradation.

This protocol ensures that any trustworthiness changes can be attributed to the compression process rather than other confounding factors.

Finding 1: Compression Degrades Trustworthiness — Often Disproportionately#

The most striking finding is that trustworthiness degradation often exceeds task performance degradation. A compression technique that causes a 2% drop in MMLU accuracy might cause a 15–30% increase in toxicity or a significant rise in stereotype bias. This asymmetry is deeply concerning because it means that standard benchmark evaluations systematically underestimate the safety impact of compression.

For example, when LLaMA-2-Chat 7B is quantized to INT4 using GPTQ:

• MMLU accuracy drops by a modest 1–3 percentage points
• Toxicity scores can increase by 20–40% relative to the uncompressed baseline
• OOD robustness degrades significantly, with the model becoming more susceptible to adversarial prompts

This disproportionate safety degradation suggests that the model components responsible for trustworthiness (refusal behaviors, bias mitigation) are more sensitive to numerical perturbation than the components responsible for factual knowledge and reasoning.

Finding 2: Different Compression Methods Affect Different Trustworthiness Dimensions#

The study reveals that no single compression technique is uniformly safest across all trustworthiness dimensions. Different methods have different safety profiles:

Quantization tends to most severely affect:

• Toxicity guardrails: The model becomes more willing to generate toxic content, suggesting that the precise numerical representations encoding “refusal” behaviors are disrupted by reduced precision
• OOD robustustness: The model’s ability to handle unusual or adversarial inputs degrades, possibly because the margin of safety in the model’s decision boundaries is reduced

Pruning tends to most severely affect:

• Fairness: Removing neurons or attention heads can disproportionately affect the model’s ability to make equitable decisions across demographic groups
• Privacy: Pruned models sometimes show increased memorization of training data, possibly because pruning reduces the model’s capacity for “benign generalization” and forces it to rely more on memorized examples

Knowledge Distillation presents a nuanced picture:

• Distilled models can inherit the teacher’s safety properties to some degree, but the transfer is imperfect
• The student model’s trustworthiness is bounded by the teacher’s trustworthiness but often falls short, particularly in nuanced safety scenarios that require sophisticated reasoning about harm

Finding 3: More Aggressive Compression Leads to More Trustworthiness Degradation — But Not Linearly#

As expected, more aggressive compression (lower bit-width quantization, higher pruning ratios) generally leads to worse trustworthiness. However, the relationship is nonlinear and sometimes exhibits threshold effects:

• INT8 quantization often preserves trustworthiness reasonably well, with degradation that is modest and proportional to task performance loss
• INT4 quantization represents a critical threshold where trustworthiness degradation accelerates significantly, even for methods like AWQ and GPTQ that are designed to minimize performance loss
• INT3 and below leads to severe trustworthiness degradation across most dimensions, making these levels unsuitable for safety-critical applications

For pruning, similar thresholds exist: 20–30% sparsity is often tolerable, but beyond 50% sparsity, trustworthiness degrades rapidly and unpredictably.

Finding 4: Larger Models Are More Robust to Compression-Induced Trustworthiness Degradation#

An important and somewhat counterintuitive finding is that larger models retain trustworthiness better under compression than smaller models. When LLaMA-2-Chat 70B is quantized to INT4, the relative trustworthiness degradation is less severe than when LLaMA-2-Chat 7B is quantized to the same level.

This suggests that larger models have redundant capacity for safety-related behaviors — the alignment and safety training is distributed across more parameters, providing resilience when some of those parameters are perturbed or removed. Smaller models, with less redundant capacity, are more fragile.

This finding has practical implications: for safety-critical deployments, it may be preferable to compress a large model moderately than to use a natively small model or aggressively compress a medium-sized model.

Finding 5: Safety Alignment Is Fragile Under Compression#

Perhaps the most concerning finding is that safety alignment appears to be more fragile than task performance under compression. The alignment training (RLHF, SFT) that teaches models to refuse harmful requests, avoid toxic outputs, and treat demographic groups equitably seems to be encoded in a way that is particularly vulnerable to the numerical perturbations introduced by compression.

This fragility manifests in several ways:

1. Refusal degradation: Compressed models are more likely to comply with harmful requests that the uncompressed model correctly refuses
2. Increased toxicity: Even in benign contexts, compressed models produce more toxic language
3. Bias amplification: Compression can amplify existing biases in the model, leading to more stereotypical and unfair outputs
4. Privacy leakage: Compressed models may be more susceptible to prompting strategies that extract memorized training data

The authors hypothesize that this fragility arises because safety alignment often operates through subtle adjustments to the model’s output distribution — shifting probabilities away from harmful outputs and toward safe alternatives. These adjustments, which may involve small changes to many parameters across the model, are easily disrupted by the coarse approximations introduced by compression.

Finding 6: Quantization-Aware Methods Offer Some Mitigation#

The study finds that quantization methods that consider the model’s activation patterns (AWQ, SmoothQuant) tend to preserve trustworthiness better than naive methods (RTN) or purely weight-focused methods (GPTQ). This suggests that protecting the parameters most involved in processing safety-relevant inputs can mitigate some trustworthiness degradation.

However, even the best methods show non-trivial degradation at aggressive compression levels, indicating that post-hoc fixes alone are insufficient.

For Model Deployers#

If you are deploying compressed LLMs in production, this work has several actionable implications:

1. Don’t rely solely on task benchmarks: Standard benchmarks like MMLU and GSM8K do not capture trustworthiness degradation. You must evaluate compressed models on safety-specific benchmarks before deployment.

2. Prefer larger models with moderate compression over small models: A 70B model quantized to INT4 may be safer than a 7B model at INT8, even if both have similar memory footprints and task performance.

3. Use activation-aware compression methods: Methods like AWQ and SmoothQuant that account for activation patterns tend to preserve trustworthiness better than purely weight-focused approaches.

4. Establish trustworthiness budgets: Define acceptable thresholds for toxicity, bias, and other safety metrics before compression, and validate that compressed models meet these thresholds.

5. Consider safety fine-tuning after compression: The study’s findings suggest that applying additional safety training (even brief RLHF or safety-focused SFT) after compression could partially recover lost trustworthiness.

For the Research Community#

1. Trustworthiness-aware compression: Future compression methods should be designed to explicitly preserve trustworthiness properties, not just task performance. This could involve identifying and protecting safety-critical parameters during compression.

2. Safety-aware evaluation protocols: Compression papers should include trustworthiness evaluation alongside standard benchmarks as a mandatory part of their evaluation protocol.

3. Mechanistic understanding: More research is needed to understand why safety alignment is fragile under compression — which parameters, layers, and attention heads are most critical for maintaining trustworthiness, and how compression affects them specifically.

4. Compression-robust alignment: Developing alignment training procedures that produce safety behaviors more robust to subsequent compression could address the root cause of the problem.

The Geometry of Safety Degradation#

One of the paper’s most insightful contributions is its analysis of how compression degrades safety at a mechanistic level. By examining the internal representations of compressed and uncompressed models, the authors find that:

• Safety-related features in the model’s representation space (directions that distinguish harmful from safe content) are compressed less faithfully than general language features
• The decision boundaries between safe and unsafe outputs become noisier and less well-calibrated after compression
• Refusal subspaces — the directions in representation space associated with refusing harmful requests — are particularly sensitive to quantization-induced perturbation

This geometric analysis suggests that future compression methods could benefit from explicitly preserving the subspace structure of safety-related features, using techniques from representation engineering or contrastive learning to maintain the integrity of safety boundaries.

Cross-Dimensional Interactions#

The study also reveals interesting interactions between trustworthiness dimensions. For example:

• Models that show increased toxicity under compression also tend to show increased bias, suggesting common underlying mechanisms
• Privacy leakage and OOD robustness are correlated — models that are more easily confused by adversarial inputs are also more susceptible to memorization extraction attacks
• Fairness degradation is somewhat independent of toxicity and bias, suggesting that compression affects different aspects of safety through partially distinct mechanisms

These interactions have practical implications: a deployment that is primarily concerned about toxicity should also monitor for bias increases, as they tend to co-occur under compression.

The Knowledge Distillation Paradox#

Knowledge distillation presents a particularly interesting case. On one hand, distilling from a well-aligned teacher model should transfer safety properties to the student. On the other hand, the student model has inherently less capacity, and the distillation process may not perfectly capture the nuanced safety reasoning of the teacher.

The study finds that:

• Distilled models perform reasonably well on clear-cut safety cases (obviously harmful requests that the teacher refuses)
• Distilled models struggle with ambiguous or nuanced safety scenarios where the correct behavior requires sophisticated reasoning about context, intent, and potential harm
• The safety gap between teacher and student grows as the capacity difference between them increases

This suggests that knowledge distillation for safety-critical applications should include dedicated safety-focused distillation phases, where the student is specifically trained on safety-relevant examples with careful attention to edge cases and ambiguous scenarios.