Cross-Modal Attack Surfaces#

The most significant new threat in multimodal models is the cross-modal attack surface. In a unimodal text model, the attack surface consists of the text input. In a unimodal vision model, the attack surface consists of the image input. But in a multimodal model, the attack surface includes:

• Text input: Traditional text-based jailbreaks and adversarial prompts.
• Image input: Adversarial images designed to manipulate model behavior.
• Text-image interaction: The combination of text and image inputs, which can produce emergent behaviors not triggered by either input alone.
• Image-embedded text: Text rendered within images, which may bypass text-based safety filters while being processed by the model’s visual pipeline.
• Cross-modal inconsistencies: Deliberate mismatches between the text prompt and the visual content that confuse the model’s safety mechanisms.

Emergent Unsafe Behaviors#

Multimodal models can exhibit unsafe behaviors that emerge from the interaction of modalities rather than from any single modality in isolation. For example:

• A model might refuse to describe a violent image when asked directly in text, but comply when the request is encoded in an image caption or embedded text within the image.
• A model might correctly identify harmful content in an image but fail to flag it when the accompanying text frames the request as an educational or analytical query.
• A model’s visual processing might be exploited to inject instructions that override its text-based safety training — a form of visual prompt injection that has no analog in text-only systems.

Inadequate Transfer of Unimodal Safety#

Many multimodal models are built by combining a pre-trained language model with a vision encoder, connected through a projection layer or cross-attention mechanism. The safety training applied to the language model component (typically through RLHF or constitutional AI methods) may not transfer effectively to the multimodal setting. The vision encoder can process information that the safety-trained language model was never trained to handle, creating gaps in the model’s safety coverage.

Five Trustworthiness Dimensions#

Dataset Design#

MMDT uses a carefully constructed evaluation dataset that covers diverse scenarios across all five trustworthiness dimensions. The dataset includes:

Adversarial Text-Image Pairs: Pairs of text prompts and images specifically designed to test each trustworthiness dimension. These are crafted through a combination of automated generation, manual curation, and red-team testing.

Real-World Scenarios: Evaluation scenarios derived from real-world use cases where multimodal model failures could have significant consequences — medical image analysis, content moderation, educational applications, and autonomous driving.

Stress Test Scenarios: Extreme cases designed to push models to their limits and reveal failure modes that might not appear in standard evaluation. These include ambiguous images, adversarial prompts, culturally complex scenarios, and ethically challenging situations.

Diverse Demographic Representation: Images and scenarios that represent a wide range of demographic groups, cultural contexts, and geographic regions to ensure that fairness evaluations capture bias across all relevant dimensions.

Evaluation Protocol#

The MMDT evaluation protocol is designed for rigor and reproducibility:

Automated Metrics: Where possible, trustworthiness is measured through automated metrics that can be computed at scale. These include toxicity scores, demographic parity metrics, hallucination rates, and privacy leakage measures.

LLM-as-Judge Evaluation: For dimensions where simple automated metrics are insufficient, a carefully calibrated LLM-based judge evaluates model outputs against trustworthiness criteria. The judge is validated against human evaluations to ensure reliability.

Human Evaluation: For the most complex and nuanced trustworthiness dimensions, structured human evaluation is conducted by trained annotators with diverse backgrounds and expertise.

Red-Team Evaluation: The benchmark includes an adversarial component where red-teamers actively attempt to elicit unsafe behavior from the models, simulating real-world attack scenarios.

Evaluated Models#

MMDT evaluates a comprehensive set of multimodal foundation models spanning different architectures, scales, and training approaches:

• Proprietary models: GPT-4o, Gemini Pro Vision, Claude 3.5 Sonnet
• Open-source models: LLaVA (various sizes), Qwen-VL, InternVL, IDEFICS, CogVLM
• Specialized models: Models fine-tuned for specific domains (medical imaging, document understanding)

This broad coverage enables meaningful comparison across the multimodal model landscape and identification of architecture-specific vulnerabilities.

Finding 1: Multimodal Models Are Significantly Less Safe Than Their Text-Only Counterparts#

The most striking finding is that multimodal models are substantially more vulnerable to safety attacks than equivalent text-only models. Safety guardrails that work effectively in text-only settings are frequently bypassed through cross-modal attacks. Specifically:

• Embedded text attacks succeed against the majority of evaluated models, with success rates ranging from 40% to over 80% depending on the model and attack strategy.
• Image-triggered jailbreaks that use adversarial or carefully selected images to override text-based safety training are effective against all evaluated models to varying degrees.
• Multimodal jailbreak chains that progressively erode safety through multi-turn interactions succeed more often than single-turn attacks, highlighting the vulnerability of multimodal models to sophisticated adversarial strategies.

Finding 2: Cross-Modal Hallucination Is a Pervasive Problem#

The truthfulness evaluation reveals that multimodal models frequently hallucinate in ways that are unique to the multimodal setting:

• Suggestion-induced hallucination: When text prompts suggest the presence of specific objects or features in an image, models often “see” things that are not there. This effect is stronger in multimodal models than in vision-only models, suggesting that the language model component’s tendency to complete patterns can override visual evidence.
• Detail confabulation: When asked to describe images in detail, models frequently fabricate specific details (colors, numbers, text, small objects) that sound plausible but are not present in the image. This confabulation is particularly concerning in applications like medical image analysis or forensic investigation where accuracy is critical.
• Overconfidence in uncertainty: Models rarely express appropriate uncertainty when visual evidence is ambiguous. Instead, they produce confident but incorrect descriptions, creating a false impression of reliability.

Finding 3: Fairness Gaps Are Amplified by Visual Context#

The fairness evaluation demonstrates that visual context can amplify existing biases and introduce new ones:

• Occupational stereotyping is more pronounced when models process images of people alongside occupational queries. The model’s description of a person’s likely profession is more strongly influenced by demographic characteristics visible in the image than by the text query alone.
• Action interpretation bias — where the same action is interpreted differently based on the demographic characteristics of the people in the image — is a significant concern across all evaluated models.
• Cross-cultural insensitivity is prevalent, with models defaulting to Western-centric interpretations of visual scenes from non-Western cultural contexts.

Finding 4: Robustness Varies Dramatically Across Models and Attack Types#

Robustness evaluation reveals a wide disparity among models:

• Proprietary models generally exhibit better robustness than open-source models, likely due to more extensive adversarial training and red-teaming during development.
• Typographic attacks — where misleading text labels are overlaid on images — are alarmingly effective, causing most models to override their visual understanding with the misleading textual information.
• Adversarial perturbations targeted at the visual input can cause dramatic changes in text output, revealing fundamental vulnerabilities in the vision-language connection.

Finding 5: Privacy Leakage Through Visual Processing Is Underappreciated#

The privacy evaluation uncovers a significant and underappreciated risk:

• Multimodal models can extract personally identifiable information from images with high accuracy, including names on badges, text on screens, license plate numbers, and addresses on envelopes.
• Cross-modal information aggregation attacks — where information from multiple images and text prompts is combined to infer private details — are effective against all evaluated models.
• Current safety training does not adequately prevent the model from revealing private information extracted from images when users explicitly ask for it.

For Model Developers#

The MMDT findings carry several actionable implications for teams building multimodal foundation models:

Invest in multimodal-specific safety training. Safety training that works for text models does not automatically transfer to multimodal settings. Dedicated safety training that addresses cross-modal attack vectors — embedded text, image-triggered jailbreaks, multimodal jailbreak chains — is essential.

Treat the vision-language interface as a critical security boundary. The projection layer or cross-attention mechanism that connects the vision encoder to the language model is a critical vulnerability point. Adversarial inputs to this interface can bypass safety training applied to either component in isolation.

Implement cross-modal consistency checking. Safety systems should verify that the model’s outputs are consistent with both the visual and textual inputs, flagging cases where the model may be confabulating or where cross-modal conflicts are being resolved unsafely.

Develop multimodal red-teaming capabilities. Standard text-based red-teaming is insufficient for multimodal models. Red-team exercises should systematically explore cross-modal attack vectors and evaluate the model’s behavior under adversarial multimodal inputs.

For Regulators and Policymakers#

The MMDT benchmark provides a framework for evaluating multimodal model safety that can inform regulatory standards:

Multimodal-specific safety standards. Regulations that address AI safety should include multimodal-specific provisions, recognizing that safety evaluations conducted on text-only versions of models are insufficient for multimodal deployments.

Mandatory safety evaluation. The finding that even the most capable proprietary models exhibit significant safety vulnerabilities suggests that mandatory safety evaluation using standardized benchmarks like MMDT should be a prerequisite for deployment in high-stakes applications.

Transparency requirements. Model developers should be required to disclose the results of multimodal safety evaluations, enabling informed deployment decisions by downstream users.

For the Research Community#

MMDT opens several important research directions:

Multimodal safety alignment. How can we develop alignment techniques that are specifically designed for the multimodal setting, addressing the unique challenges of cross-modal attacks and emergent unsafe behaviors?

Cross-modal adversarial training. How can we train models to be robust to adversarial inputs that exploit the interaction between modalities?

Privacy-preserving visual processing. How can we build multimodal models that can process visual information without extracting or revealing private information?

Hallucination mitigation for multimodal models. How can we reduce the tendency of multimodal models to confabulate visual details, especially when prompted to describe images in detail?

Evolving Model Capabilities#

Multimodal models are advancing rapidly, with new architectures, training techniques, and capabilities emerging regularly. The benchmark will need to evolve to capture new failure modes and attack vectors as models become more capable. The current evaluation captures the state of the art at the time of publication, but the safety landscape is dynamic.

Cultural and Linguistic Coverage#

While MMDT makes efforts to include diverse cultural contexts and languages, the coverage is inevitably incomplete. Expanding the benchmark to include more languages, cultural contexts, and region-specific safety concerns is an important priority for making the evaluation globally relevant.

Extension to Additional Modalities#

The current benchmark focuses on text-image multimodal models. As models incorporate additional modalities — audio, video, 3D, code, structured data — the benchmark will need to extend to cover these new modality combinations and their associated safety risks.

Dynamic and Adaptive Evaluation#

The current benchmark uses a static evaluation dataset. In practice, adversaries adapt their attacks based on model behavior. Developing dynamic evaluation protocols that simulate adaptive adversaries would provide a more realistic assessment of real-world safety.

Generative Multimodal Safety#

MMDT primarily evaluates multimodal understanding — models that process images and generate text. As multimodal generative models (those that generate images, video, or audio from multimodal inputs) become more capable, the benchmark will need to extend to cover the safety implications of multimodal generation.