Short answer
Multimodal AI systems work across media types, for example describing images, answering questions about PDFs, or generating pictures from text.
Why it matters
Real work mixes formats. Multimodal models reduce brittle pipelines that stitch separate vision and language tools by hand—though they introduce new error modes.
Article
Risk
A fluent text answer can hide a misread image. Require evidence citations or human checks for critical visual decisions.
Examples
Receipt Q&A
A user photographs a receipt; the model extracts totals and answers “what did I spend on tax?” with quoted fields.
FAQ
Is a text-to-image model multimodal?
Yes—it maps text to images. Vision-language chat models are multimodal in the understanding direction.
Sources
-
On the Opportunities and Risks of Foundation Models
Stanford CRFM / arXivFoundation models report from Stanford CRFM.
-
Google AI for Developers
GoogleOfficial Google Gemini / generative AI developer docs.
-
OpenAI API Documentation
OpenAIOfficial OpenAI platform documentation.