# Google Gemini explained: models, the 59.4% MMMU score and business use > Google Gemini is Google DeepMind's family of multimodal AI models. What each version added, what its 59.4% MMMU score meant and how businesses use it. - URL: https://computese.com/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/ - Author: Duong Quan Nguyen, CEO, Computese - Published: 2023-12-09 - Updated: 2026-09-25 - Topics: AI & automation ## In short - Google Gemini is Google DeepMind's family of multimodal AI models. It launched on December 6, 2023 as Gemini 1.0 Ultra, Pro and Nano; as of September 2026 the newest stable model is Gemini 3.8 Flash. - The 59.4% is Gemini 1.0 Ultra's zero-shot, one-answer score on MMMU's 900 validation questions, ahead of GPT-4V's 56.8%. With 32 sampled answers and a majority vote, it reached 62.4%. - Human experts scored 76.2% to 88.6% on the same questions, so 59.4% was a state-of-the-art result for a model in December 2023, not expert-level performance. - Benchmark scores move with the prompting method, training-data contamination and shortcuts: the harder MMMU-Pro cut model scores by 16.8 to 26.9 points. Test candidate models on your own tasks. - Businesses reach Gemini through Workspace and the Gemini app, Google AI Studio and the Gemini API, Gemini Enterprise Agent Platform (formerly Vertex AI) and Gemini Nano on Android. The free API tier has different data terms. Google Gemini is a family of multimodal AI models from Google DeepMind, first announced on December 6, 2023 in three sizes: Ultra, Pro and Nano. The 59.4% figure is Gemini 1.0 Ultra's launch score on MMMU, a benchmark of college-level questions that mix text and images; the previous best, GPT-4V, had 56.8%. Much has changed since that launch. Gemini is now also the name of Google's assistant app (called [Bard until February 2024](https://blog.google/products-and-platforms/products/gemini/bard-gemini-advanced-app/)), and the models have gone through generations 1.5, 2.0, 2.5 and 3, followed by a quick run of 3.x releases. As of September 2026 the newest stable model is Gemini 3.8 Flash. This explainer dates each generation, explains what MMMU tests and what the 59.4% did and did not prove, shows why a benchmark score is a weak basis for choosing a model, and covers the four ways businesses actually use Gemini. How language, image and multimodal models work in general is covered in [what generative AI is](https://computese.com/the-rise-of-generative-ai/). ## What Google Gemini is "Gemini" names three related things, which is why searches for it return such mixed results: - **The models.** A family of large multimodal models trained by Google DeepMind. Each generation (1.0, 1.5, 2.0, 2.5, 3 and its point releases) comes in tiers such as Pro, Flash and Flash-Lite. - **The Gemini app.** Google's assistant on the web and on phones. It launched as Bard in 2023 and was renamed Gemini on February 8, 2024. - **Gemini inside other Google products.** Gmail, Docs, Sheets and Meet in Google Workspace; AI Mode in Google Search, which [got Gemini 3 on the model's launch day](https://blog.google/products-and-platforms/products/gemini/gemini-3/); and Android features that run Gemini Nano on the device. How Search itself finds and ranks pages is a separate system, covered in [how the Google search engine works](https://computese.com/how-google-search-engine-works/). Multimodality is the defining design choice. [Google says](https://blog.google/innovation-and-ai/technology/ai/google-gemini-ai/) it pre-trained Gemini on different kinds of data from the start, instead of training separate components for images or audio and stitching them together afterwards. The [Gemini 1.0 technical report](https://arxiv.org/abs/2312.11805) describes Transformer decoders trained on Google's TPU accelerators with a context window of 32,768 tokens, able to take text interleaved with photos, charts, screenshots, PDFs, audio and video. The first generation came in three sizes: | Gemini 1.0 size | Designed for | Detail from Google's technical report | | --------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------- | | Ultra | Highly complex tasks | The largest model, trained on a large fleet of TPUv4 accelerators | | Pro | Scaling across a wide range of tasks | Optimized for cost and latency; powered Bard from launch day | | Nano | On-device tasks | Two versions, 1.8 billion and 3.25 billion parameters, distilled from larger models and 4-bit quantized | ## Gemini versions and release dates Google has shipped a new generation roughly once a year, with point releases in between. The dates below come from Google's announcement posts and the [Gemini API release notes](https://ai.google.dev/gemini-api/docs/changelog). | Date | Release | What changed | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Dec 6, 2023 | Gemini 1.0 Ultra, Pro and Nano | The first Gemini family. Bard moved to a tuned Gemini Pro, the Pixel 8 Pro ran Nano, and developers got Gemini Pro through the Gemini API on December 13 | | Feb 8, 2024 | Bard renamed Gemini | Ultra 1.0 opened to the public through the paid Gemini Advanced tier | | Feb 15, 2024 | [Gemini 1.5 Pro](https://blog.google/innovation-and-ai/products/google-gemini-next-generation-model-february-2024/) | A Mixture-of-Experts (MoE) architecture, quality comparable to 1.0 Ultra with less compute, and a 128,000-token context window (1 million in a private preview) | | May 2024 | Gemini 1.5 Flash | A faster, lower-cost tier: preview on May 10, generally available on May 23 | | Dec 11, 2024 | [Gemini 2.0 Flash (experimental)](https://blog.google/innovation-and-ai/models-and-research/google-deepmind/google-gemini-ai-update-december-2024/) | Aimed at agents: native image and audio output and native tool use. Generally available on February 5, 2025 | | Mar 25, 2025 | [Gemini 2.5 Pro (experimental)](https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-model-thinking-updates-march-2025/) | A thinking model that reasons through its thoughts before answering. Stable on June 17, 2025 | | Nov 18, 2025 | Gemini 3 Pro (preview) | Shipped the same day in AI Mode in Search, the Gemini app, AI Studio, Vertex AI and Google Antigravity, Google's agentic development platform | | Dec 17, 2025 | Gemini 3 Flash (preview) | The fast, lower-cost tier of generation 3 | | Feb 19, 2026 | Gemini 3.1 Pro (preview) | Replaced Gemini 3 Pro Preview, which shut down on March 9, 2026 | | May 19, 2026 | Gemini 3.5 Flash | Generally available, aimed at agentic and coding tasks | | Jul 21 to Sep 2, 2026 | Gemini 3.6, 3.7 and 3.8 Flash | Three Flash releases in six weeks; 3.8 Flash arrived with 3.8 Flash Cyber, a security variant for trusted defenders | The names follow a pattern. The number is the generation and the word is the tier: Pro is aimed at the hardest problems, Flash is the fast, lower-cost workhorse, Flash-Lite is the cheapest option for high volume, and Nano runs on the device. Ultra was a model size in Gemini 1.0. As of September 2026 no Ultra model appears on the Gemini API's model list, and the word survives as Google AI Ultra, a consumer subscription plan. ### The Gemini lineup as of September 2026 According to Google's [model list for the Gemini API](https://ai.google.dev/gemini-api/docs/models) (last updated September 24, 2026): - **Gemini 3.8 Flash** is the newest stable model, generally available since September 2, 2026. [Google calls it](https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/) its most intelligent workhorse model, built for long-horizon coding and autonomous agents. Consumers get it with Google AI Pro and Ultra subscriptions in the Gemini app, AI Mode in Search and Gemini in Google Sheets. - **Gemini 3.7, 3.6 and 3.5 Flash** remain available as stable models; Google now calls 3.5 Flash its legacy Flash model. - **Gemini 3.5 Flash-Lite and 3.1 Flash-Lite** are the low-cost, high-throughput tier. - **Gemini 3.1 Pro** is still labelled preview in the Gemini API. - **Gemini 2.5 Pro, Flash and Flash-Lite** are still served, but since September 18, 2026 only to users who have used them before. Google points new projects to 3.5 Flash-Lite or 3.8 Flash. - **Retired:** the Gemini 1.5 models shut down in the API on September 29, 2025, and Gemini 2.0 Flash on June 1, 2026. The brand also covers specialized models: the Nano Banana image models (Gemini 3.1 Flash Image and Gemini 3 Pro Image), Gemini 3.8 Live for voice agents, Gemini 3.5 Transcribe for speech-to-text, and Gemini Embedding for search and retrieval. ## What the MMMU benchmark measures MMMU stands for Massive Multi-discipline Multimodal Understanding and Reasoning. The MMMU team, led by first author Xiang Yue, [published it](https://arxiv.org/abs/2311.16502) in November 2023, and it was presented at CVPR 2024. It tests whether a model can answer the kind of question a university student meets, where the answer depends on reading an image. Whether AI helps students learn is a separate question, covered in [AI in education](https://computese.com/ai-in-education-learning-and-beyond/). From the paper: - **11,550 questions** taken from college exams, quizzes and textbooks, collected by more than 50 college students. - **Six disciplines, 30 subjects and 183 subfields:** Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. - **30 image types**, including charts, diagrams, maps, tables, music sheets, chemical structures and medical images. 97.5% of questions carry an image. - **Mostly multiple choice:** 94% of questions, with the rest open answers. - **Three splits:** 150 development questions for few-shot examples, 900 validation questions and 10,500 test questions. Models are tested zero-shot: no solved examples in the prompt and no fine-tuning on the benchmark. Scoring is mechanical. A rule-based pipeline pulls the final choice or number out of the model's response and compares it with the answer key, and the score is plain accuracy across all questions (micro-averaged). If no valid answer can be found, a multiple-choice question gets a random choice. The reasoning steps themselves are not graded. ![An exam card holds a small chart, lines of a question and four blank answer boxes. A chip reads the card, drawing on an open textbook, and picks one box, shown in orange, which an answer key confirms.](https://computese.com/images/blog/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/mmmu-question.bab3b58b3e-1536.webp) *Each MMMU question pairs an image with college-level knowledge, and only the final choice is scored, never the reasoning behind it.* Two baselines put any score in context. Picking a random option scores 22.1% on the validation set, and always picking the most frequent option in each subject scores 26.8%. At the other end, the authors asked 90 senior college students, three per subject, to answer the 900 validation questions with their textbooks open but no internet. The lowest, median and best expert results were 76.2%, 82.6% and 88.6%. ### MMMU is not MMLU Google's launch post quoted both benchmarks, and the acronyms differ by one letter, so they are easy to mix up: | | MMLU | MMMU | | -------------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------- | | Full name | [Massive Multitask Language Understanding](https://arxiv.org/abs/2009.03300) | Massive Multi-discipline Multimodal Understanding and Reasoning | | Published | September 2020 | November 2023 | | Input | Text only | Text with one or more images | | Scope | 57 tasks, from elementary mathematics and US history to law | 11,550 questions across 30 college subjects | | Gemini 1.0 Ultra at launch | 90.0% (32 chain-of-thought samples) | 59.4% (one answer per question) | | Human reference | 89.8% expert level, as gauged by the benchmark's authors | 76.2% to 88.6% (senior college students) | ## What Gemini Ultra's 59.4% MMMU score means The number comes from Table 7 of Google's Gemini 1.0 technical report. Read with its conditions, it says something precise: Gemini Ultra answered 59.4% of MMMU's 900 validation questions correctly, zero-shot, giving one answer per question (pass@1) with greedy decoding, and reading each image from its pixels alone, without an OCR tool extracting the text first. The same table reports a second number, 62.4%, under a different method called Maj1@32: the model samples 32 answers per question and the most common one is graded. The report's introduction leads with 62.4%, while Google's launch blog quoted 59.4%. Both are correct; they measure different things, which is why both still circulate. ![A chip gives one answer card in the top row. In the bottom row it produces many small answer cards marked with shapes, a ballot box collects them, and the most common shape comes out as one orange card.](https://computese.com/images/blog/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/majority-vote.360e1d39cf-1536.webp) *Sampling 32 answers and taking the majority lifted Gemini Ultra from 59.4% to 62.4%: always check which method a score used.* | Model (December 2023) | MMMU validation accuracy | | ------------------------------------------ | ------------------------ | | Random choice (baseline) | 22.1% | | Gemini 1.0 Nano-1 | 26.3% | | Gemini 1.0 Nano-2 | 32.6% | | Gemini 1.0 Pro | 47.9% | | GPT-4V (previous best) | 56.8% | | Gemini 1.0 Ultra, one answer (pass@1) | 59.4% | | Gemini 1.0 Ultra, majority of 32 (Maj1@32) | 62.4% | | Human experts, lowest to best | 76.2% to 88.6% | By discipline, the picture is mixed. With 32-sample voting, Ultra beat GPT-4V in five of six disciplines, all but Science (49.3% against 54.7%). With a single answer, GPT-4V stayed ahead in Business (59.3% against 56.7%) as well as Science (54.7% against 48.0%), while Ultra's widest lead was in Tech & Engineering (47.1% against 36.7%). So what did 59.4% show? That in December 2023 Gemini Ultra was the strongest published model on college-level questions that depend on an image, a few points ahead of GPT-4V, and that it got there without an OCR step. What it did not show is expert performance. The score sat about 17 points below the lowest human expert result, and the model still missed roughly four questions in ten. ### The 90.0% MMLU figure needs the same care Google's launch post said Gemini Ultra was the first model to outperform human experts on MMLU, with 90.0%. The technical report shows how. That score used what Google calls uncertainty-routed chain-of-thought: the model writes 32 reasoning samples and takes the consensus answer if agreement passes a threshold, falling back to a greedy answer otherwise. Under the standard 5-shot setting, Gemini Ultra scored 83.7%, below the 86.4% reported for GPT-4. With plain greedy decoding the two were level: 84.0% for Ultra and 84.2% for GPT-4. ## Why benchmark scores are not the whole story A benchmark is a fixed test with a known answer format. That makes it useful for comparing models and weak at predicting how a model will do on your work. Gemini's launch numbers show four reasons. 1. **The method changes the number.** One answer or the majority of 32, five examples or chain-of-thought: the same model scored 59.4% or 62.4% on MMMU, and 83.7% or 90.0% on MMLU. Compare scores only when the settings match. 2. **Test questions leak into training data.** Google's own report warns that benchmark results may be affected by data contamination. In one experiment, a hundred extra fine-tuning steps on web extracts matching the HellaSwag benchmark's training set raised Gemini Ultra's validation accuracy to 96.0%, and after its leak analysis Google chose not to report some results, such as LAMBADA. MMMU's test answers, scored through an evaluation server since December 2023, were [published on February 12, 2026](https://mmmu-benchmark.github.io/) so anyone can score locally, which makes contamination checks on newer results more important. 3. **Models find shortcuts.** In September 2024 the MMMU team released [MMMU-Pro](https://arxiv.org/abs/2409.02813) after finding that text-only models could answer some MMMU questions without seeing the image, using prior knowledge or patterns in the answer options. MMMU-Pro removes those questions, raises the options from four to as many as ten and adds a vision-only setting where the whole question arrives as a screenshot or photo. Model scores fell by 16.8 to 26.9 points, depending on the model. ![A chip that cannot see images still reaches the right answer on an exam card whose picture is covered. On a harder card shown as a screenshot with ten options, its arrow stops at a closed gate.](https://computese.com/images/blog/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/text-shortcut.43cfb5bd9d-1536.webp) *A question a model can answer without its image measures text knowledge, not multimodal understanding, so MMMU-Pro removed them.* 4. **Benchmarks age.** Each generation launches against newer, harder tests. In November 2025 [Google reported 81%](https://blog.google/products-and-platforms/products/gemini/gemini-3/) for Gemini 3 Pro on MMMU-Pro, a different test whose scores cannot be set against the 59.4%. In September 2026 its post for Gemini 3.8 Flash led with software-engineering, finance and legal agent benchmarks instead. Benchmarks also leave out what a business pays for. Google notes that Gemini 3.8 Flash may use more tokens on complex tasks to maximize performance, especially at higher effort levels, so a better score can cost more per task. Latency, price, rate limits, where data is processed and how the model handles your own documents appear on no leaderboard. The reliable way to choose is a small evaluation set built from your own work: real questions, documents and expected answers, run against each candidate model, scored for accuracy, cost and speed, and run again whenever the model changes. Our [AI automation service](https://computese.com/services/ai-automation/) builds that evaluation set from your real cases, and a change of model, prompt or source goes live only after passing it. ## How businesses use Gemini There are four main routes. They differ most in where your data goes. | Route | What it is | Suited to | Watch for | | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | | Gemini in Google Workspace and the Gemini app | AI in Gmail, Docs, Sheets, Meet and Chat, plus the Gemini app, [included in Business and Enterprise plans](https://workspace.google.com/blog/product-announcements/empowering-businesses-with-AI) | Drafting and summarizing email and documents, meeting notes, research | It retrieves whatever each user has permission to open, so fix over-shared files first | | [Google AI Studio](https://ai.google.dev/gemini-api/docs/ai-studio-quickstart) and the Gemini API | A browser workspace for trying models and prompts, and the API your code calls | Prototypes, and features built into your own software | The free tier's data terms (below) | | [Gemini Enterprise Agent Platform](https://cloud.google.com/products/gemini-enterprise-agent-platform) (formerly Vertex AI) | Google Cloud's platform to build, govern and run models and agents | Production systems on Google Cloud | A Google Cloud project, access control and billing to set up | | [Gemini Nano on Android](https://developer.android.com/ai/gemini-nano) | An on-device model served by Android's AICore system service | Features that must work offline or keep data on the phone | Speed depends on the device's hardware | ### Gemini in Workspace and the Gemini app In January 2025 Google folded its Gemini features into the Workspace Business and Enterprise plans instead of selling them as an add-on. Gemini helps people summarize, draft and find information across their email, chats and files, and the Gemini app handles longer research and analysis. Because Gemini retrieves only data the user already has permission to access, the sharing settings on your drives decide what it can surface. At that time Google listed certifications for Gemini in Workspace and the Gemini app that included SOC 1/2/3, ISO 27001 and ISO 42001, which matters for regulated teams. ### Google AI Studio and the Gemini API Google AI Studio is where a developer tries a model and a prompt in the browser, then uses "Get code" to turn the result into an API call. As of September 2026, [Google's quickstart](https://ai.google.dev/gemini-api/docs/get-started) uses the Interactions API, launched on December 11, 2025; the older GenerateContent API is still supported. A minimal call with a pinned model looks like this: ```bash export GEMINI_API_KEY="YOUR_API_KEY" curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "gemini-3.8-flash", "input": "Summarize this support ticket in two sentences: ..." }' ``` > [!IMPORTANT] > Check which tier your API key is on before you send real data. Under the [Gemini API terms](https://ai.google.dev/gemini-api/terms), on the free tier (unpaid quota, and AI Studio without a billing account), Google may use prompts and responses to improve its products, human reviewers may read them, and the terms tell you not to submit sensitive, confidential or personal information. On the paid tier, through a Cloud project with active billing, Google does not use prompts or responses to improve its products and keeps logs for a limited period to detect abuse. In the European Economic Area, Switzerland and the UK, the paid-tier data terms apply to all use. ### Gemini Enterprise Agent Platform (formerly Vertex AI) In April 2026 Google Cloud [relaunched Vertex AI as Gemini Enterprise Agent Platform](https://cloud.google.com/blog/products/ai-machine-learning/introducing-gemini-enterprise-agent-platform) and said all Vertex AI services would be delivered through it from then on. It is the route for companies that want Gemini under Google Cloud's controls: more than 200 Google and third-party models and tools, with Anthropic's Claude models and Google's open Gemma models in Model Garden, tuning, a model evaluation service, and tools to build and govern agents. For a small team shipping one feature, the Gemini API is simpler; for a company already on Google Cloud, the platform keeps AI under the same projects, access control and billing as everything else. ### Gemini Nano on Android Gemini Nano runs on the phone itself through Android's AICore system service, which keeps the model up to date and applies safety filters. Apps reach it through the ML Kit GenAI APIs for summarization, proofreading, rewriting, image description, speech recognition and custom prompts. Prompts are processed locally and no network connection is needed, which suits private or offline features; the trade-off is that speed depends on the device's hardware. ## How to choose a Gemini model and plan for change The release pace is the practical problem. 2026 alone brought a new Pro preview, new Flash models in May, July, August and September, and the shutdown of Gemini 2.0 Flash. A few habits keep that pace from breaking your product: 1. **Write down the task and its limits first:** the inputs (text, scans, audio or video), the latency you can accept, the monthly volume, and whether the data may leave your systems. 2. **Choose the route by where the data lives.** Content already in Workspace suits Gemini in Workspace; features in your own product suit the paid Gemini API or Agent Platform; private or offline features on phones suit Gemini Nano. 3. **Start with the current stable Flash model,** then test Flash-Lite for cost and Pro for the hardest cases. Keep the cheapest model that passes your evaluation set. 4. **Pin an exact stable model name,** such as `gemini-3.8-flash`, in production. Google says stable models usually don't change. A `latest` alias such as `gemini-flash-latest` is switched to each new release, with two weeks' email notice for breaking changes, and preview models can be deprecated with as little as two weeks' notice. 5. **Watch the [deprecations page](https://ai.google.dev/gemini-api/docs/deprecations) and the release notes.** The shutdown dates listed there are the earliest possible dates. Gemini 2.0 Flash, for example, was announced for shutdown on February 18, 2026 and switched off on June 1, 2026. 6. **Rerun your evaluation set on every model change,** and compare accuracy, cost per task and latency with the model you run today. If the model will answer from your company's records, the data behind it matters as much as the model. Our [data platform service](https://computese.com/services/data-platform/) builds the pipelines and governed warehouse or lakehouse that feed dashboards, APIs and AI, and our [AI automation service](https://computese.com/services/ai-automation/) puts a model into your workflows with human approval, evaluations and audit logs. ## Key terms - **Multimodal model**: A model that accepts more than one kind of input, such as text, images, audio and video, in the same prompt. Google says it pre-trained Gemini on several modalities from the start. - **MMMU**: Massive Multi-discipline Multimodal Understanding and Reasoning: a benchmark of 11,550 college-level questions across 30 subjects, nearly all of which include an image such as a chart, diagram or chemical structure. - **MMLU**: Massive Multitask Language Understanding: a text-only multiple-choice test covering 57 subjects, from elementary mathematics to law, published in 2020. - **Zero-shot**: Testing a model with no solved examples in the prompt and no fine-tuning on the benchmark. In a 5-shot test, five solved examples are shown first. - **pass@1**: Accuracy when the model gives one answer per question and that single answer is graded. - **Maj1@32 (majority voting)**: The model samples 32 answers per question and the most common one is graded. It usually scores higher than pass@1 but needs 32 generations per question instead of one. - **Chain-of-thought**: Prompting a model to write out intermediate reasoning before its final answer. Google's 90.0% MMLU score used 32 chain-of-thought samples with a confidence threshold. - **Data contamination**: Benchmark questions or answers appearing in a model's training data, which inflates its score on that benchmark without making it more capable. - **Context window**: How much input, counted in tokens, a model can take in at once. Gemini 1.0 was trained on 32,768-token sequences; Gemini 1.5 Pro offered 128,000 tokens as standard. - **Mixture-of-Experts (MoE)**: An architecture that splits a model into smaller expert networks and activates only the relevant ones for each input, making it cheaper to train and serve. Gemini adopted it with version 1.5. ## Common questions ### What was Google Gemini's MMMU score? Gemini 1.0 Ultra scored 59.4% on the MMMU validation set at its December 2023 launch, answering once per question and reading images without an OCR tool. That was ahead of GPT-4V's 56.8%. With 32 sampled answers and a majority vote, the same model reached 62.4%; Gemini 1.0 Pro scored 47.9%. ### What is the difference between MMMU and MMLU? MMLU (2020) is a text-only test across 57 subjects, where Gemini Ultra's launch score was 90.0%. MMMU (2023) has 11,550 college-level questions built around images such as charts, diagrams and chemical structures, where Gemini Ultra scored 59.4%. The two numbers measure different things and cannot be compared. ### Did Gemini Ultra beat human experts? On MMLU, Google reported 90.0% against a human-expert level of 89.8%, using 32 chain-of-thought samples; with the standard 5-shot method it scored 83.7%. On MMMU it did not come close: human experts scored between 76.2% and 88.6% on the validation questions, against Ultra's 59.4%. ### Is Google Gemini the same as Bard? Bard was Google's chatbot, launched in 2023. It switched to a tuned Gemini Pro model on December 6, 2023, and was renamed Gemini on February 8, 2024, when Google also launched Gemini Advanced with the Ultra 1.0 model. ### What is the latest Gemini model? As of September 2026, the newest stable model is Gemini 3.8 Flash, generally available since September 2, 2026. The Pro tier is Gemini 3.1 Pro, still labelled preview in the Gemini API. Google releases often, so check the Gemini API model list before you pin a version. ### Does Google use the data I send to Gemini? It depends on the route. On the Gemini API's free tier and in Google AI Studio without billing, Google may use prompts and responses to improve its products, and human reviewers may read them. On the paid tier it does not use them for that. In Workspace, Google says your existing data security controls apply to Gemini automatically. ## Sources 1. [Introducing Gemini: Google's most capable AI model yet](https://blog.google/innovation-and-ai/technology/ai/google-gemini-ai/), Google 2. [Gemini: A Family of Highly Capable Multimodal Models](https://arxiv.org/abs/2312.11805), Gemini Team, Google (arXiv) 3. [Google Bard is now Gemini: How to try Ultra 1.0 and new mobile app](https://blog.google/products-and-platforms/products/gemini/bard-gemini-advanced-app/), Google 4. [Introducing Gemini 1.5, Google's next-generation AI model](https://blog.google/innovation-and-ai/products/google-gemini-next-generation-model-february-2024/), Google 5. [Google introduces Gemini 2.0: A new AI model for the agentic era](https://blog.google/innovation-and-ai/models-and-research/google-deepmind/google-gemini-ai-update-december-2024/), Google 6. [Gemini 2.5: Our newest Gemini model with thinking](https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-model-thinking-updates-march-2025/), Google 7. [Gemini 3: Introducing the latest Gemini AI model from Google](https://blog.google/products-and-platforms/products/gemini/gemini-3/), Google 8. [Gemini API release notes](https://ai.google.dev/gemini-api/docs/changelog), Google AI for Developers 9. [Introducing Gemini 3.8 Flash and 3.8 Flash Cyber](https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/), Google 10. [Gemini models](https://ai.google.dev/gemini-api/docs/models), Google AI for Developers 11. [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://arxiv.org/abs/2311.16502), MMMU Team (arXiv, CVPR 2024) 12. [MMMU benchmark and leaderboard](https://mmmu-benchmark.github.io/), MMMU Team 13. [Measuring Massive Multitask Language Understanding](https://arxiv.org/abs/2009.03300), Hendrycks et al. (arXiv, ICLR 2021) 14. [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://arxiv.org/abs/2409.02813), MMMU Team (arXiv, ACL 2025) 15. [Google Workspace enables the future of AI-powered work for every business](https://workspace.google.com/blog/product-announcements/empowering-businesses-with-AI), Google Workspace 16. [Google AI Studio quickstart](https://ai.google.dev/gemini-api/docs/ai-studio-quickstart), Google AI for Developers 17. [Getting started with the Gemini API](https://ai.google.dev/gemini-api/docs/get-started), Google AI for Developers 18. [Gemini API Additional Terms of Service](https://ai.google.dev/gemini-api/terms), Google AI for Developers 19. [Introducing Gemini Enterprise Agent Platform](https://cloud.google.com/blog/products/ai-machine-learning/introducing-gemini-enterprise-agent-platform), Google Cloud 20. [Gemini Enterprise Agent Platform (formerly Vertex AI)](https://cloud.google.com/products/gemini-enterprise-agent-platform), Google Cloud 21. [Gemini Nano](https://developer.android.com/ai/gemini-nano), Android Developers 22. [Gemini deprecations](https://ai.google.dev/gemini-api/docs/deprecations), Google AI for Developers