What does the EU AI Act actually do?

It creates harmonized, enforceable rules across EU member states. The regulation's own text states that it ensures the free movement of AI-based goods and services cross-border, preventing member states from imposing their own restrictions unless the regulation explicitly allows it — a single-market mechanism, not merely a policy statement.

Its central device is risk tiering: obligations scale with the danger a system poses, from prohibited practices through high-risk requirements to lighter duties for general-purpose models. Because it is a regulation rather than a directive, it applies directly in every member state, which is what makes the EU the reference point whenever someone asks whether AI is regulated, restricted, or banned somewhere.

The cost of that enforceability is compliance overhead, borne longest by the smallest developers, and definitional drift — edge cases that courts and regulators will spend years resolving. A binding law must draw lines; every line becomes litigation.

General-purpose models got their own layer of duties in the final text — documentation, transparency, and copyright-related obligations that attach regardless of the use case a downstream deployer chooses. That choice is why foundation-model developers watch Brussels even when their customers, not they, sit in the high-risk tiers.

Enforcement is also layered: a dedicated AI Office inside the Commission coordinates, national authorities execute, and penalties scale with the severity class of the violation. In practice, the first years of any such regime are dominated by guidance documents more than fines.

How does the US approach differ?

By being voluntary at the federal baseline. NIST's AI Risk Management Framework, published by the US standards institute, is explicitly intended for voluntary use, to improve the ability of organizations to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems (NIST).

Voluntary does not mean unserious. The framework was built through a consensus-driven public process — requests for information, multiple draft versions, workshops — and is accompanied by a playbook and roadmap. It functions as shared vocabulary that procurement rules, contracts, and sector regulators increasingly reference: soft power hardening at the edges, without statutory penalties at the center.

Its structure matters for how it gets used. The framework separates its core from a playbook of suggested actions, letting organizations adopt the vocabulary without signing up to a fixed checklist. That modularity is why it travels well across industries that would never accept a single uniform procedure — and why its uptake, not its enforcement, is the metric of its success.

The practical difference shows up in enforcement. An EU high-risk system's failures can trigger legal consequences under the regulation; a US framework gap is a governance finding, not a violation, unless some other law — consumer protection, civil rights, sector regulation — happens to apply.

What role does the OECD standard play?

The connective tissue between jurisdictions. The OECD Recommendation on AI is the first intergovernmental standard on AI, with 47 adherents — countries committed to its principles for trustworthy, human-centered AI. Its definition of an AI system, a machine-based system that infers from inputs how to generate outputs influencing environments, has been copied into other frameworks, including the EU's.

The OECD layer matters because national rules only interoperate if they share concepts. When the EU, the US, and Asian regulators can point to a common definition and principle set, cross-border AI products get one compliance conversation instead of thirty contradictory ones. The principles are not enforceable; they are the grammar the enforceable rules are written in.

Its work programs extend the same grammar into practice — policy trackers, incident monitors, and reporting frameworks for advanced AI developers — which keeps the standard updating faster than treaty-level law could move. For smaller countries without the capacity to write their own AI rules from scratch, adherence is a shortcut to a credible position; that is a quiet but real form of regulatory influence.

Which regime governs a product you use?

Follow the market, not the headquarters. An AI product sold into the EU meets the AI Act regardless of where its developer sits; the same product in the US meets NIST-style expectations mostly through procurement and sector rules; the OECD layer describes what all participating governments have agreed AI should look like.

For teams building AI tools, the working sequence is practical:

1. Map markets: EU exposure triggers the regulation's tiering analysis first.
2. Adopt the vocabulary: NIST's categories and the OECD definition translate across regimes.
3. Document choices: every framework rewards evidence of considered risk decisions over assertions of good intent.

Multi-market products therefore converge on the strictest applicable tier as their engineering baseline — a de facto Brussels-effect dynamic that makes EU compliance architecture useful everywhere.

Where the approaches may still diverge is speed. A regulation changes by amendment; a framework changes by revision and adoption; an intergovernmental standard changes by consensus of adherents. The voluntary instruments will track new model classes faster, while the binding one delivers what only law can — consequences that do not depend on goodwill.

Where do the approaches converge?

On process. All three regimes, despite their different legal force, ask the same underlying questions: what is the system's purpose, what risks follow from that purpose, who is accountable, and what evidence documents the answers. A team that can answer those questions well is, almost incidentally, compliant-adjacent in every regime; a team that cannot will fail the voluntary frameworks as surely as the binding one.

They also converge on lifecycle thinking. Each instrument treats AI governance as continuous — design, test, deploy, monitor — rather than a one-time certification, which is a meaningful shared conclusion given how differently the three bodies work. Whatever the next decade of AI regulation adds, the assumption that systems must be watched after release appears settled on all three tracks.

The divergence that remains is consequence. Europe legislates, America standardizes, the OECD harmonizes — and the difference stops mattering only for products that never cross a border.

What is a training data pipeline?

A training data pipeline is a staged data-processing system that collects raw text, cleans it, removes duplicates, filters low-quality pages, and mixes the survivors into a final pretraining corpus. Its output quality is measurable: models pretrained on better-curated corpora score higher on standard evaluations, which is why dataset design has become an engineering discipline rather than an afterthought.

The raw input is usually web-scale. Common Crawl, the free corpus underlying much of the field, states on its get-started documentation that crawl data is free to access by anyone, hosted on AWS through an open-data sponsorship, with petabytes collected regularly since 2008. That scale is the reason pipelines exist: no human team can read even a fraction of the input, so quality control has to be automated, measured, and ablated experiment by experiment.

Hugging Face's dataset card for FineWeb describes the result as more than 18.5 trillion tokens of cleaned and deduplicated English web data, processed on its datatrove library and optimized for LLM performance. The card, together with the FineWeb paper, is one of the few fully documented examples of how an open pretraining corpus is actually manufactured from end to end.

Why does the pipeline matter more than it used to? Because model architectures have converged while data has not. Two labs with similar compute budgets can produce very different models purely on the strength of their corpora, which moves the competitive frontier from "who has the best network" toward "who has the best distillation apparatus for the web."

How does a raw web crawl become training tokens?

Through a sequence of extraction, cleaning, filtering, and deduplication stages, each of which strips volume while trying to preserve the text that helps models learn. The FineWeb paper states that its authors carefully document and ablate all of the design choices used in the dataset, including in-depth investigations of deduplication and filtering strategies.

The pipeline stages, in the order a production system typically applies them:

1. Crawl acquisition: download raw WARC archives from a crawler such as Common Crawl, whose data is free to process in the AWS cloud or download over HTTPS.
2. Text extraction: strip HTML, boilerplate, navigation, and advertising to recover the readable content of each page.
3. Language and quality filtering: run language identification plus heuristic or model-based quality classifiers to drop junk pages.
4. Deduplication: remove exact and near-duplicate documents and subsequences, one of the choices the FineWeb authors studied in depth.
5. Sampling and mixing: reweight domains and subsets so the final corpus reflects deliberate choices about what the model should learn.

Each stage discards enormous quantities of data. A page that survives one filter may die at the next; the surviving fraction of a raw crawl is small, and the settings of each filter are as consequential to model behavior as the architecture of the network trained on the output.

Extraction deserves its own emphasis, because it is where most silent damage happens. The same HTML can yield clean article text or a soup of menus and cookie banners depending on the extractor; a corpus built on sloppy extraction teaches a model to speak navigation rather than prose, and no amount of later filtering fully repairs that.

Why does deduplication change model quality?

Because repeated text teaches models to memorize rather than generalize, and duplication also wastes training budget. The FineWeb paper treats deduplication as a first-class research question, running ablations that compare deduplicating at different granularities — document level, sub-document level, and across versus within snapshots — and measuring the effect on downstream model performance.

The paper reports that FineWeb produces better-performing LLMs than other open pretraining datasets, a claim tied to its named evaluations in the study rather than to marketing language. The authors also introduced FineWeb-Edu, a 1.3-trillion-token subset filtered for educational content, built on the finding that model-based quality signals can sharpen a corpus further than hand-written heuristics alone.

The mechanics of why duplication hurts are intuitive. A model that sees the same passage hundreds of times allocates capacity to reproducing it exactly, capacity that would otherwise support generalization. Duplication also skews what the model treats as "common knowledge" — a repeated fringe claim looks, to a frequency-driven learner, like a consensus.

For anyone evaluating a model's training claims, deduplication policy is a useful probe: a lab that cannot describe how it handled duplicate web text generally cannot describe what its model memorized, either.

What does a documented pipeline look like in numbers?

The public record supplies a few anchor figures for one well-documented open pipeline:

These are documented figures from the dataset's own publishers, not third-party estimates. Together they sketch the scale gap between raw crawl data, measured in petabytes at Common Crawl, and the distilled token counts that pretraining runs consume — a funnel that turns the open web into a curriculum.

The numbers also make the economics legible. If quality filtering can cut a corpus by an order of magnitude while improving downstream scores, then the highest-leverage compute in a training run is sometimes spent not on GPUs but on the data stage that decides what the GPUs see.

How do teams know a pipeline worked?

By training small models on competing corpora and comparing them on named evaluations — the ablation method the FineWeb authors use throughout their work. Each design choice gets a control experiment: deduplicate or not, filter with heuristic A or classifier B, keep domain X or drop it. The differences in benchmark scores, not intuition, decide the configuration.

This is why open documentation matters to outsiders. When a lab publishes its ablations, as the FineWeb paper does, observers can distinguish datasets designed for performance from datasets assembled for volume. When nothing is published, the same observers are being asked to trust a process they cannot inspect.

What cannot a pipeline fix?

A pipeline cannot manufacture consent, correctness, or freshness. Web corpora contain personal data, outdated claims, and errors, and filtering heuristics reduce rather than eliminate those problems. The FineWeb authors' willingness to publish failure modes and ablations is what makes the dataset a useful teaching example; labs that disclose less leave outside observers unable to check any of it.

The pipeline also encodes editorial choices — which languages survive, which domains are upweighted, what counts as quality. Those choices are invisible in the finished model unless the dataset documentation discloses them, which is precisely why the open record, not the model card alone, is where data claims go to be verified.

What is inference, and why does it cost money?

Training a model is a one-time capital expense; inference is the recurring one. Every prompt runs real hardware: accelerator chips doing the arithmetic, memory shuttling the model's weights and the conversation's tokens, and electricity for both. A model with hundreds of billions of parameters cannot be reloaded for each request, so providers keep resident capacity warm — GPUs that are idle still cost money, which is why pricing is designed to smooth utilization across customers.

That is also why the bill is denominated in tokens, the chunks of text the model reads and writes. Input tokens are cheaper than output tokens because generating text requires the model to compute every next token in sequence, while reading a prompt can be parallelized and, increasingly, cached. None of this is exotic — it is capacity economics, the same shape as any compute utility.

Two costs sit inside every served request and they scale differently. Compute scales with the number of tokens processed and with the model's active parameters; memory scales with the model's total size, because all weights must be resident whether or not they fire. Much of the past three years of inference engineering — sparser models, quantized weights, paged key-value caches — is an attack on the memory term, because memory is the resource that has historically set how many simultaneous users a given cluster can serve.

How fast are costs actually falling?

The reference point is Stanford HAI's 2025 AI Index, the eighth edition of the annual report, whose stated scope includes novel estimates of inference costs alongside new analyses of AI hardware trends. Its headline finding: the inference cost for a system performing at the level of GPT-3.5 dropped over 280-fold in roughly two years — from late 2022 to late 2024 — holding capability constant and letting the models shrink.

Two supporting curves run underneath. The report puts hardware cost reductions at about 30 percent per year, and energy efficiency gains at roughly 40 percent per year. Multiply better silicon, leaner models and cheaper serving software, and the price of a fixed unit of intelligence falls faster than any single component — a compounding effect, not a one-off price war.

A caution on labels: that 280-fold figure measures cost at a fixed capability level, not the price of frontier models. The most capable systems remain expensive precisely because they use more parameters, more tokens and more compute per query. Cheap intelligence and frontier intelligence are two different markets moving at two different speeds.

What is actually driving the decline?

The report's analysts attribute the collapse to several stacked forces, and the mechanics are worth understanding separately:

1. Hardware economics — accelerator cost per unit of compute falling roughly 30 percent a year, per the AI Index, with successive chip generations.
2. Energy efficiency — roughly 40 percent annual improvement in work done per watt, which compounds directly into serving cost.
3. Distillation — small models trained to mimic large ones deliver GPT-3.5-class quality at a fraction of the parameter count.
4. Quantization — running weights at lower numeric precision, which trades negligible quality for large memory and speed savings.
5. Competition — multiple providers selling overlapping capability tiers, with open-weight alternatives putting public price discipline on the market.

The forces compound rather than add. A distilled model that needs a quarter of the parameters, running on silicon that is 30 percent cheaper per year per unit of work, served with a cache that halves effective input cost, multiplies out to far more than any single improvement — which is how a 280-fold result becomes plausible in roughly 24 months rather than a decade of any one trend.

None of the mechanisms is proprietary in principle, which is why the curve has held across providers rather than depending on any single vendor's pricing strategy. Distillation and quantization are published techniques; hardware economics follow the accelerator market; caching is a systems discipline. The decline is an industry learning curve, not a promotion.

How should a team pick a tier without overpaying?

The practical discipline follows from the curve. Because cost at a fixed capability level keeps falling, the cheapest acceptable model today is a moving target — capability tiers that were premium-priced two years ago now sit near the bottom of public price lists, per the AI Index's framing of capability-adjusted cost. Teams that benchmark their own workload against several tiers quarterly capture the decline automatically; teams that pin to one model inherit its price trajectory.

The measurement that matters is cost per completed task, not per token. A cheaper model that requires longer prompts, more retries or more output tokens to finish the same job can be more expensive end to end. Rigorous evaluations on real workloads — with pass rates, token counts and retry rates recorded — turn the falling price curve into actual savings rather than an interesting statistic.

How do providers charge for inference?

The unit is almost always dollars per million tokens, split into input and output rates — output typically several times input. Around that base, providers have built discounting machinery that rewards predictable usage: cached input, where repeated prefixes of a prompt are stored and re-sold cheaply; batch processing, where non-urgent jobs run at reduced rates on off-peak capacity; and tiered latency, where slower or faster responses carry different prices.

For builders, the lever hierarchy is fixed. Model choice dominates — dropping one capability tier usually beats any optimization. Prompt caching comes next for applications with repeated context, then batching for anything offline. Token discipline, such as trimming system prompts and truncating history, remains the unglamorous third lever that shows up in every serious engineering write-up.

What still keeps AI bills high?

Unit prices fall; usage does not. Three structural costs resist the curve. Output-heavy workloads pay the expensive token every time, and content generation or long-form synthesis cannot dodge that ratio. Agentic systems multiply inference by looping — a task that calls a model twenty times inherits twenty bills, plus retries. And long context re-reads large prompts, which caching softens but does not eliminate.

The second-order effect is the oldest one in energy economics: when a resource gets radically cheaper, consumption rises to meet it. Teams that priced an AI feature as a premium add-on in 2023 are now embedding model calls into core loops — voice interfaces, code review, document pipelines — because per-call cost no longer forbids it. The 280-fold decline, in other words, is not just a savings story; it is the reason AI features stopped being a line item and started being an assumption.

What is the difference between an agent and a workflow?

A workflow is a pipeline the developer draws in advance: retrieve, summarize, classify, each step an LLM call wired into fixed code. An agent is given a goal and decides its own path — which tools to call, in what order, when to stop. Anthropic's guide draws the line precisely: workflows are systems where LLMs and tools are orchestrated through predefined code paths, while agents are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

The practical consequence is a trade between predictability and flexibility. Workflows are debuggable, cheap, and auditable, because behavior is bounded by the code. Agents handle open-ended tasks but consume more tokens, fail in stranger ways, and need guardrails. The guide's most quoted finding is that the most successful implementations were not using complex frameworks or specialized libraries — they relied on simple, composable patterns.

What are agents built from?

The anatomy is smaller than the marketing suggests. Per Anthropic's guide, the basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory. Everything else — the agent loop, the planner, the multi-agent swarm — is arrangement of those pieces.

The engineering discipline is in the boundaries: which tools exist, what each returns, and when the loop must stop. An agent with a vague stopping condition will happily run until the budget does, which is why cost ceilings and step limits are architectural features, not afterthoughts.

What is the agent loop, concretely?

Strip the marketing and a single control cycle remains. The model receives a goal and a tool list; it emits either a tool call or a final answer; the environment executes the call and returns the result; the model looks at the result and chooses again. That is the whole engine — one loop, iterated until the model produces an answer, hits a step limit, or trips a cost ceiling. Everything called an agent framework is scaffolding around this cycle: formatting tool results, persisting the running transcript, retrying failed calls, and enforcing the guardrails.

The transcript, usually called the context or scratchpad, deserves particular attention because it is both the agent's memory and its bottleneck. Everything the agent knows about its task must fit there, which is why long tasks degrade: the loop accumulates tool outputs, the context fills, and earlier instructions lose influence. Production architectures manage this deliberately — summarizing completed steps, offloading state to external stores, and splitting tasks before the context forces the issue. An engineer who can draw the loop on a whiteboard and say where its context overflows understands agent architectures better than most product pages explain them.

How do multi-agent systems divide labor?

When one loop is not enough, systems split into specialists: a coordinating agent decomposes the task, subagents own bounded slices with their own tool access, and results flow back through a synthesis step. Anthropic's guide describes this as a pattern worth reaching for when subtasks are genuinely parallel and separable — different tools, different contexts, different responsibilities — and cautions against it otherwise, because every additional agent multiplies token spend and adds a failure surface.

The division of labor follows organizational logic more than computational necessity. A research agent with web tools, a code agent with a sandbox, and a writing agent with document access mirror the desks of a small team, and the coordinator plays project manager. The design questions are the same ones any manager faces: who owns what, who reports to whom, and what happens when one contributor fails. The A2A protocol extends that org chart across company boundaries, letting the research agent come from one vendor and the code agent from another while both speak the same task language.

How do agents talk to each other?

Single-agent systems hit a ceiling quickly, and 2025-2026 produced a standard answer: the Agent2Agent protocol, A2A. Google introduced it in April 2025 and donated it to the Linux Foundation, and the Foundation announced on April 9, 2026, that the protocol surpassed 150 supporting organizations in its first year, with deep integration across Google, Microsoft, and AWS platforms and active production deployments across multiple industries.

A2A addresses the horizontal problem: agents built on different frameworks, by different vendors, discovering each other, exchanging task descriptions, and negotiating capabilities. It is deliberately complementary to the tool-facing protocols — one standard connects agents to tools and data, another connects agents to agents. Together they let an enterprise compose a travel-booking agent from one vendor with an expense agent from another without bespoke integration.

How do you evaluate an agent framework's claims?

Because the space is crowded and the demos are polished, a repeatable checklist beats impressions:

1. Ask whether the product is a workflow or an agent; the label is often swapped for the more exciting one.
2. Identify the tools the agent can call and who controls their permissions.
3. Check what bounds the loop: step limits, cost limits, and stopping conditions.
4. Look for named evaluations of the underlying model on the task's domain, not aggregate benchmark scores.
5. Confirm how state and memory persist across sessions, and who can inspect them.

What can't these systems do yet?

Documentation is candid if read closely. Anthropic's guide recommends starting with the simplest solution and adding complexity only when measured performance demands it — an admission that agentic behavior still costs reliability. The Linux Foundation's first-year A2A report is a milestone of adoption, not of capability: production deployments exist, and cross-vendor agent collaboration at scale is still early. Long-horizon autonomy, verified correctness of multi-step plans, and predictable cost remain open engineering problems, and any architecture that claims to have solved them is claiming more than its documentation supports. The reliable pattern so far: small, composable, instrumented — and simple until proven insufficient. That is also the best summary of the architecture as a whole: one loop, a handful of tools, a guardrail, and a standard way to talk to the next agent over.

How does an AI-enabled medical device reach the US market?

The route is the one pacemakers and imaging machines take, adapted for software that changes. A manufacturer classifies its device, gathers evidence of safety and effectiveness, and files through a premarket pathway — 510(k) clearance for substantial equivalence to an existing device, De Novo classification for novel lower-risk devices, or premarket approval for high-risk ones. The FDA also reviews modifications that could significantly affect safety or effectiveness, which is where adaptive software strains a framework built for static hardware.

The January 2025 draft guidance addresses exactly that strain. The document, listed on the FDA's guidance page as Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, provides recommendations on the contents of marketing submissions for devices with AI-enabled software functions, per the FDA guidance record.

What does the lifecycle guidance actually ask for?

According to the guidance document, the recommendations cover what documentation and information will support the FDA's evaluation of safety and effectiveness, and reflect a comprehensive approach to managing risk throughout the device total product life cycle. The draft also proposes recommendations for the design, development, and implementation of AI-enabled devices that manufacturers may consider across that life cycle.

Read as an industry map, the guidance tells builders three things. First, evidence expectations now extend across the full life of the device, not just the submission moment. Second, documentation is the deliverable: how the model was built, trained, validated, and monitored becomes review material. Third, planned change is a first-class subject — the framework is being reshaped around software that legitimately updates after clearance.

What are the Good Machine Learning Practice principles?

Underneath the guidance sits a practice layer. The FDA's Good Machine Learning Practice page describes 10 guiding principles developed through international cooperation: the final document comes from the IMDRF, the international medical device regulators' forum, and builds on guiding principles released in October 2021 by the FDA, Health Canada, and the UK's MHRA. The page describes the principles as a call to action to standards organizations, international regulators, and other collaborative bodies to further advance GMLP, with content current as of December 19, 2025 on the FDA's GMLP page.

For hospitals and digital-health builders, the principles function as the audit checklist regulators are converging on: data quality and representativeness, independence of training and test sets, human performance in deployment, and monitored updating. A vendor who cannot answer questions mapped to these principles is signaling where the regulatory conversation will stall.

What is the difference between clearing a device and regulating software in general?

The FDA's position, stated plainly in its own materials, is that the trigger is the device definition, not the algorithm. Software intended for diagnosis, cure, mitigation, treatment, or prevention of disease — or to affect the structure or function of the body — is a device; the same code deployed for administrative scheduling would not be. That is why hospital AI shows up in some departments and not others: the dividing line runs through intended use, not through model architecture.

The consequence for builders is that "we use AI" is not a regulatory category. A triage tool that flags scans for a radiologist is a device with a cleared claim; a documentation assistant that drafts notes for a clinician to sign may fall outside device review entirely. The January 2025 guidance is addressed to the first kind — AI-enabled device software functions — and its life-cycle recommendations only bind once a product sits inside the device perimeter.

For buyers, the same logic converts into a question to ask any vendor: is this cleared, and for what claim? A precise answer is a good sign. A vague one usually means the product lives outside the regulated perimeter, which may be fine — and means the buyer, not the FDA, is the quality control.

Where is this technology actually deployed?

The documented deployments cluster where a regulatory pathway already existed: software that analyzes images, flags readings for clinician review, or augments a measurement, authorized as devices through the pathways above. The pattern matters for expectations. Hospital AI arrives as cleared tools embedded in existing workflows — not as autonomous decision-makers.

That framing also explains the pace. Software in medicine updates on regulatory time, not software time: each significant change can itself be reviewed, per the FDA's stated approach to modifications that could significantly affect safety or effectiveness. The parts of healthcare adopting AI fastest are the ones whose risk tolerance fits that cadence — diagnostics support, workflow triage, and measurement tools, each cleared for a specific claim.

It also explains the vocabulary hospital vendors use. Products are described by their cleared function — flag, detect, measure, prioritize — rather than by what the underlying model could conceivably do. The cleared claim is the product; the model is a component. Teams evaluating multiple vendors should compare claims, not model announcements, because the claim is what the evidence supports and what the regulator reviewed.

How should a hospital evaluate an AI-enabled device?

A disciplined evaluation follows the regulatory record:

1. Confirm the device's authorization status and pathway in the FDA's public guidance and device records.
2. Read the cleared indication — what population, what task, and what the output is for.
3. Ask the vendor for validation evidence and how it maps to the GMLP principles.
4. Establish local monitoring for performance drift and a change-control process for updates.
5. Train clinical staff on the device's documented limits, not its marketing claims.

The evaluation list also works in reverse for vendors. A company that can answer all five steps from documented evidence — authorization record, cleared indication, validation mapped to GMLP, drift monitoring, and staff training on limits — has effectively pre-answered the questions a hospital committee and a regulator will ask. The January 2025 guidance moves the whole industry toward that posture, because it makes life-cycle documentation part of what is reviewed rather than an afterthought.

The comparison with other industries is stark. In consumer software, a model update ships on a Friday and reverts on a Monday if metrics dip. In regulated medicine, the same update is a documented event with a risk assessment attached. Neither cadence is wrong for its context — but any team building AI for healthcare should understand which industry it is actually in before writing the deployment plan.

AI in healthcare is best understood as a regulated-device story with an unusually fast-moving technology inside it. The FDA record — pathways, lifecycle guidance, and international practice principles — is the map of where it actually works.

What does open-source AI actually require?

The Open Source AI Definition, version 1.0, states that the preferred form for making modifications to a machine-learning system must include three elements: data information, code, and parameters. Each element is defined in unusually concrete terms. Data information means enough detail about training data for a skilled person to build a substantially equivalent system, including provenance, scope, labeling procedures, and processing methods.

Code means the complete source code used to train and run the system, from data filtering to training arguments and inference. Parameters means the model weights, potentially including checkpoints from intermediate training stages. The definition is explicit that missing any element breaks the claim, because a user without all three cannot meaningfully exercise the four freedoms. The full text is on the OSI's Open Source AI Definition page.

This matters because the label "open source" carries legal and commercial weight. A company building a product on a genuinely open-source model can fork it, retrain it, audit it, and move it between providers. A company building on a weights-only release is renting access to a frozen artifact under someone else's contract.

What is the difference between open weights and open source?

Open weights means the model's parameters can be downloaded and run locally. That is genuinely useful — it enables local inference, fine-tuning, and offline deployment — but it is a subset of open source. The gap is everything the definition asks for beyond the weights: the training code, the data description, and the rights to redistribute modifications freely.

Licenses are the other half of the gap. The Apache License 2.0, one of the most widely used permissive licenses in software, grants reuse, reproduction, modification, and distribution, and adds an explicit patent grant from contributors. A model released under Apache 2.0 with its code and data information can plausibly satisfy an open-source standard. A model released under a custom contract with usage conditions cannot, however friendly the marketing reads.

The practical test is simple: could a competent third party rebuild a substantially equivalent model from what was published? If the answer is no, the release is open weights, not open source.

The distinction is not a judgment about usefulness. Open-weight models power a large share of local inference, research reproduction, and fine-tuning work, precisely because downloadable parameters lower the barrier to entry. The point is narrower and practical: open weights carry obligations the downloader does not control. If the publisher changes hosting terms, withdraws old versions, or adds usage conditions in a later release, downstream projects inherit every one of those moves. Open source exists as a legal category to prevent exactly that inheritance.

Which licenses do the major model makers actually use?

The big labs mostly publish their own license text rather than adopting a standard one. Meta's Llama 4 Community License Agreement, with an effective date of April 5, 2025 as stated in the agreement, grants a non-exclusive, worldwide, non-transferable, royalty-free right to use, reproduce, distribute, and modify the "Llama Materials" — with conditions attached.

The full terms are on the Llama 4 license page. The naming itself is informative: a "Community License" is a bespoke agreement, and it is not on the OSI's list of approved licenses. That does not make it a bad deal for every user — it means the rights need to be read as a contract before anything is built on top.

What changed when the definition arrived?

Before October 2024, "open-source AI" had no agreed meaning, which made every comparison an argument about vibes. The definition settled the vocabulary. It distinguishes an AI model — which it describes as consisting of the model architecture and model parameters, together with the inference code needed to run it — from the broader system around it. That distinction lets a license be judged component by component rather than all-or-nothing.

The immediate effect was on disclosure. After the definition shipped, model cards and release notes started being read against a concrete checklist: is the architecture documented, are the weights complete, is the training code present, is the data information sufficient to rebuild. Labs that had described releases as "open source" began adding qualifiers — "open weights," "open research" — a small rhetorical retreat that itself signals the definition did its job.

The second-order effect is on procurement. Public bodies and large enterprises now cite the definition in vendor questionnaires, asking not "is your model open?" but "which of the three elements do you publish, and under what terms?" A question that used to be answered with marketing now has to be answered with a table, and tables are harder to blur. For an independent publication covering the space, the definition is the neutral yardstick that neither the open-weight labs nor the closed labs control.

Why do labs release weights but not data?

Training data is where most open-weight releases fail the open-source test. Labs cite three recurring reasons: proprietary data pipelines are a competitive moat; much training data is licensed or scraped under terms that forbid republication; and disclosing datasets creates legal exposure. The result is that the data-information element of the definition is almost never satisfied by frontier labs.

There is also a commercial logic to "open-washing" — borrowing the reputation of open source while keeping the moat. A weights release buys community adoption, security research, and ecosystem lock-in at relatively low cost. Full openness would hand competitors the entire recipe. Reporting on model releases should therefore treat the word "open" as a claim to be checked, not a fact to be repeated.

How should a team evaluate an "open" model claim?

A structured check takes under an hour and prevents expensive surprises:

1. Read the license text itself, not the announcement — conditions usually live in the early sections.
2. Confirm the training and fine-tuning code actually ships, not only inference code.
3. Look for data information: provenance, scope, filtering methodology — the element the definition demands.
4. Check what parameters are published, including whether intermediate checkpoints are available.
5. Map license conditions against your product — user-count caps and acceptable-use clauses decide the answer.

The open-source model landscape is best read as a spectrum: permissively licensed models with full code at one end, weights-only releases under custom contracts at the other. The definition's value is that it turns a marketing argument into a checklist — and most current releases fail the checklist on the same item, data.

Why did matrix multiplication end up owning the chip?

Neural networks compute with tensors — grids of numbers representing weights, activations, and gradients — and nearly every step of training and inference reduces to matrix multiplies and accumulations. Google's TPU documentation states that TPUs are optimized for workloads dominated by matrix computations, and it is equally blunt about what they are bad at: linear algebra with frequent branching, lots of element-wise operations, or requirements for high-precision arithmetic. The chip is a specialist, and the software stack exists to keep the specialist fed.

That feeding happens through compilation. The TPU documentation explains that code goes through the XLA compiler, which turns the linear algebra, loss, and gradient portions of a model graph into TPU machine code while everything else executes on the host machine. On the NVIDIA side, the same specialization appears as tensor cores inside GPUs — matrix engines placed next to general-purpose graphics hardware — which is why a GPU can still render a game while a TPU cannot, yet both excel at the same multiply-accumulate flood. The architectural difference is proportion, not kind.

How do the three chip types divide the work?

Google's documentation lays out the split explicitly, and it is the clearest published guide to choosing AI hardware that exists. CPUs handle quick prototyping that requires maximum flexibility, simple models, small batch sizes, and workloads limited by input-output or networking bandwidth. GPUs fit models with a significant number of custom PyTorch or JAX operations and medium-to-large batches. TPUs take models dominated by matrix math with no custom operations in the main training loop, training that runs for weeks or months, and ultra-large embeddings common in ranking and recommendation workloads.

The table doubles as a warning label. The documentation's list of what TPUs are not suited for is as specific as the list of what they excel at, which is the honest way to evaluate any AI accelerator: the benchmark that matters is the one shaped like your model, not the one shaped like the vendor's marketing deck.

What does a modern AI system look like at rack scale?

NVIDIA's announcement of Blackwell Ultra, made at GTC on March 18, 2025, shows the current ceiling. The GB300 NVL72 connects 72 Blackwell Ultra GPUs and 36 Arm Neoverse-based Grace CPUs in a rack-scale design, and the company claims 1.5 times the AI performance of the previous GB200 NVL72. NVIDIA also claims the platform increases the revenue opportunity for AI factories by 50 times compared with Hopper-based systems — a company-claimed figure tied to throughput assumptions, not audited results.

The announcement's framing matters for another reason: it targets test-time scaling inference — the art of applying more compute during inference to improve accuracy, in NVIDIA's own words — for reasoning, agentic, and physical AI. The newest hardware is optimized not only for training runs but for the inference explosion that follows them, when a deployed model answers queries millions of times a day, each answer potentially extended by deliberate multi-step reasoning.

Why can't software just catch up instead?

Because the bottleneck is arithmetic volume, not code quality. Reasoning models that deliberate over longer chains multiply the inference load; NVIDIA's announcement quotes its chief executive saying reasoning and agentic AI demand orders of magnitude more computing performance. Training a large model is a months-long stream of matrix multiplies over trillions of parameters, and no compiler trick removes the underlying operations — compilation, as the TPU docs describe it, only maps them onto the silicon as efficiently as possible.

That is also why the accelerators keep getting larger and more tightly connected. A single chip's memory and interconnect limit how big a model slice can be, so vendors build racks and pods — groups of chips acting as one logical accelerator — which Google's documentation describes as slices that workloads scale across with minimal code changes. At that point networking becomes part of the computer: the speed of the links between chips determines how often any one of them sits idle waiting for data. The system, not the chip, is the unit of AI compute.

What should a buyer actually take away?

Match the hardware to the shape of the workload, using the maker's own documentation as the first filter. If a model spends its time in standard matrix operations at large batch sizes, TPU-class or tensor-core hardware earns its cost. If it is riddled with custom operations, branching logic, or unusual precision requirements, general-purpose GPUs or even CPUs may finish first — the TPU documentation says so directly, and that candor is rare enough to be worth trusting.

Second, treat multi-x performance claims as company-claimed until an independent, named evaluation says otherwise. NVIDIA's 1.5x figure compares specific systems under specific conditions, and the 50x revenue-opportunity figure is a business projection layered on top of hardware claims. Third, budget for the whole system — interconnect, memory, and the compiler stack — because the documented fit between workload and silicon is exactly where the differences between a fast deployment and an expensive idle cluster hide.

What about memory — why does it dominate the bill?

Because matrix multiplies move more data than they compute, relatively speaking. Every multiply needs its operands fetched from memory and its result written back, and on models with hundreds of billions of parameters, the parameters themselves are the data. Chip designers respond by stacking high-bandwidth memory next to the compute engines and by linking chips into the slices and racks described above, so that memory capacity grows with the accelerator count rather than bottlenecking behind it.

The consequence for buyers is that accelerator counts are only half the sizing exercise. A cluster with too little memory per chip runs models in more, smaller shards — adding communication that can dominate the runtime — while a cluster with too much memory per chip idles capacity. The same documentation-first rule applies: model size, batch size, and precision determine memory demand, and the vendor's system pages state the memory configuration per slice explicitly. Reasoning workloads sharpen the tradeoff further, since test-time scaling keeps activations in memory for longer deliberation chains.

What is AI safety, as a field?

AI safety is the engineering and policy practice of anticipating and limiting harms from AI systems: bias in decisions, misuse for fraud or weapons development, privacy leakage, unreliable outputs presented as fact, and, at the frontier, loss of control over increasingly capable systems. Its input is measurement — evaluations that probe what a model can and cannot do — and its output is mitigation: training choices, access controls, usage policies, and deployment limits.

The field distinguishes near-term harms, which exist in shipped products today, from frontier risks, which concern capabilities that do not exist yet but plausibly could. The distinction matters because the tooling differs: near-term harms are addressed with audits, red-teaming, and documentation, while frontier risks require evaluating models for capabilities their own developers hope never emerge. Both tracks share a methodological core — treat capability claims as testable, and treat test results as the basis for decisions.

AI safety is also distinct from AI ethics in scope and from AI security in mechanism, though the three overlap heavily in practice. Security asks whether a system can be attacked; safety asks whether it causes harm functioning as intended; ethics asks whether its intended function is acceptable. A single incident — a jailbroken model producing harmful content — can sit in all three categories at once.

How does the NIST AI Risk Management Framework work?

NIST's AI Risk Management Framework, released January 26, 2023, is the most widely referenced public document in the field. Per the institute's own description, it is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems, and it was built through a consensus-driven, open process of public comments, drafts, and workshops.

The framework is organized around four functions — Govern, Map, Measure, and Manage — which read as a management loop rather than a compliance checklist. Govern assigns accountability and risk tolerance. Map catalogs the system's context: what it does, for whom, under what conditions. Measure applies quantitative and qualitative evaluation to the mapped risks. Manage allocates resources to the highest-priority risks and monitors the results. The structure deliberately mirrors older NIST cybersecurity and privacy frameworks, so organizations can bolt AI risk onto existing governance rather than invent a parallel apparatus.

Its voluntary status is the framework's main limit and its main strength. Nothing forces adoption, and audits against it vary widely in rigor. But precisely because it is not a regulation, it has been picked up across jurisdictions as a common vocabulary — including by organizations that otherwise answer to entirely different legal regimes.

How do labs set capability thresholds?

Frontier developers publish internal frameworks that define which model capabilities would trigger which mitigations. Google DeepMind's Frontier Safety Framework, published May 17, 2024, is a representative example: the company describes it as a set of protocols for proactively identifying future AI capabilities that could cause severe harm, built around Critical Capability Levels in high-risk domains such as autonomy, biosecurity, cybersecurity, and machine learning research.

The mechanism is a tripwire ladder. Models are periodically run through early warning evaluations; when one approaches a defined Critical Capability Level in a high-risk domain, the framework requires mitigations — tightened security around the model's weights, restricted deployment, or both — before it proceeds. The thresholds are set below the level of severe harm deliberately, so that mitigations arrive with margin rather than in reaction.

Published thresholds make lab claims checkable in a way marketing statements are not: a framework either enumerates its dangerous-capability domains and its evaluation schedule, or it does not. They also make the limits visible. The evaluations probe specific capabilities on specific benchmarks, and a model that scores safely on all of them may still fail in ways no evaluation covered — a gap the labs themselves acknowledge in the framing of these documents.

What can regulators actually do?

Regulators have three real levers, all now in use somewhere. Product and sector law applies existing rules — consumer protection, medical device regulation, financial supervision — to AI systems within their remit, without needing new AI-specific statutes. Procurement and standards leverage operates through requirements like the U.S. federal government's, which direct agencies to manage AI risk along NIST-framework lines for systems they buy and deploy. Dedicated statutes, of which the EU's AI Act is the leading example, classify systems by risk tier and attach obligations and penalties to each tier.

Each lever has a characteristic failure. Sector law arrives piecemeal and misses cross-cutting harms. Procurement rules bind only government business. Comprehensive statutes take years to pass and longer to implement, and they risk hardening around a snapshot of the technology. This is why the voluntary and lab-internal tracks matter in the meantime: they are the only instruments whose update cycle matches the technology's.

What does AI safety not cover yet?

Three gaps recur across every framework published so far. Evaluation coverage: tests exist for a bounded list of capabilities, and absence of evidence on an untested capability is not evidence of absence. Open weights: once model weights are public, access-control mitigations — the strongest tool in most frameworks — no longer apply, and governance shifts entirely to downstream use. Measurement validity: benchmarks can be trained toward, gamed, or simply misread, so threshold decisions inherit the benchmark's blind spots.

A reader evaluating any safety claim can apply the field's own method to it in four steps. Identify what was evaluated, by name. Identify who ran the evaluation and who published the result. Check whether the claimed mitigation is verifiable — restricted deployment can be observed; a training-data improvement generally cannot. And note what the claim conspicuously does not assert, which in safety documents is usually the most informative sentence.

How do the public and private tracks connect?

The two tracks are converging on shared vocabulary rather than shared enforcement. NIST's framework supplies the management grammar — govern, map, measure, manage — that organizations use to structure their AI risk work regardless of what they build. Lab frameworks supply the measurement content: named dangerous-capability domains, evaluation procedures, and thresholds. A regulator drafting obligations can, and in practice does, borrow from both: duties framed as risk-management processes from the former, capability-tiered requirements from the latter.

Procurement is where the connection becomes concrete. When a government agency requires AI suppliers to document risk management along NIST-framework lines, the voluntary framework acquires contractual force for that market without any new statute. The same dynamic operates downstream of the labs: when a model's deployment restrictions are published, enterprise buyers inherit them as conditions of use, and a private threshold becomes a de facto product requirement.

The result is a layered system no single actor controls: measurement standards from public institutes, capability thresholds from developers, binding rules arriving piecemeal from legislatures and buyers. For the foreseeable future, AI safety governance will look less like a single regulator and more like this stack — with each layer's limits, discussed above, setting the agenda for the next one.

What should a non-expert actually take away?

That safety claims in AI are checkable statements, not vibes, and the checking is the same skill as reading any technical claim. The strongest documents in the field share a shape: they name domains, name evaluations, state thresholds, and define mitigations that an outsider could in principle observe. The weakest share the opposite shape — assurances of care with no named test, no stated threshold, and no failure mode acknowledged.

The second takeaway is that the field's honesty about its own gaps is unusually high, and worth taking at face value. The published frameworks say plainly that evaluations are partial, that open weights break access control, and that benchmarks mislead. A reader who internalizes those three caveats knows most of what distinguishes an informed observer of AI safety from a consumer of its press releases.

What do the official guides agree on?

Both Anthropic's and Google's guides treat prompting as an engineering loop rather than incantation. Anthropic's overview page frames the precondition honestly: "This guide assumes that you have: A clear definition of the success criteria for your use case Some ways to empirically test against those criteria A first draft prompt you want to improve If not, spend time establishing that first," per the prompt engineering overview in Claude's documentation. The writing comes third, after measurement exists.

Google's Gemini documentation takes the same empirical stance and spends its pages on structure: system instructions, format specification, and strategies for steering a response before it begins. Neither guide promises magic words, and both implicitly concede the opposite: a prompt that cannot be tested cannot be improved, only reworded.

How should a prompt be structured, per Claude's documentation?

Anthropic's technique list is the backbone, and each entry answers a specific failure mode. Clarity means instructions a competent new hire could follow without guessing. Examples demonstrate the desired output instead of describing it, which removes an entire class of format surprises. XML structuring separates instructions from data with explicit tags, so the model can tell which words are orders and which are material.

The remaining three techniques scale the reasoning. Role prompting establishes who the model should act as, which constrains vocabulary and defaults. Thinking reserves space for the model to work through a problem before committing to an answer. Prompt chaining decomposes a complex job into a sequence of smaller, individually checkable prompts, which converts one unmanageable failure into several findable ones.

Applied selectively they compose into a documented workflow: a role, a task in tags, two worked examples, and a chain of steps covers most real jobs better than a single overloaded paragraph ever does.

What strategies does Google document for Gemini?

Google's prompt design guide for the Gemini API concentrates on controlling the response before generation starts. One documented section addresses output shape directly: "You can give instructions that specify the format of the response. For example, you can ask for the response to be formatted as a table, bulleted list, elevator pitch, keywords, sentence, or paragraph," per the prompt design strategies in Google's documentation. Format instructions move decisions out of the model's discretion and into the prompt where they belong.

The same guide documents system instructions as a separate channel that persists across turns. Its worked example instructs the model to answer comprehensively with detail unless the user requests a concise response, which shows the pattern: a standing rule in the system layer, task-specific content in the prompt layer. Google also documents the completion strategy, in which the prompt supplies the beginning of the desired output and the model continues it, a technique that also helps enforce a format.

The guide's own worked examples, including a sample question about starting a DVD business in 2026 answered by gemini-2.5-flash, are worth reading in full, because they show the vendors testing their own advice in public rather than asserting it.

What is the documented workflow for improving a prompt?

Assembled from both vendors' pages, the loop is short and repeatable:

1. Define success criteria for the task before writing anything, per Anthropic's overview.
2. Build a way to test empirically against those criteria, even a small fixed set of cases.
3. Write a first draft prompt and run it against the tests.
4. Change one variable at a time: format, role, examples, or structure.
5. Chain the task apart when a single prompt keeps failing, and re-test each stage.

The discipline in steps four and five is what separates iteration from thrash. A prompt with three simultaneous changes gives no evidence about which change mattered, and the guides' insistence on empirical testing exists precisely to prevent that ambiguity.

What can't prompting fix?

The guides are candid at the margins. Anthropic's overview is framed around learning when prompt engineering is the right solution at all, implying a category of problems it is not: unclear goals, unmeasurable outputs, and tasks the underlying model cannot do regardless of wording. Better instructions remove interference; they do not add capability.

Diagnosing which of the two is failing is the actual skill the documentation teaches. If a clean, chained, well-formatted prompt still fails on a fixed test set, the issue is the model or the task framing, not the adjectives, and no template will paper over it.

There is also a documented middle category: tasks the model can do but only with the right structure. Long documents that fail as one giant prompt often pass when split across a chain; outputs that drift in tone tighten with a system instruction; answers that miss required fields comply when the format is specified up front. These are the cases where the guides genuinely pay off, and they are identifiable only because the testing loop flagged them in the first place.

The compounding effect is organizational, not personal. A team that writes its success criteria and test cases once ends up with a reusable evaluation harness, and every later prompt, model swap, or provider comparison runs against the same yardstick. That artifact outlives any individual prompt, which is the quiet argument both vendors' guides are making.

What is an AI benchmark, and who publishes one?

A benchmark is a fixed set of tasks plus a scoring rule, run the same way for every system that attempts it. The publisher matters as much as the score: a result is only as trustworthy as the organization that maintains the tasks, keeps them uncontaminated, and publishes the method. Reputable publishers name the test, publish the tasks or a paper describing them, and update the suite when it saturates.

ARC Prize, the nonprofit behind the ARC-AGI series, describes ARC-AGI-2 as a benchmark "designed to stress-test the capabilities of state-of-the-art AI reasoning systems, provide useful signal on AGI progress, and inspire researchers to work on new ideas" on its official benchmark page. The tasks are visual puzzles that look like grids of colored shapes, and the scoring rule is blunt: a system either produces the correct completion or it does not.

Benchmarks differ in what they try to measure. Some test knowledge, some test reasoning, and some test whether a model can act usefully in a real environment. Reading a leaderboard without knowing what the tasks look like is the fastest way to be misled by a number.

Why does ARC-AGI-2 exist when ARC-AGI-1 already did?

ARC-AGI-1 endured five years of competitions and a reported 50,000x scale-up of base models with little progress until late 2024, when test-time adaptation methods changed the picture. Once frontier systems began scoring well, the benchmark stopped separating them. ARC Prize's answer was a harder successor, announced in the organization's launch post on March 24, 2025: "Pure LLMs score 0% on ARC-AGI-2, and public AI reasoning systems achieve only single-digit percentage scores."

The design goal is asymmetry. Tasks are easy for people and hard for machines, which is the opposite of most tests that reward memorized expertise. Every task in the set has been solved by at least two humans in under two attempts, according to the same post, which keeps the human baseline grounded in demonstration rather than estimation.

ARC-AGI-2 also changed what the leaderboard reports. Alongside accuracy, ARC Prize reports a cost axis, because a system that solves problems at a thousand times the human cost is a different engineering proposition from one that approaches human efficiency. A competition with $1,000,000 in prizes, hosted on Kaggle and opened alongside the benchmark, was set up to reward efficient systems rather than brute-force scale.

How does SWE-bench test real coding claims?

SWE-bench, described in a paper first submitted to arXiv on October 10, 2023, turns real software maintenance into a measurable task. The authors introduce "an evaluation framework consisting of 2,294 software engineering problems drawn from real GitHub issues and corresponding pull requests across 12 popular Python repositories," where a model receives a codebase and an issue description and must edit the code to resolve it, per the paper's abstract.

The tasks resist shortcuts. The abstract notes that resolving issues "frequently requires understanding and coordinating changes across multiple functions, classes, or even files simultaneously," which is closer to daily engineering work than answering quiz questions. A maintained family of leaderboards, including a Verified split reviewed by human engineers, publishes resolution rates and cost per resolved instance.

What problem is HELM trying to solve?

Not every benchmark is a single score. Holistic Evaluation of Language Models, published by Stanford researchers in a paper submitted on November 16, 2022, argues that "language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks are not well understood." HELM therefore evaluates across many scenarios and measures several metrics at once, including accuracy and calibration, in a framework the authors built for transparency, per the paper submitted on November 16, 2022.

The practical lesson for readers is that a model ranking depends on which scenarios and metrics were chosen. HELM makes those choices explicit and notes where coverage is thin, which is a discipline worth expecting from any leaderboard. The paper openly flags gaps in its own coverage, from question answering in neglected English dialects to metrics for trustworthiness, and that admission is a feature of the method rather than an embarrassment.

Multi-metric evaluation also changes what a "better model" means. A system can win on accuracy and lose on calibration, which means its confident answers are wrong at a predictable rate. Buyers who ship models into products care about that second number more than the first, and single-score leaderboards simply do not carry it.

How should a reader read a benchmark number?

A score means nothing without its context, and assembling that context takes about five minutes. The habit separates people who track model progress from people who retell marketing.

1. Identify the publisher and confirm the test is named, public, and currently maintained.
2. Check what the tasks actually look like, and whether they resemble the intended use case.
3. Check the metric: accuracy, pass rate, resolution rate, and human-normalized scores are not interchangeable.
4. Check the cost axis where one exists, since efficiency is part of the result, not a footnote.
5. Check the date and the model versions, because leaderboards go stale quickly.

None of this requires inside information. Every fact cited in this piece sits on a publisher's own page, which is exactly where benchmark claims should live before anyone repeats them.

What does Gemma 4 actually do per its documentation?

Gemma 4 is a set of open-weight language models designed for on-device AI development. Google's announcement states the family supports multi-step planning, autonomous action, offline code generation and audio-visual processing without specialized fine-tuning, with support for more than 140 languages. The smallest variants, E2B and E4B, target ultra-mobile and browser deployment on hardware like Pixel phones and Chrome; the dense 31B model bridges toward server use; and a 26B A4B variant mixes the two regimes with sparse activation.

The documentation also specifies the engineering behind the speed claims. Every Gemma 4 model includes a dedicated draft model for speculative decoding, a technique where a small model proposes tokens that the large model verifies, cutting inference latency with no quality loss according to Google. Google publishes approximate memory requirements per size and precision, from 0.84 GB for a text-only mobile E2B build up to 69.9 GB for a 16-bit 31B deployment, which lets developers match a variant to a device before writing any code.

Why does the Apache 2.0 switch matter?

The license is the quiet headline. Previous Gemma generations shipped under Google's custom terms, which developers found restrictive, and Ars Technica's report explains that the switch to Apache 2.0 removes the overbearing terms of use and commercial restrictions that had made many developers apprehensive about building on Gemma. Apache 2.0 is a permissive standard license that Google cannot unilaterally reinterpret later, which matters for companies betting products on open weights they do not control at the source.

The release also lands in a specific competitive context. Open-weight families compete on what developers may legally build, and a permissive license widens the pool of commercial users who can ship Gemma-derived products without counsel reviewing custom terms. That is a distribution decision as much as a technical one.

What can these models not do yet?

Open models trade capability for portability. Gemma 4's largest documented size, 31B parameters, is far below frontier-scale closed models, and its headline capabilities have not been evaluated in the announcement against named third-party benchmarks. On-device agents also inherit the memory ceilings of phones and laptops, so the audio-visual and planning features depend on which of the five variants a device can host.

What the release demonstrably provides, per the documentation, is a licensed, multi-size model family built for edge deployment, with the Agent Skills application in Google's AI Edge Gallery showing multi-step workflows, such as querying Wikipedia and building study flashcards, running entirely on a phone. The gap between that demonstration and unsupervised daily-use agents remains the open question, and it is a question of memory, not of licensing.