# AI in everyday devices: NPUs, on-device AI and what still goes to the cloud

> Phones and PCs now run small AI models on a built-in NPU and send bigger requests to the cloud. What runs where, why, what TOPS means and what to buy in 2026.

- URL: https://computese.com/ai-in-everyday-devices-transforming-technology/
- Author: Duong Quan Nguyen, CEO, Computese
- Published: 2024-08-01
- Updated: 2026-09-25
- Topics: AI & automation

## In short
- Phones, PCs and browsers now run small AI models on a neural processing unit (NPU) for dictation, summaries, camera effects and search, and send requests that need a larger model or fresh web knowledge to the cloud.
- TOPS counts peak trillions of operations per second, usually at INT8. A Copilot+ PC needs a 40+ TOPS NPU, 16 GB of RAM and 256 GB of storage, but memory and software decide how fast a model actually runs.
- Apple's Private Cloud Compute and Google's Private AI Compute send bigger requests to attested servers the vendor says it cannot read; ChatGPT in Siri goes to OpenAI only when you allow it, under different terms.
- On-device describes where the model runs, not what the app does with the result. Check each feature's own documentation, and for business devices set the policies before rollout.
- Before buying in 2026, check the feature list for your language and region, the memory, and which features depend on servers with daily limits, not only the TOPS figure.

AI in everyday devices means phones, laptops and browsers that run AI models on their own chips, usually a neural processing unit (NPU), for quick private tasks such as dictation, summaries and camera effects, and hand bigger requests to models in the cloud. What stays on the device comes down to model size, speed, battery and privacy.

The ChatGPT integration Apple shipped in December 2024 made that split easy to see: one request is answered on the iPhone, the next on Apple's servers, a third by OpenAI. This guide explains what an NPU is and what TOPS measures, how Microsoft's Copilot+ PCs, Apple Intelligence and Android's Gemini Nano divide the work, which features run where and why, what that means for your privacy, what the platforms have said is coming next, and what to check before you buy a phone or PC in 2026. [What AI assistants can do for you](https://computese.com/ai-assistants-are-set-to-revolutionize-daily-life/), and AI's effect on society, are separate topics and get only a line here. So are the tiny models that run on microcontrollers inside sensors and appliances, covered in [machine learning on embedded systems](https://computese.com/machine-learning-on-embedded-systems/).

## What on-device AI is, and why it moved onto the device

A trained model does two kinds of work: training, which happens once in a data center, and inference, which happens every time you use it. On-device AI means the inference runs on your own hardware. The first wave of generative AI ran in data centers, because the models were far too large for a phone. What changed is that vendors built small language models (SLMs) and shipped chips with dedicated AI hardware to run them.

The consumer timeline is short and well documented:

- **December 6, 2023:** the Pixel 8 Pro became the first phone running [Gemini Nano](https://blog.google/products-and-platforms/devices/pixel/pixel-feature-drop-december-2023/), Google's on-device model, on its Tensor G3 chip. It powered summaries in the Recorder app and Smart Reply in Gboard.
- **May 20, 2024:** Microsoft [introduced Copilot+ PCs](https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/), Windows laptops built around an NPU of 40+ TOPS, on sale from June 18.
- **June 10, 2024:** Apple [announced Apple Intelligence](https://www.apple.com/newsroom/2024/06/introducing-apple-intelligence-for-iphone-ipad-and-mac/), with a model that runs on the iPhone, iPad or Mac and a larger one on Apple's own servers.

None of these is purely local. Microsoft described Copilot+ PCs as small models on the device working together with large models in its Azure cloud, and Apple and Google both run a second tier of models in their data centers. The useful question for any feature is therefore not "is it AI?" but "where does this request go?"

## What an NPU is and what TOPS measures

An NPU is a block inside the system on a chip (SoC), next to the CPU and the GPU, built for one job: the matrix math that neural networks are made of. Microsoft's [Copilot+ PC developer guide](https://learn.microsoft.com/en-us/windows/ai/npu-devices/) describes NPUs as designed specifically for deep learning operations, working in parallel and using less energy on AI tasks than a CPU or GPU, which is where the battery-life claims come from. The names differ by vendor: Apple calls its NPU the Neural Engine, Qualcomm's is the Hexagon NPU, and Google's phone chips are called Tensor. On Windows 11, Task Manager now shows NPU usage alongside the CPU and GPU.

The core of that math is the multiply-accumulate (MAC) operation: multiply two numbers and add the result to a running total. An NPU is, in essence, thousands of MAC units side by side. Qualcomm's [guide to TOPS](https://www.qualcomm.com/news/onq/2024/04/a-guide-to-ai-tops-and-npu-performance-metrics) gives the formula the industry uses: a MAC counts as two operations, so TOPS = 2 × MAC units × clock frequency / 1 trillion. A hypothetical NPU with 10,000 MAC units at 2 GHz would be rated at 2 × 10,000 × 2 billion = 40 trillion operations per second, or 40 TOPS. The figure is quoted at peak frequency, and the current standard is to measure it at INT8 (8-bit integer) precision.

![A chip package with CPU cores, a GPU block and an orange grid of multiply units for the NPU, fed with model weights from two memory chips beside it.](https://computese.com/images/blog/ai-in-everyday-devices-transforming-technology/npu.c98bd1e793-1536.webp)

*The TOPS figure counts the multiply units; the memory feeding them decides how many are busy.*

### Why a higher TOPS figure is not the whole story

TOPS is a ceiling, not a measurement of any real task. Qualcomm itself says memory bandwidth, software optimization and system integration decide real performance, and points buyers to benchmarks such as Procyon AI that run actual models. Four things matter as much as the headline number:

- **Precision.** Models are usually trained in 16- or 32-bit floating point. Microsoft notes that many NPUs only do integer math such as INT8, so models must be converted (quantized) before they run there. A figure quoted at a lower precision is not comparable with an INT8 one.
- **Memory.** A language model reads its weights for every token it produces. Apple's announcement of its [M5 Ultra](https://www.apple.com/newsroom/2026/08/apple-introduces-m6-and-m5-ultra-for-a-big-leap-in-performance-and-ai-compute/) (August 2026) ties higher unified memory bandwidth directly to more tokens per second.
- **Software.** An NPU does nothing until software targets it. On Windows, Microsoft recommends Windows ML, which detects the NPU, picks the matching execution provider (the chip vendor's back end) and falls back to the GPU or CPU if the NPU is unavailable.
- **Other accelerators.** GPUs run AI too. Apple's M6 (August 2026) puts a Neural Accelerator in every GPU core next to its Neural Engine, and Microsoft's Phi Silica model can run, in experimental Windows builds, on recent NVIDIA and AMD graphics cards with 6 GB or more of video memory.

For reference points, Qualcomm quoted up to 45 TOPS for its Snapdragon X Series laptop chips in April 2024, and Apple quoted 38 trillion operations per second for the [M4's Neural Engine](https://www.apple.com/newsroom/2024/05/apple-introduces-m4-chip/) in May 2024.

## Why some AI runs on the device and some goes to the cloud

Every built-in AI feature is a trade between four things.

**Latency.** A local model skips the network round trip, the same reason [edge AI on 5G networks](https://computese.com/exploring-the-future-ai-machine-learning-technology/) runs models near the device. Apple measured its 2024 on-device model on an iPhone 15 Pro at about 0.6 milliseconds per prompt token before the first word appears (so roughly 0.6 seconds for a 1,000-token prompt), then 30 tokens per second. Microsoft built its Mu model for the agent in Windows Settings after a larger Phi model missed the latency target; Mu answers in under 500 milliseconds on the NPU. Android's documentation adds the caveat: without the network delay, speed depends entirely on the device's hardware.

**Privacy and cost.** Google's [Gemini Nano page](https://developer.android.com/ai/gemini-nano) sums up the case for local inference: prompts run on the device, sensitive data stays there, features work offline and there is no per-request server cost.

**Battery.** An NPU is efficient, but local inference still drains your battery rather than a data center's power supply. Android's AICore enforces per-app quotas, including a battery-use quota, and only lets apps run the model while they are in the foreground. (Our guide to [extending laptop battery life](https://computese.com/how-to-extend-your-laptops-battery-life/) covers the other drains.)

**Model size.** This is usually the deciding factor. A model's memory footprint is roughly its parameter count times the bits stored per weight. Apple's 2024 on-device model had about 3 billion parameters: at 16 bits per weight that would be about 6 GB for the weights alone, but Apple [compressed it](https://machinelearning.apple.com/research/introducing-apple-foundation-models) to an average of 3.7 bits per weight, about 1.4 GB. Its 2025 model went to [2 bits per weight](https://machinelearning.apple.com/research/apple-foundation-models-2025-updates), about 0.75 GB. Even so, Apple says that model is not designed to be a chatbot for general world knowledge. The models that are sit in data centers, or on machines like the Mac Studio with M5 Ultra, which Apple says can hold models with hundreds of billions of parameters in up to 512 GB of unified memory.

![A phone and a laptop each hold a small orange model block in their memory, while a much larger model block sits in a server under a cloud, reached by a dashed arrow.](https://computese.com/images/blog/ai-in-everyday-devices-transforming-technology/fit.72e4a0579b-1536.webp)

*Whether a model fits in memory next to everything else often matters more than how fast the chip is.*

Size also explains why built-in AI has hardware floors. Chrome's built-in model needs at least 22 GB of free space on the drive that holds the Chrome profile, and either a GPU with more than 4 GB of video memory or 16 GB of RAM and four CPU cores. Microsoft [said in June 2026](https://blogs.windows.com/msedgedev/2026/06/02/expanding-on-device-ai-in-microsoft-edge-new-models-and-apis-for-the-web/) that the hardware requirements of Phi-4-mini, the 4-billion-parameter model it had used in Edge, limited how many devices could use it. The practical result everywhere is a router: small, personal, frequent tasks stay local, while long documents, image generation and anything that needs fresh information from the web go to a server.

## Copilot+ PCs: Microsoft's 40 TOPS line for Windows

A Copilot+ PC is a Windows 11 PC that meets a hardware floor Microsoft set for its on-device AI features. As listed on the [Windows 11 specifications page](https://www.microsoft.com/en-us/windows/windows-11-specifications) in September 2026, it needs:

| Component | Copilot+ PC minimum                                                                                                     |
| --------- | ----------------------------------------------------------------------------------------------------------------------- |
| Processor | An NPU capable of 40+ TOPS: AMD Ryzen AI 300 and 400 series, Intel Core Ultra 200V and 300V series, Snapdragon X series |
| Memory    | 16 GB DDR5 or LPDDR5                                                                                                    |
| Storage   | 256 GB SSD or UFS                                                                                                       |

Microsoft is explicit that a laptop with a 40+ TOPS NPU but less memory or storage is not a Copilot+ PC and does not get features such as Click to Do and the improved Windows search. The features reserved for Copilot+ PCs include Recall (still labelled a preview), Click to Do, improved Windows search, Live Captions that translate audio from over 40 languages into English, image tools in Paint and Photos, and the agent in Settings. Microsoft says many of them, once downloaded, run without an internet connection. A few are hybrid: Paint's Cocreator creates images on the device but needs an internet connection and a Microsoft account for cloud services that check the output for responsible use.

Not everything needs 40 TOPS. The basic Windows Studio Effects (automatic framing, background blur, eye contact and voice focus) run on any Windows 11 device with an NPU of 10+ TOPS, because they process every video frame of a call, which is exactly the constant low-power work an NPU exists for.

Two features show what "on-device" means in practice:

- **Recall** saves a snapshot of your active screen every few seconds, analyses it locally and lets you search your history in plain language. According to [Microsoft Support](https://support.microsoft.com/en-us/windows/ai/ai-features/retrace-your-steps-with-recall), it is opt-in, snapshots are encrypted on the local drive and not shared with Microsoft or third parties, and opening Recall requires Windows Hello Enhanced Sign-in Security with a face or fingerprint. It also needs device encryption or BitLocker and at least 50 GB of free space.
- **The agent in Settings** turns a plain request such as "increase brightness" into the right settings change, using [Mu](https://blogs.windows.com/windowsexperience/2025/06/23/introducing-mu-language-model-and-how-it-enabled-the-agent-in-windows-settings/), a 330-million-parameter model Microsoft built to run entirely on the NPU (June 2025).

## Apple Intelligence and Private Cloud Compute

Apple Intelligence splits work between an on-device model and larger models on Apple's servers, which Apple calls Private Cloud Compute (PCC). Apple introduced both on June 10, 2024. The current generation arrived with iOS 27, iPadOS 27 and macOS 27 on [September 14, 2026](https://www.apple.com/newsroom/2026/09/siri-ai-a-profoundly-more-capable-and-personal-assistant-is-here/), including Siri AI, a rebuilt Siri released as a beta in English. Apple says its new foundation models were custom-built in collaboration with Google and its Gemini models, and run both on device and on PCC.

Apple's [security team's design write-up](https://security.apple.com/blog/private-cloud-compute/) starts from a real problem: a cloud model needs to read your request in plain text, so the end-to-end encryption that protects iMessage cannot apply, and ordinary cloud privacy promises are hard for anyone outside to verify. PCC's answer has several parts:

1. **Custom servers.** PCC nodes use Apple silicon with the Secure Enclave and Secure Boot, and a stripped-down operating system with no remote shell, no interactive debugging and no general-purpose logging.
2. **Encryption to specific machines.** The device encrypts each request to the public keys of a small set of nodes it has verified. Load balancers and gateways in between cannot decrypt it.
3. **A relay that hides you.** Requests pass through a third-party OHTTP relay that hides your IP address, and carry no identifying information, so an attacker cannot steer one person's requests to a compromised server.
4. **Nothing kept.** Data is deleted once the response is returned, and the storage encryption keys change at every reboot.
5. **Verifiable software.** Every production software image is recorded in an append-only transparency log and published for researchers, and a device only sends data to nodes whose remote attestation matches a release in that log.

![A phone sends a padlocked envelope through a relay box to three servers, after checking an orange public ledger that lists the software each server is allowed to run.](https://computese.com/images/blog/ai-in-everyday-devices-transforming-technology/pcc.1a668bb430-1536.webp)

*The phone refuses any server whose software is not in the public log, so a quiet change on the server side would be visible.*

Two practical limits apply in 2026. Apple says server-backed features, including Siri AI and Image Playground, have daily usage limits, and that expanded access will be available for a fee in the future. And availability varies: Apple Intelligence on iOS 27 needs an iPhone 15 Pro, 15 Pro Max or any iPhone 16 or later, or an iPad or Mac with M1 or later, and [Siri AI is not initially available](https://www.apple.com/newsroom/2026/06/apple-introduces-siri-ai-a-profoundly-more-capable-and-personal-assistant/) in the EU on iPhone, iPad and Apple Watch, or in China.

## How ChatGPT and other assistants are built into phones and PCs

"Built in" can mean three different things, and each has its own privacy terms.

**A third-party model behind the system assistant.** Apple added ChatGPT to Siri, Writing Tools and visual intelligence with iOS 18.2 on [December 11, 2024](https://www.apple.com/newsroom/2024/12/apple-intelligence-now-features-image-playground-genmoji-and-more/). It is opt-in and does not need a ChatGPT account. Apple's [current support page](https://support.apple.com/guide/iphone/turn-on-chatgpt-iph00fd3c8c2/ios) spells out what is sent: your request and attachments, plus limited data such as time zone, country, device type and language. Your IP address is hidden from OpenAI, although your general location is shared. Without an account, OpenAI must not store the request or use it to train its models; once you sign in to a ChatGPT account, OpenAI's own policies apply. Siri asks before sending anything unless you turn confirmations off, and the extension can be blocked in Screen Time settings.

**A partner's model inside the vendor's own stack.** Apple's 2026 foundation models were built with Google's Gemini but run on Apple's devices and PCC, under Apple's privacy design. App developers can also plug models such as Claude or Gemini into Apple's Foundation Models framework (June 2026).

**The vendor's own cloud models.** Google's Pixel features use Gemini Nano locally and larger Gemini models in the cloud; Microsoft pairs local models on Copilot+ PCs with cloud models in Azure. Standalone assistant apps, and what they do with your conversations, are a subject for a separate post.

## Gemini Nano and on-device AI on Android and Chrome

On Android, Gemini Nano runs inside AICore, a system service that downloads the model, keeps it updated and applies safety filters. Apps reach it through the ML Kit GenAI APIs, which offer summarization, proofreading, rewriting, image description, speech recognition and a general Prompt API. Because every app shares the one model already on the phone, apps do not each download their own copy.

Google designed AICore to the same principles as Android's Private Compute Core: it is isolated from most other packages, has no direct internet access (model downloads go through the open-source Private Compute Services app), and does not keep a record of inputs or outputs after processing.

Support depends on the phone, and so does the model. The [ML Kit device list](https://developers.google.com/ml-kit/genai) in September 2026 covers the Pixel 9, 10 and 11 series, the Galaxy S25 and S26 series and recent Galaxy foldables, and flagships from Honor, Motorola, OnePlus, OPPO, vivo, Xiaomi and others, running Gemini Nano versions from nano-v2 to nano-v4. Google warns developers that different versions can give different answers to the same prompt. In April 2026 Google said Gemini Nano was on over 140 million devices, and that [Gemini Nano 4](https://android-developers.googleblog.com/2026/04/gemma-4-new-standard-for-local-agentic-intelligence.html), based on its open Gemma 4 model, is up to 4x faster and uses up to 60% less battery than the previous version; the September list shows it on the Pixel 11 series and Samsung's Galaxy Z Fold8 and Z Flip8.

For requests too big for the phone, Google announced [Private AI Compute](https://blog.google/innovation-and-ai/products/google-private-ai-compute/) on November 11, 2025: Gemini models on Google's TPUs inside what it calls Titanium Intelligence Enclaves, reached through remote attestation and encryption, so that the data is accessible only to the user and not to Google. The uses Google announced with it were Magic Cue suggestions on the Pixel 10 and Recorder summaries in a wider range of languages. Recorder shows how the line moves: its summaries launched on the device in 2023, and two years later used the private cloud to reach more languages.

The same idea has reached the browser. Chrome on desktop ships Gemini Nano for its [built-in AI APIs](https://developer.chrome.com/docs/ai/get-started) (Summarizer, Writer, Rewriter, Proofreader and Prompt API), and Google says that once the model is downloaded, using it sends no data to Google or any third party; Chrome on Android and iOS does not support these APIs yet. Edge offers similar APIs and in June 2026 previewed a smaller model, Aion-1.0-Instruct, for devices with weaker GPUs or none at all.

## Which features run on the device and which go to the cloud

The table shows where a sample of well-known features run, according to each vendor's documentation as of September 2026, and the reason that placement makes sense.

| Feature                                                    | Where it runs                                                    | Why there                                           |
| ---------------------------------------------------------- | ---------------------------------------------------------------- | --------------------------------------------------- |
| Windows Studio Effects (blur, framing, eye contact)        | NPU on the PC (10+ TOPS)                                         | Runs on every video frame: latency and battery      |
| Recall search                                              | On the PC; snapshots encrypted locally                           | A record of your screen should not leave the device |
| Agent in Settings                                          | Mu model, entirely on the NPU                                    | Needs instant answers to short requests             |
| Paint Cocreator                                            | Image made on the device, with a cloud check for responsible use | Generation is local; moderation needs the service   |
| Apple notification and email summaries (2024 design)       | The on-device model with task-specific adapters                  | Personal text, short input                          |
| Siri AI and Image Playground (iOS 27)                      | On-device models plus Private Cloud Compute, with daily limits   | Larger models and knowledge from the web            |
| Siri with the ChatGPT extension                            | OpenAI's servers, when you allow it                              | World knowledge beyond Apple's own models           |
| ML Kit summaries, proofreading and rewriting on Android    | Gemini Nano in AICore                                            | Private, offline, no server cost for the app        |
| Magic Cue and Recorder summaries in more languages (Pixel) | Google Private AI Compute                                        | Needs Gemini models larger than the phone can hold  |
| Chrome Summarizer, Writer and Prompt API                   | Gemini Nano in Chrome on the desktop                             | The model sends nothing to Google once downloaded   |

Placement is not permanent. Vendors move features between tiers as models shrink or grow, so treat any list, this one included, as a snapshot and check the feature's own documentation.

## What on-device AI means for your privacy

On-device processing removes the biggest exposure, sending your input to someone else's computer, but it is not a privacy guarantee on its own.

> [!IMPORTANT]
> "On-device" describes where the model runs, not what the app does with the result. An app can run a local model and still upload the summary, the label or the transcript it produced. Read the app's privacy terms, not just the chip's marketing.

- **Local records are still records.** Recall creates something that did not exist before: a searchable history of your screen. Microsoft protects it with encryption, Windows Hello and on-device filtering of sensitive information (on by default), but it remains a detailed record that exists only because the feature is on.
- **Sync can move local data.** Apple's new Siri app syncs conversation history across your devices through iCloud, which Apple describes as private. Know which features sync before assuming everything stays on one device.
- **Private clouds make checkable promises.** PCC and Private AI Compute are designed so that the vendor cannot read requests, and PCC publishes its software for inspection. That is a stronger position than an ordinary cloud service's privacy policy, which you can only take on trust.
- **Third parties have their own terms.** ChatGPT through Siri follows the terms Apple describes only while you are not signed in to a ChatGPT account.

For organizations, the defaults matter. Microsoft [disables and removes Recall on managed devices](https://learn.microsoft.com/en-us/windows/client-management/manage-recall) unless an administrator allows it, and even then administrators cannot switch on snapshots for a user; policies also cap snapshot storage and retention (30 to 180 days). Apple's ChatGPT extension can be blocked through Screen Time restrictions. These are device-management decisions, best made before the laptops ship: our [IT support service](https://computese.com/services/it-support/) enrols laptops and phones in Intune or Jamf and keeps them compliant from one console, which is where settings like these belong.

## Where everyday AI is heading: the dated plans

Forecasts are cheap, so this section lists only what the platform owners have announced, with their dates, as of September 25, 2026.

- **October 2026, Apple:** Siri AI adds French, Japanese, Korean, Portuguese and Spanish, a month after its English beta.
- **October and November 2026, Microsoft:** according to its [Phi Silica documentation](https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica) (updated July 2026), a new on-device model, Aion Instruct, reaches Windows Insider devices in October and retail devices in November, when Phi Silica is removed.
- **Late 2026, Apple:** Live Rewind (the last 15 seconds of a conversation as text) and Siri Recap (summaries of conversations) come in beta to Apple Watch Series 12 and Ultra 4, paired with an iPhone 16 or later other than the 16e, with daily limits on their server-side parts.
- **Later in 2026, Google:** Gemini Nano 4 on new flagship Android phones, announced in April; by September it was listed on the Pixel 11 series and the Galaxy Z Fold8 and Z Flip8.
- **No date yet:** Apple says paid access beyond the daily limits for server-backed features will come later, and that Image Playground will support the SynthID standard so AI-edited images can be identified. Google says it is working to standardize the browser's built-in AI APIs across browsers.

Read together, the plans point the same way. On-device models are getting smaller and more capable (Edge's new model targets weaker GPUs and even CPU-only devices; Nano 4 claims better speed and battery use), the private-cloud tier is growing for everything that does not fit, and access is increasingly shaped by region, language and daily quotas. Assistants are also starting to act inside apps, from Siri AI's app actions to the Windows agent in Settings; what that means for you is covered in our post on [AI assistants](https://computese.com/ai-assistants-are-set-to-revolutionize-daily-life/).

## Building on-device AI into your own app

If you build software, the platform models are now available to your app, which changes the cost of adding AI to a feature. The options as of September 2026:

- **Apple:** the Foundation Models framework (2025) gives apps the on-device model with guided generation into Swift types and tool calling. Since [June 2026](https://www.apple.com/newsroom/2026/06/apple-aids-app-development-with-new-intelligence-frameworks-and-advanced-tools/) it also reaches server models on PCC, free of cloud API cost for members of the App Store Small Business Program with fewer than 2 million first-time downloads, and other providers' models; a new Core AI framework runs your own models on the Neural Engine.
- **Android:** the ML Kit GenAI APIs on AICore. Plan for the per-app quotas, the foreground-only rule and different Nano versions on different phones.
- **Windows:** Windows ML over ONNX Runtime for your own models, with the Olive toolchain to quantize them for NPUs, and the Windows AI APIs for Microsoft's built-in model.
- **The web:** Chrome's and Edge's built-in AI APIs, with a fallback for browsers and devices that do not meet the hardware requirements.

The design rule follows from the sections above: put private, short and frequent tasks on the device, send large-context or knowledge-heavy tasks to a server, and always build the fallback, because not every user's device meets the hardware floor. Our guide to [intelligent apps](https://computese.com/artificial-intelligence-and-intelligent-apps/) covers the wider architecture, and the [AI frameworks comparison](https://computese.com/latest-ai-tools-and-frameworks-a-comparative-analysis/) covers runtimes such as ONNX Runtime and llama.cpp. If you are weighing whether to build a feature at all, our [AI and automation service](https://computese.com/services/ai-automation/) starts with a review of one real workflow before anything is built, and sometimes the answer is a feature you already license.

## Buying a phone or PC for AI in 2026

Buy for the features you will actually use, then check the hardware that those features require.

1. **Check the feature list for your language and region first.** Siri AI launched in English and not on EU iPhones; Recall is optimized for six languages; Copilot+ image tools are optimized for English prompts. A feature you cannot use is not worth paying for.
2. **On Windows, look for the Copilot+ PC label, not just the TOPS figure.** That guarantees the 40+ TOPS NPU, 16 GB of RAM and 256 GB of storage. If you want Recall, budget for 50 GB of free space, BitLocker or device encryption and a fingerprint reader or face camera for Windows Hello. Our guide to [setting up Windows on a new computer](https://computese.com/how-to-set-up-windows-for-a-new-computer/) covers the rest.
3. **Treat memory as the ceiling.** More RAM decides which local models fit next to your apps; TOPS only decides how fast they run once they fit.
4. **On Apple devices, check the model.** Apple Intelligence on iOS 27 needs an iPhone 15 Pro, 15 Pro Max or iPhone 16 or later; on iPad and Mac, M1 or later (plus the iPad mini with A17 Pro and the MacBook Neo). For company phones, [the iPhone 16 guide](https://computese.com/iphone-16-series-launch-sparks-excitement/) covers support dates and the MDM settings that restrict Apple Intelligence.
5. **On Android, check Google's device list.** Gemini Nano support, and its version, varies by model, including between phones from the same maker.
6. **Expect limits on server-backed features.** Apple already applies daily limits and plans paid access beyond them; assume cloud-backed AI features may cost more later.
7. **Check how long the device gets updates.** AI features arrive through operating system updates, so a device near the end of its support period will miss them. See [how to update your computer or phone](https://computese.com/how-to-update-your-computer-or-phone/).
8. **For business devices, set the policies before rollout.** Decide on Recall, the ChatGPT extension and similar features once, centrally.

For how Google's model family fits together beyond the phone, see our explainer on [Google Gemini](https://computese.com/google-unveils-gemini-the-most-advanced-and-versatile-ai-model-yet/).

## Key terms
- **On-device AI**: Running an AI model's inference on the phone, PC or browser itself instead of on a server. The model has to fit the device's memory, power and heat limits.
- **NPU (neural processing unit)**: A block of a system on a chip built for the matrix math of neural networks, with many multiply-accumulate units working in parallel. Apple calls its NPU the Neural Engine.
- **TOPS**: Trillions of operations per second: an NPU's theoretical peak, calculated as 2 × MAC units × clock frequency. The industry quotes it at INT8 precision.
- **Small language model (SLM)**: A language model with millions to a few billion parameters, small enough to run on a device. Examples: Apple's ~3B on-device model, Gemini Nano, Microsoft's Phi Silica and Mu.
- **Quantization**: Storing a model's weights and math at lower precision, such as 8, 4 or 2 bits instead of 16 or 32, so it uses less memory and runs on integer hardware like an NPU.
- **Copilot+ PC**: Microsoft's class of Windows 11 PCs with a 40+ TOPS NPU, at least 16 GB of RAM and 256 GB of storage, required for features such as Recall and Click to Do.
- **Private Cloud Compute (PCC)**: Apple's server system for Apple Intelligence requests too large for the device, built so request data is not kept and devices only talk to servers running publicly logged software.
- **AICore**: The Android system service that holds Gemini Nano, updates it and runs it for apps through the ML Kit GenAI APIs, without direct internet access.
- **Remote attestation**: A server proving to a device, with hardware-backed cryptography, exactly which software it runs, so the device can refuse to send data to anything else.

## Common questions

### What is on-device AI?

On-device AI means the AI model runs on your phone, laptop or browser rather than on a company's server. It answers faster for short tasks, works offline and keeps the input on the device, but the model must be small enough to fit the device's memory and battery.

### What is an NPU, and do I need one?

An NPU is a processor block built for the math inside neural networks, and it does that work with less power than a CPU or GPU. Built-in features such as Copilot+ PC features on Windows, Apple Intelligence and Gemini Nano on Android depend on one, along with enough memory; ordinary apps and cloud AI services run without one.

### What does TOPS mean on a laptop or phone spec sheet?

TOPS is trillions of operations per second, the NPU's peak at its highest clock speed, normally measured at INT8 precision. It is useful as a threshold (Microsoft requires 40+ TOPS for a Copilot+ PC), but real speed also depends on memory, bandwidth and whether the software uses the NPU at all.

### Does Apple Intelligence send my data to the cloud?

Some of it. Requests the on-device model can handle stay on the device; larger ones go to Private Cloud Compute, which Apple says keeps no data and cannot be read even by Apple, and which researchers can inspect. Requests to ChatGPT go to OpenAI only when you allow it.

### Is on-device AI more private than cloud AI?

For the model's input, usually yes, because nothing needs to leave the device. But an app can still upload the result, a feature like Recall creates a new local record of your screen, and a well-built private cloud can offer strong guarantees too. Judge each feature by where its data goes and what is kept.

### Which phones can run Gemini Nano?

As of September 2026, Google's ML Kit list includes the Pixel 9, 10 and 11 series, the Galaxy S25 and S26 series and recent foldables, and flagships from Honor, OnePlus, OPPO, vivo, Xiaomi and others. Different phones run different Gemini Nano versions, so check the current list for your model.

## Sources
1. [Copilot+ PCs developer guide](https://learn.microsoft.com/en-us/windows/ai/npu-devices/), Microsoft Learn
2. [A guide to AI TOPS and NPU performance metrics](https://www.qualcomm.com/news/onq/2024/04/a-guide-to-ai-tops-and-npu-performance-metrics), Qualcomm
3. [Introducing Apple's On-Device and Server Foundation Models](https://machinelearning.apple.com/research/introducing-apple-foundation-models), Apple Machine Learning Research
4. [Updates to Apple's On-Device and Server Foundation Language Models](https://machinelearning.apple.com/research/apple-foundation-models-2025-updates), Apple Machine Learning Research
5. [Windows 11 specs and system requirements](https://www.microsoft.com/en-us/windows/windows-11-specifications), Microsoft
6. [Introducing Copilot+ PCs](https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/), Microsoft
7. [Retrace your steps with Recall](https://support.microsoft.com/en-us/windows/ai/ai-features/retrace-your-steps-with-recall), Microsoft Support
8. [Manage Recall for Windows clients](https://learn.microsoft.com/en-us/windows/client-management/manage-recall), Microsoft Learn
9. [Introducing Mu language model and how it enabled the agent in Windows Settings](https://blogs.windows.com/windowsexperience/2025/06/23/introducing-mu-language-model-and-how-it-enabled-the-agent-in-windows-settings/), Microsoft Windows Experience Blog
10. [Get started with Phi Silica in the Windows App SDK](https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica), Microsoft Learn
11. [Private Cloud Compute: A new frontier for AI privacy in the cloud](https://security.apple.com/blog/private-cloud-compute/), Apple Security Research
12. [Introducing Apple Intelligence for iPhone, iPad, and Mac](https://www.apple.com/newsroom/2024/06/introducing-apple-intelligence-for-iphone-ipad-and-mac/), Apple Newsroom
13. [Apple Intelligence now features Image Playground, Genmoji, Writing Tools enhancements, seamless support for ChatGPT, and visual intelligence](https://www.apple.com/newsroom/2024/12/apple-intelligence-now-features-image-playground-genmoji-and-more/), Apple Newsroom
14. [Turn on ChatGPT on iPhone](https://support.apple.com/guide/iphone/turn-on-chatgpt-iph00fd3c8c2/ios), Apple Support
15. [Apple introduces Siri AI, a profoundly more capable and personal assistant](https://www.apple.com/newsroom/2026/06/apple-introduces-siri-ai-a-profoundly-more-capable-and-personal-assistant/), Apple Newsroom
16. [Siri AI, a profoundly more capable and personal assistant, is here](https://www.apple.com/newsroom/2026/09/siri-ai-a-profoundly-more-capable-and-personal-assistant-is-here/), Apple Newsroom
17. [Apple accelerates app development with new intelligence frameworks and advanced tools](https://www.apple.com/newsroom/2026/06/apple-aids-app-development-with-new-intelligence-frameworks-and-advanced-tools/), Apple Newsroom
18. [Apple introduces M6 and M5 Ultra for a big leap in performance and AI compute](https://www.apple.com/newsroom/2026/08/apple-introduces-m6-and-m5-ultra-for-a-big-leap-in-performance-and-ai-compute/), Apple Newsroom
19. [Apple introduces M4 chip](https://www.apple.com/newsroom/2024/05/apple-introduces-m4-chip/), Apple Newsroom
20. [Google Pixel December feature drop: New Gemini AI-powered updates](https://blog.google/products-and-platforms/devices/pixel/pixel-feature-drop-december-2023/), Google
21. [Gemini Nano](https://developer.android.com/ai/gemini-nano), Android Developers
22. [Overview of the ML Kit GenAI APIs](https://developers.google.com/ml-kit/genai), Google for Developers
23. [Gemma 4: The new standard for local agentic intelligence on Android](https://android-developers.googleblog.com/2026/04/gemma-4-new-standard-for-local-agentic-intelligence.html), Android Developers Blog
24. [Private AI Compute: our next step in building private and helpful AI](https://blog.google/innovation-and-ai/products/google-private-ai-compute/), Google
25. [Get started with built-in AI](https://developer.chrome.com/docs/ai/get-started), Chrome for Developers
26. [Expanding on-device AI in Microsoft Edge: New models and APIs for the web](https://blogs.windows.com/msedgedev/2026/06/02/expanding-on-device-ai-in-microsoft-edge-new-models-and-apis-for-the-web/), Microsoft Edge Blog