# Intelligent apps: what they are and how to build AI into an application

> Intelligent apps build AI into their core features. How they are built, the four ways to add AI to an app, and the hard parts: data, evaluation, cost and law.

- URL: https://computese.com/artificial-intelligence-and-intelligent-apps/
- Author: Duong Quan Nguyen, CEO, Computese
- Published: 2024-10-12
- Updated: 2026-09-25
- Topics: AI & automation, Custom software

## In short
- An intelligent app builds AI into the features people already use (predictions, recommendations, language and vision) instead of adding a separate chatbot beside the product.
- Every intelligent app has three layers: data, one or more models, and application code that turns model output into checked actions. Retrieval-augmented generation (RAG) connects a language model to your own content at request time.
- Start with the least custom option that works, a prebuilt AI API or a foundation model with RAG, and fine-tune or train your own model only when an evaluation set shows the simpler option falls short.
- The hard parts are data quality, evaluation, latency and cost, and the law. The EU AI Act's transparency rules apply from August 2, 2026, and its rules for high-risk uses such as recruitment and credit scoring from December 2, 2027.
- Keep a person in the loop for decisions that change records, move money or affect people's rights, and log every correction: it is the best new test case you will get.

An intelligent app is an application with artificial intelligence built into its core features: it predicts, recommends, understands language or recognizes images inside the work people already do, instead of sending them to a chatbot bolted onto the side. Underneath, it has three layers: data, one or more models, and application code that turns model output into checked actions.

This guide covers how intelligent apps are built, including retrieval-augmented generation and on-device versus cloud inference; the four ways to add AI to an application, from a prebuilt API to your own model; examples by industry; and the hard parts that decide whether a feature survives production: data, evaluation, latency and cost, the law, and keeping a person in the loop.

## What makes an app intelligent

Traditional software follows rules a developer wrote: if an invoice total is over a limit, send it for approval. An intelligent app also relies on a model that learned a pattern from data, so it can handle input no rule anticipated, such as a scanned invoice in a layout nobody has seen before. Microsoft researchers call these [AI-infused products](https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/), and the difference shows up in three ways:

- **Some outputs are probabilities, not certainties.** A model can be wrong, so the app needs confidence thresholds, a fallback and a way for people to correct it.
- **Quality is measured, not asserted.** Unit tests prove the code runs; only an evaluation against real examples shows how often the model is right.
- **Behaviour changes when the data or the model changes,** even if nobody touches the code.

The distinction that matters most is between AI in the feature and AI beside it. An expense app with a chat window that answers policy questions has AI beside the product. An expense app whose form fills itself from a photo of the receipt, flags a likely duplicate and asks a person only when unsure has AI in the feature. The second is an intelligent app; the first is an ordinary app with a chatbot attached.

### Intelligent apps and analytics

People often search for intelligent apps and analytics together, because both run on the same data. They answer different questions at different times:

|                      | Traditional app            | Analytics and BI                 | Intelligent app                              |
| -------------------- | -------------------------- | -------------------------------- | -------------------------------------------- |
| What it does         | Stores and shows records   | Summarizes what happened         | Predicts or generates a result for this case |
| When                 | During the transaction     | Afterwards, in reports           | During the transaction                       |
| Who acts on it       | The user                   | A person reading a dashboard     | The app, with a person checking what matters |
| How it is tested     | Unit and integration tests | Data tests and reconciled totals | Those tests plus an evaluation set           |
| How it usually fails | A bug                      | A wrong number in a report       | A confident wrong answer                     |

Analytics tells a manager that returns rose last quarter. An intelligent app warns about the order likely to be returned while the customer is still on the page. A well-run analytics platform is often where an intelligent feature's training data comes from.

## What AI does inside an intelligent app

Almost every intelligent feature is one of four jobs, or a combination of them:

| Job                              | What the model produces                     | Example features                                                    |
| -------------------------------- | ------------------------------------------- | ------------------------------------------------------------------- |
| Predict                          | A score, a class or a forecast              | Fraud risk on a payment, churn risk on an account, demand next week |
| Recommend and rank               | An ordered list                             | Products, articles or next actions; search results by relevance     |
| Understand and generate language | Labels, extracted fields, summaries, drafts | Routing email, reading invoice fields, answering from documentation |
| See and hear                     | Labels, text, detected objects, transcripts | Reading receipts and forms, inspecting parts, transcribing calls    |

Large language models now handle most language jobs that once needed a separate model per task, and [multimodal models read images](https://computese.com/ai-new-frontier-solving-visual-puzzles/) as well. That does not make them the right tool for everything. Predictions from your own structured records (payments, orders, sensor readings) are usually better served by a conventional machine learning model trained on that history: it is smaller, faster, cheaper per call and easier to evaluate. A good intelligent app often uses both.

## How an intelligent app is built: data, model and application layers

Whatever the feature, the architecture has the same three layers. Keeping them separate is what lets you swap a model without rewriting the app, and test each part on its own.

### The data layer

The data layer holds what the model learns from and what it reads at run time: transactions, events, documents, images and the labels that record what the right answer was. Most of the work in an intelligent app lives here. Google's [Rules of Machine Learning](https://developers.google.com/machine-learning/guides/rules-of-ml) open with two pieces of advice that still hold: do not be afraid to launch without machine learning, and design and implement your metrics before building the model. The same guide warns about training-serving skew, where the data a model sees in production differs from what it was trained on, and recommends logging the features used at serving time so the two can be compared.

### The model layer

The model layer is one or more models behind an interface the application calls: a prebuilt API, a foundation model, a fine-tuned model or your own. Treat each one as an outside dependency, like a payment provider: pin its version, set a timeout and a budget, and write down what the app does when the model is slow, down or suddenly worse.

### The application layer

The application layer is ordinary software, and it decides whether the feature can be trusted. It builds the request (a prompt, or the input features), validates the output against a schema, applies thresholds, falls back to a rule or a person when confidence is low, presents the result so people can check and correct it, and logs what happened. A language model's output is untrusted input like any other: parse it, validate it and never execute it.

### Retrieval-augmented generation: connecting a model to your data

A foundation model knows what was in its training data, not your price list, policies or case history. Retrieval-augmented generation (RAG), introduced in a [2020 NeurIPS paper by Lewis and colleagues](https://arxiv.org/abs/2005.11401), closes that gap at request time:

1. Your documents are split into chunks, each chunk is turned into an [embedding](https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/rag-engine/rag-overview) (numbers that place similar meaning close together), and the chunks are stored in an index.
2. When a request arrives, the app searches the index, usually combining keyword and vector search and reranking the results, and keeps the few most relevant passages.
3. It sends those passages to the model with the question and an instruction to answer from them and cite them.
4. The app checks the answer and shows it with its sources.

![A phone sends a question to an app server, which pulls one matching passage from a shelf of documents and passes it with the question to a model chip. The answer returns to the phone with a link.](https://computese.com/images/blog/artificial-intelligence-and-intelligent-apps/rag.8d80a8101b-1536.webp)

*The model answers from passages fetched at request time, so updating a document updates the answer without retraining anything.*

Every major cloud offers a managed version: Azure AI Search, Amazon Bedrock Knowledge Bases and the RAG Engine on Google's Gemini Enterprise Agent Platform. Two details decide whether RAG is safe in production. Retrieval must respect permissions, so a user only receives passages they are allowed to open: [Microsoft's RAG guidance](https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview) says users and agents must only retrieve authorized content, and [Bedrock's managed knowledge bases](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) can filter documents by their access control lists at retrieval time. And the index must stay current, because stale content produces confident, stale answers.

### On-device or cloud inference

Inference, running a model on new input, can happen in the cloud or on the user's device. Decide per feature, not per app:

|              | On the device                                  | In the cloud                                            |
| ------------ | ---------------------------------------------- | ------------------------------------------------------- |
| Privacy      | Input, inference and output stay on the device | Data leaves the device; provider terms and region apply |
| Connectivity | Works without a reliable connection            | Needs the network, and adds a round trip                |
| Cost         | No server cost per call                        | Paid per call or per token                              |
| Capability   | Smaller models, shorter inputs                 | The largest models, longest context, hardest reasoning  |
| Reach        | Only recent, supported devices                 | Any device with a connection                            |

Both mobile platforms now give apps an on-device language model. Apple's [Foundation Models framework](https://developer.apple.com/documentation/foundationmodels) (iOS 26 and later) exposes the on-device model behind Apple Intelligence, with guided generation that returns typed Swift data structures and tool calling; Apple's documentation points developers to Private Cloud Compute or a server model when a task needs more reasoning or context. On Android, the [ML Kit GenAI APIs](https://developers.google.com/ml-kit/genai) run Gemini Nano through the AICore system service for summarization, proofreading, rewriting, image description, speech recognition and custom prompts, but only on a published list of recent devices.

![A phone drawn in cutaway runs a small orange model chip inside it and keeps its data behind a padlock, while a dashed line leads to a cloud and a server rack for occasional harder requests.](https://computese.com/images/blog/artificial-intelligence-and-intelligent-apps/ondevice.e7155d0501-1536.webp)

*Put the quick, private task on the device and keep the cloud for what the phone cannot do. The app needs a path for both.*

The practical pattern is hybrid: run the quick, private task on the device when it is available, and fall back to a cloud model for older devices and harder requests.

## Four ways to add AI to an app

From least to most custom, there are four ways to put a model behind a feature. Start at the top, and move down only when an evaluation shows the simpler option falls short.

| Option               | What you get                                        | What you need                               | Examples (September 2026)                                            |
| -------------------- | --------------------------------------------------- | ------------------------------------------- | -------------------------------------------------------------------- |
| Prebuilt AI API      | A trained model for a common task, called over REST | Only your input data                        | Microsoft Foundry Tools, Google Cloud Vision API, Amazon Rekognition |
| Foundation model API | A general model steered by instructions and context | Good instructions, your content, test cases | Microsoft Foundry, Amazon Bedrock, Gemini Enterprise Agent Platform  |
| Fine-tuning          | A foundation model adapted to your task or format   | Hundreds to thousands of labelled examples  | Foundry fine-tuning, Gemini supervised tuning, Bedrock customization |
| Your own model       | A model trained on your data for your prediction    | History with known outcomes, ML engineering | Amazon SageMaker AI, open-source libraries such as PyTorch           |

### Prebuilt AI APIs

Prebuilt services cover jobs that look the same in every business: speech to text, translation, reading text in images, extracting fields from documents, detecting objects or unsafe content. Microsoft's [Foundry Tools](https://learn.microsoft.com/en-us/azure/ai-services/what-are-ai-services) include Speech, Translator, Language, Document Intelligence, Vision and Content Safety. Google's [Cloud Vision API](https://docs.cloud.google.com/vision/docs) labels images, detects faces and landmarks, reads text (OCR) and tags explicit content, and [Amazon Rekognition](https://docs.aws.amazon.com/rekognition/latest/dg/what-is.html) analyzes images and video. You send data and get a result, with no training. The limit is that you get the provider's definition of the task, which may not match yours.

### Foundation model APIs

A foundation model is a large model trained on broad data, which you steer with instructions, examples, retrieved context and a required output format. The cloud platforms act as catalogues. As of September 2026, [Microsoft Foundry](https://learn.microsoft.com/en-us/azure/foundry/what-is-foundry) offers more than 10,000 models from Microsoft, OpenAI, Anthropic, Meta and others, and [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) more than 100 foundation models from providers including Amazon, Anthropic, DeepSeek, OpenAI and xAI. Google's platform, formerly Vertex AI, is now [Gemini Enterprise Agent Platform](https://cloud.google.com/products/gemini-enterprise-agent-platform). For most language features this is the default starting point, combined with RAG for your own facts. Pin the model version: a provider's upgrade can change behaviour your tests depended on.

### Fine-tuning

Fine-tuning trains a foundation model further on your own examples. [Microsoft's guidance](https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/fine-tuning-considerations) puts the typical dataset at hundreds to a few thousand high-quality examples. The gains are consistent format and tone, shorter prompts, and smaller models that match larger ones on a narrow task at lower cost and latency. The costs are representative training data, extra charges to train and host, and tuning again when your data or the base model changes. Google recommends [supervised tuning](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/supervised-tuning) for well-defined tasks with labelled data, such as classification and summarization, and [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html) adds reinforcement fine-tuning and distillation, where a smaller model learns from a larger one. Keep facts that change out of the weights: a tuned model has to be retrained to learn a new price, while a RAG index only needs the document updated.

### Your own models

Training your own model makes sense when the prediction is specific to your business and you have history with known outcomes: which payments turned out to be fraud, which parts failed inspection, how long deliveries really took. Managed platforms such as [Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html) (renamed from Amazon SageMaker on December 3, 2024) handle training and hosting, and the models themselves are usually [built with open-source libraries](https://computese.com/artificial-intelligence-with-python/). You own the whole lifecycle: features, training, evaluation, deployment and retraining.

> [!TIP]
> The cheapest intelligent feature is sometimes a rule. The first of Google's Rules of Machine Learning is not to be afraid to launch without machine learning: a simple heuristic often gets you a good part of the benefit while you collect the data a model would need.

## Intelligent app examples by industry

The same four jobs appear in every sector. What changes is the data, the cost of a wrong answer and who checks it.

| Industry              | Intelligent feature                                             | AI job             | Where a person stays involved                         |
| --------------------- | --------------------------------------------------------------- | ------------------ | ----------------------------------------------------- |
| Retail and e-commerce | Recommendations, search by meaning, demand forecasts per store  | Recommend, predict | Merchandisers set exclusions the model must respect   |
| Financial services    | A fraud score on each payment; documents read during onboarding | Predict, language  | Analysts review flagged payments; credit rules below  |
| Healthcare            | Fields extracted from referral letters; draft visit summaries   | Language, vision   | A clinician confirms anything that enters the record  |
| Manufacturing         | Visual inspection on the line; maintenance alerts from sensors  | Vision, predict    | An operator confirms rejects and schedules the work   |
| Logistics             | Arrival-time estimates; reading shipping and customs documents  | Predict, language  | Planners override estimates when they know better     |
| Professional services | Sorting inbound email; pulling key terms from contracts         | Language           | Staff work the exceptions queue                       |
| Human resources       | Matching candidates to roles                                    | Rank               | Required: this is a high-risk use under the EU AI Act |

Two rows deserve a second look before you build. The European Commission names CV-sorting software for recruitment and credit scoring that can deny someone a loan as examples of high-risk AI, along with AI safety components of regulated products. Those carry the strictest obligations, covered below.

## The hard parts: data, evaluation, latency and cost

Demos are easy. The difference between a demo and a feature people rely on sits in areas that have little to do with choosing a model.

### Data quality

Poor data produces poor models: Microsoft's fine-tuning guidance warns that unrepresentative training data leads to over-fitting, under-fitting or bias. In practice the problems are mundane: labels nobody checked, categories that changed meaning over time, a missing month, one region over-represented, superseded documents still in the index. Before any model work, find where the data lives, who owns it, how fresh it is and whether you may use it for this purpose. If the answer involves five systems and a spreadsheet, the first project is the data itself: pipelines, a governed store and metrics your teams can trust, which is [data platform](https://computese.com/services/data-platform/) work.

### Evaluation

A model is never right every time, so the questions are how often, on which cases, and whether a change made it better or worse. Answer them with an evaluation set: real inputs with known good outputs, including the awkward ones, run automatically before every change to a model, prompt, index or threshold. For a document-extraction feature, each case can be one line:

```json
{"id": "inv-0412", "input": "cases/inv-0412.pdf", "expected": {"total": "1284.50", "currency": "CAD", "due": "2026-10-15"}, "tags": ["scanned", "two pages"]}
{"id": "inv-0413", "input": "cases/inv-0413.pdf", "expected": {"is_invoice": false}, "tags": ["credit note", "must not extract"]}
```

Pick metrics that match the cost of each error. Accuracy misleads when the event you care about is rare: Google's [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall) points out that when positives are 1% of cases, a model that always predicts negative scores 99% accuracy and is useless. Precision (how many flagged cases were real) and recall (how many real cases were flagged) tell you more, and the threshold trades one against the other. For generated answers, measure groundedness (whether the answer is supported by the retrieved sources) and relevance. [Microsoft Foundry's evaluators](https://learn.microsoft.com/en-us/azure/foundry/concepts/observability) include both and can run as automated quality gates in a CI/CD pipeline. NIST's [AI Risk Management Framework](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf), released in January 2023, makes this kind of measurement one of its four core functions: govern, map, measure and manage.

![A new model version moves along a track past a row of test cards. One test card has failed, so a barrier stays down in front of the production servers.](https://computese.com/images/blog/artificial-intelligence-and-intelligent-apps/evalgate.d240915dbf-1536.webp)

*A model, prompt or data change ships only when it passes the same test cases as the version it replaces.*

After launch, keep measuring. Production monitoring tracks latency, token use, errors and quality scores, and a regular sample of real cases reviewed by people catches drift that no test set anticipated.

### Latency and cost

Foundation models are usually billed per token, so cost grows with everything you send, including retrieved passages and the conversation so far, while users expect answers in seconds. The levers, roughly in order of effect:

- **Send less.** Retrieve a few relevant passages instead of whole documents; a model's input limit is far smaller than a document library.
- **Cache what repeats.** [Prompt caching](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html), offered by Amazon Bedrock among others, reuses a long, unchanged prompt prefix such as instructions or a shared document, which cuts both latency and input token cost.
- **Batch what is not interactive.** Overnight classification or enrichment does not need real-time calls; Bedrock's [batch inference](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html) processes a file of prompts asynchronously.
- **Use the smallest model that passes the evaluation,** fine-tuned or distilled if needed, and a conventional model where the job is a prediction.
- **Stream long answers,** so people read the first words while the rest is generated.
- **Move quick tasks onto the device** where it is supported, which removes the server cost per call.

Set a spend limit and an alert per feature before launch. A retry loop or a traffic spike can multiply the bill overnight.

### Security

Language models add an attack ordinary code does not have. The OWASP Gen AI Security Project lists [prompt injection as LLM01:2025](https://genai.owasp.org/llmrisk/llm01-prompt-injection/): text the model reads, typed by a user or hidden in a web page or file it retrieves, changes its behaviour. OWASP notes that it is unclear whether any fool-proof prevention exists, so contain the damage in the application layer: give the model only the data and tools the task needs, validate its output with deterministic code, and require human approval for high-risk actions. The rest is ordinary [secure coding](https://computese.com/best-practices-for-secure-coding/).

## Privacy, regulation and the EU AI Act

### Privacy and automated decisions

Start with what data the model actually needs, and remove the rest before it leaves your systems. Then read the provider's terms for the exact service and deployment type you use. For Foundry Models sold by Azure, [Microsoft states](https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy) that prompts and completions are not used to train, retrain or improve the base models, and that Global and DataZone deployments may process prompts outside your chosen region (anywhere the model is deployed, or anywhere in the data zone). Choose the deployment that matches your residency rules.

Automated decisions about people carry their own rules. Under Article 22 of the [GDPR](https://eur-lex.europa.eu/eli/reg/2016/679/oj), people have the right not to be subject to a decision based solely on automated processing, including profiling, that has legal or similarly significant effects on them. Where such a decision is allowed because it is necessary for a contract or the person gave explicit consent, the organization must at least offer human intervention, a chance to give their view and a way to contest the decision.

### The EU AI Act risk tiers and dates

The EU AI Act ([Regulation (EU) 2024/1689](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202401689)) entered into force on August 1, 2024, and has applied in general since August 2, 2026. It sorts AI systems into four risk tiers. The Digital Omnibus on AI ([Regulation (EU) 2026/1744](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744)), in force since July 27, 2026, moved the high-risk dates later and added a ninth prohibited practice. As of September 2026, according to the [European Commission](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai) and the regulation texts:

| Risk tier    | Examples                                                                                        | What applies                                                                                    | From                                                                                     |
| ------------ | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Unacceptable | Social scoring; emotion recognition at work or school; scraping faces for recognition databases | Banned                                                                                          | February 2, 2025; the ninth ban (AI "nudification" apps) from December 2026              |
| High         | CV sorting for recruitment, credit scoring, exam scoring, AI safety components of products      | Risk management, data quality, logging, documentation, human oversight, accuracy and robustness | December 2, 2027 (Annex III uses); August 2, 2028 (AI in products such as lifts or toys) |
| Transparency | Chatbots, generated content, deep fakes                                                         | Tell people they are dealing with AI; mark generated content in a machine-readable way          | August 2, 2026 (December 2, 2026 for marking by generators already on the market)        |
| Minimal      | Spam filters, AI in video games                                                                 | No new obligations                                                                              | Not applicable                                                                           |

Two obligations already apply to most teams: since February 2, 2025, providers and deployers must take measures to ensure their staff have sufficient AI literacy, and since August 2, 2025, the companies that provide general-purpose AI models have had their own duties. Most intelligent app features, such as recommendations, search, document extraction or demand forecasts, sit in the minimal tier, with transparency duties when users talk to an AI or receive generated content. The tier follows the use, not the technology: the same ranking model is minimal-risk for products and high-risk for job candidates.

> [!IMPORTANT]
> The AI Act reaches beyond the EU. It applies to providers placing AI systems on the EU market wherever they are established, and to providers and deployers outside the EU when the system's output is used in the EU. If you have EU users, work out your tier early, and take legal advice for anything near the high-risk list.

## Keeping a person in the loop

Human review is not a failure of automation: it is how a model that is right most of the time fits into a process that must be right every time. Decide per decision, with three questions:

1. **What does a wrong answer cost?** If the output changes a record, moves money or affects someone's rights, a person approves it first. For significant decisions about people, the GDPR and, for high-risk systems, the AI Act already require that a person can step in.
2. **How confident is the model?** Send cases below a threshold to a review queue and let the rest through. The threshold comes from the evaluation set, not a guess.
3. **Can the user see and correct it?** Show suggestions as suggestions, with their source or reason, and make accepting, editing and rejecting one action each.

Article 14 of the AI Act spells out what effective oversight means for high-risk systems: the people overseeing must understand the system's capabilities and limits, stay aware of automation bias (the tendency to over-rely on its output), interpret the output correctly, and be able to disregard, override or reverse it. Those are good design goals for any intelligent feature. So are Microsoft Research's 18 guidelines for human-AI interaction, tested with 49 design practitioners against 20 AI-infused products: make clear what the system can do and how well, support efficient correction and dismissal, explain why the system did what it did, and narrow what it attempts when in doubt.

Log every override. The cases people correct are the best additions to your evaluation set, and a rising override rate is often the first sign that the model and the world have drifted apart.

## How to start building an intelligent app

1. **Choose one feature in one workflow** where a better prediction or less typing is worth real time or money, and measure how it performs today. That baseline is what the model has to beat.
2. **Check whether you need a model at all.** A rule, a better form or a feature in software you already license may be enough.
3. **Collect real examples,** including the messy ones, and turn them into an evaluation set with agreed pass thresholds.
4. **Try the least custom option first:** a prebuilt API, or a foundation model with RAG over your content. Fine-tune or train only if it fails the evaluation.
5. **Design the application layer:** output validation, confidence thresholds, a fallback when the provider is slow or down, and a review queue.
6. **Check data and law before real data flows:** where it is processed, what the provider may do with it, your AI Act tier and any automated decisions about people.
7. **Release to a small group behind a feature flag,** logging inputs, outputs, cost and latency, and compare the results with the baseline.
8. **Re-run the evaluation on every change** to the model, prompt, index or threshold, and add the cases people corrected.

If the feature belongs inside a system you are building, our [custom software development](https://computese.com/services/custom-software-development/) service plans every outside dependency, an AI model included, with a written failure mode before it is built. For assistants that answer from your own content, document processing and evaluation sets built from your real cases, see [AI and automation](https://computese.com/services/ai-automation/), where the model proposes and a rule you signed off, or a named person, approves anything that changes data.

## Key terms
- **Intelligent app**: An application with AI built into its core features, such as predictions, recommendations, language understanding or image recognition, so part of its behaviour comes from a model learned from data rather than fixed rules.
- **Inference**: Running a trained model on new input to get an output: a score, a label, extracted fields or generated text. It runs in the cloud, on your own servers or on the user's device.
- **Foundation model**: A large model trained on broad data that can be adapted to many tasks through instructions, retrieved context or fine-tuning. Large language models (LLMs) are the best-known kind.
- **Retrieval-augmented generation (RAG)**: A pattern that searches your own content for passages relevant to a request and sends them to a language model with the question, so the answer rests on current sources it can cite.
- **Embedding**: A list of numbers that represents the meaning of a piece of text or an image. Similar content gets nearby embeddings, which is what makes vector search work.
- **Fine-tuning**: Further training a pretrained model on your own labelled examples, typically hundreds to a few thousand, to change its behaviour, output format or accuracy on one task.
- **Evaluation set**: A fixed collection of real inputs with known good outputs, run against every model, prompt, index or threshold change to measure quality before release.
- **Precision and recall**: Precision is the share of the cases a model flags that are real; recall is the share of real cases it flags. Moving the decision threshold trades one against the other.
- **Prompt injection**: An attack in which text the model reads, typed by a user or hidden in a web page or file, changes the model's behaviour in ways the developer did not intend.
- **Human in the loop**: A design in which a person reviews, approves or can override a model's output before it takes effect, usually for consequential or low-confidence cases.

## Common questions

### What is an example of an intelligent app?

A banking app that scores every card payment for fraud risk before approving it, a shop that ranks products for each visitor, an expense tool that reads the total and date from a photo of a receipt, or a field-service app that reads a meter from a picture. In each one the AI is part of the feature, not a separate chat window.

### What is the difference between an intelligent app and a traditional app?

A traditional app follows rules a developer wrote. An intelligent app also relies on a model learned from data, so some outputs are probabilities or generated results that can be wrong. That changes how you test it (with an evaluation set, not only unit tests) and how you design it (with thresholds, fallbacks and human review).

### How are intelligent apps related to analytics?

Analytics reports what happened, usually to a person reading a dashboard later. An intelligent app uses the same data to predict or suggest something inside the workflow, at the moment a decision is made. Both depend on clean, well-governed data, which is why data work often comes first.

### Do I need to train my own AI model to build an intelligent app?

Usually not. Prebuilt APIs cover speech, translation, vision and document extraction, and a foundation model with retrieval-augmented generation covers most language features. Train or fine-tune your own model when you have labelled data and an evaluation shows the ready-made options fall short, which is common for predictions from your own business records.

### Does the EU AI Act apply to my app if my company is outside the EU?

It can. The Act applies to providers that place AI systems on the EU market wherever they are established, and to providers and deployers outside the EU when the system's output is used in the EU. Most features fall in the minimal-risk tier, but chatbots and generated content carry transparency duties from August 2, 2026, and high-risk uses such as recruitment or credit scoring carry strict obligations from December 2, 2027. Take legal advice for your case.

### Is on-device AI better than cloud AI?

Neither is better in general. On-device models keep data on the phone, work offline and add no server cost per call, but they run only on recent supported devices and are smaller. Cloud models handle harder reasoning and longer inputs. Many apps run quick, private tasks on the device and send the rest to the cloud.

## Sources
1. [Guidelines for Human-AI Interaction (CHI 2019)](https://www.microsoft.com/en-us/research/publication/guidelines-for-human-ai-interaction/), Microsoft Research
2. [Rules of Machine Learning: Best Practices for ML Engineering](https://developers.google.com/machine-learning/guides/rules-of-ml), Google for Developers
3. [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020)](https://arxiv.org/abs/2005.11401), arXiv
4. [RAG Engine on Gemini Enterprise Agent Platform overview](https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/rag-engine/rag-overview), Google Cloud
5. [RAG and generative AI: Azure AI Search](https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview), Microsoft Learn
6. [Retrieve data and generate AI responses with Amazon Bedrock Knowledge Bases](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html), Amazon Web Services
7. [Overview of the ML Kit GenAI APIs](https://developers.google.com/ml-kit/genai), Google for Developers
8. [Foundation Models framework](https://developer.apple.com/documentation/foundationmodels), Apple Developer
9. [What are Foundry Tools?](https://learn.microsoft.com/en-us/azure/ai-services/what-are-ai-services), Microsoft Learn
10. [Cloud Vision API documentation](https://docs.cloud.google.com/vision/docs), Google Cloud
11. [What is Amazon Rekognition?](https://docs.aws.amazon.com/rekognition/latest/dg/what-is.html), Amazon Web Services
12. [What is Microsoft Foundry?](https://learn.microsoft.com/en-us/azure/foundry/what-is-foundry), Microsoft Learn
13. [Amazon Bedrock overview](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html), Amazon Web Services
14. [Gemini Enterprise Agent Platform (formerly Vertex AI)](https://cloud.google.com/products/gemini-enterprise-agent-platform), Google Cloud
15. [Microsoft Foundry fine-tuning considerations](https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/fine-tuning-considerations), Microsoft Learn
16. [About supervised fine-tuning for Gemini models](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/supervised-tuning), Google Cloud
17. [Customize your model to improve its performance for your use case](https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html), Amazon Web Services
18. [What is Amazon SageMaker AI?](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html), Amazon Web Services
19. [Classification: Accuracy, recall, precision, and related metrics](https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall), Google Machine Learning Crash Course
20. [Observability in generative AI](https://learn.microsoft.com/en-us/azure/foundry/concepts/observability), Microsoft Learn
21. [Artificial Intelligence Risk Management Framework (AI RMF 1.0)](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf), NIST
22. [Prompt caching for faster model inference](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html), Amazon Web Services
23. [Process multiple prompts with batch inference](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html), Amazon Web Services
24. [LLM01:2025 Prompt Injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/), OWASP Gen AI Security Project
25. [Data, privacy, and security for Foundry Models sold by Azure in Microsoft Foundry](https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy), Microsoft Learn
26. [Regulation (EU) 2016/679 (General Data Protection Regulation)](https://eur-lex.europa.eu/eli/reg/2016/679/oj), EUR-Lex
27. [AI Act](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai), European Commission
28. [Regulation (EU) 2024/1689 (Artificial Intelligence Act)](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202401689), EUR-Lex
29. [Regulation (EU) 2026/1744 (Digital Omnibus on AI)](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744), EUR-Lex