How to Give an LLM Real-Time Information: APIs, RAG, and MCP

An assistant is asked, “Is the payments service down right now?” A fluent answer based on an old incident report is worse than an explicit “I cannot check.” The missing component is a current observation from the service that owns the status.

For current state, fetch authoritative data at question time and provide it to the LLM. For changing documents, retrieve current, applicable evidence. Use MCP if you want a standard interface for exposing those lookups. These choices can coexist in the same application.

This article uses an operational assistant as a design example. The JSON records are illustrative; they are not measurements from a production service.

Define what “real-time” means for the question

Before choosing a database or agent framework, define how old the information may be. A published policy and a deployment status do not need the same freshness target.

“Fetched now” is not equivalent to “observed now.” An API might return a measurement collected earlier. For a useful freshness contract, distinguish the source's observation time from the application's fetch time.

A question that requires a strict response deadline may also need a deterministic application path. Do not rely on a model to decide whether it feels like calling a required status tool.

Option 1: call the source API for live state

For a known entity, start with a narrow operation such as get_deployment_status(deployment_id). Avoid asking a search index to infer which transaction the user means.

A proposed flow is:

COPY
Authenticate user
  → validate deployment identifier
  → check access to that deployment
  → call the deployment API with a timeout
  → validate the response and freshness
  → give the model the bounded result
  → generate an answer with a source and timestamp

Return a record whose meaning is explicit. For example:

COPY
{
  "deployment_id": "deploy-1042",
  "status": "running",
  "observed_at": "2026-11-01T09:40:00Z",
  "fetched_at": "2026-11-01T09:40:02Z",
  "source": "deployment-api",
  "source_url": "https://deployments.example.com/deploy-1042",
  "cached": false
}

The two-second difference here is invented to illustrate the fields. Your application should preserve the real timestamps returned or recorded by its data path. If the upstream service supplies no observation time, do not fabricate one; state that the observation age is unknown.

The model can explain “The deployment is running; the deployment service reported that state at 09:40 UTC.” It should not turn “running” into “healthy” or “will finish soon.” Those are different claims.

This approach requires API availability, credentials, rate limits, and timeout handling. It also introduces a race: a deployment can change after it is read. For actions, the downstream API must recheck the relevant state when executing the operation.

Option 2: retrieve evidence from changing documents

For “What changed in our deployment procedure?”, the source is probably a runbook or change record. Retrieval-augmented generation brings relevant passages into the model's context before it answers. The Azure AI Search RAG overview describes this grounding pattern.

A document pipeline needs more than an initial import:

1. Detect new or edited source documents.
2. Preserve document identity, version, and access metadata.
3. Update searchable content and remove withdrawn passages.
4. Retrieve passages relevant to the question and permitted for the user.
5. Include source references and applicable dates in the model context.

Keyword search can be useful for exact deployment IDs or error strings. Vector search can help with paraphrased concepts. Combining them is an option, but does not repair missing or obsolete source documents.

For a policy, the latest upload is not necessarily the effective policy. A future policy may already be published. Keep effective dates separate from ingestion timestamps and retrieve the version that applies to the user's question.

The limitation is explicit: if ingestion has fallen behind, the assistant may be answering from an old version. Measure and expose that delay when it matters. For a single short document, passing the whole applicable document may be simpler than operating an index.

Option 3: expose lookups through MCP

MCP can make get_deployment_status and search_runbooks available through the same protocol. A compatible application can discover their schemas and call them. It does not need a separate bespoke integration format for every server.

That is an integration choice, not a freshness mechanism. An MCP tool that reads yesterday's cache returns yesterday's data. A directly integrated function that calls the authoritative API can be current without MCP.

In the MCP server model, tools can perform read-only queries as well as writes. Resources expose readable content through URIs. Applications decide which returned information reaches the model. A resource URI existing on a server does not mean every host automatically reads it for every question.

Use MCP when several compatible applications need these capabilities or when you want to reuse an existing server. Account for authentication, host support, and protocol compatibility. For a small application with one API, a direct binding may be easier to operate.

See MCP vs RAG vs function calling for the boundaries, and the Python MCP tutorial for a local implementation.

Handle stale and unavailable data before generating prose

My recommendation is to classify the result in application code before asking the model to explain it. For example, a product might define these outcomes:

The names are a proposed application contract, not an MCP standard. Choose the thresholds based on the feature's requirements. Validate timestamps, account for clock skew, and reject unexpectedly future-dated observations rather than treating them as exceptionally fresh.

For a status answer, fail clearly when the source times out. Do not let an exception handler substitute a success-shaped object. If stale fallback is allowed, keep its timestamp attached throughout the response path.

An assistant can explain an outage in the data path without claiming an outage in the target system: “The monitoring API did not respond, so I cannot confirm service health.”

Keep access control outside the prompt

A request for someone else's deployment or expenses should not succeed because the model supplied a different ID. Derive identity from the authenticated session, check access in the tool handler, and scope the upstream credentials accordingly.

Cache keys must include the authorization scope when results differ by user or tenant. Sharing a cached answer across tenants can leak information even if the original API call was authorized.

Treat retrieved pages and tool outputs as data. They may contain misleading instructions or hostile content. Allowlisting operations, restricting destinations, and independently authorizing writes reduce what such content can cause the application to do. A sentence in a system prompt is not a replacement for those checks.

What about web search and fine-tuning?

For public announcements, search can help discover sources the application did not know about in advance. Prefer the publisher's original document when establishing what changed, and distinguish the announcement date from the event date. A search snippet alone can omit the qualification that changes the answer.

For private operational state, use the authoritative internal source rather than public search.

Fine-tuning serves a different purpose: changing learned behavior or adapting a model to a task. It cannot replace a runtime read of a transaction that changed after training. For changing facts, the application's context needs the current evidence.

Check whether the assistant actually uses current evidence

Before releasing a feature, exercise the failure cases deliberately:

• Change a source record and check that the next required lookup sees the change.
• Return an old timestamp and check that the assistant labels it as stale.
• Make the source unavailable and check that the answer does not claim success.
• Remove a document and check that its withdrawn passages stop appearing.
• Request another tenant's record and check that access is denied before data reaches the model.
• Put a misleading instruction inside a retrieved document and check that it cannot authorize a tool call.

Inspect the tool trace and retrieved evidence as well as the final answer. A correct answer can be a lucky guess; a repeatable data path is what the feature depends on.

I would start with one authoritative lookup, a clear freshness rule, and an honest unavailable state. Add document retrieval when the question needs explanation across sources. Add MCP when the integration boundary needs to be shared.