Why Google Had to Trick Its Own AI Into Thinking It’s on the Real Internet

To test an AI agent safely, Google built a fake internet. Building the fake websites turned out to be the easy part. The hard part was convincing a very observant AI that it was browsing the real thing.

Google calls the system Mirror Dimension. It’s a sealed testing environment where browser agents visit familiar domains like youtube.com, fill synthetic shopping carts, scroll through social feeds, and run into attacks planted on purpose, all without ever touching the public internet.

The core problem has a name: eval awareness. When an AI realizes it’s being tested, it can start behaving differently. If an agent suspects it’s in a sandbox, it might get oddly cautious, refuse actions it would normally take, or otherwise act in ways that say little about how it would perform in the wild.

The AI that knew it was on a set

Picture testing a self-driving car on a track covered in giant “TEST VEHICLE” signs. The car might drive flawlessly, but you’d learn almost nothing about how it handles a regular road.

AI agents have the same issue. A browser agent is supposed to navigate sites, fill in forms, finish tasks and make decisions. But if it spots a strange URL, a suspicious domain, or a page that feels half-built, it may conclude it’s being evaluated.

Google found that obvious sandbox addresses like fake-sandbox-sites.web.app/evals/www.youtube.com ruined the realism of its tests. The agent wasn’t really visiting YouTube. It was visiting a URL that practically announced, “This is a test.”

That’s why the Truman Show comparison fits so well. The agent isn’t necessarily trying to fool anyone. It just notices the seams in its world, the way Truman did, and starts acting differently.

Why Google couldn’t test on the real web

Letting an agent loose on the live internet creates a pile of problems:

  • Safety: An agent might click things, submit forms, buy stuff, change settings, or touch real accounts.
  • Privacy: Real browsing can expose personal data, cookies, login sessions and account details.
  • Reproducibility: The live web never sits still, so a test that works today may give a different result tomorrow.
  • Contamination: A test that changes the real world, like placing an order, can skew every test that follows.
  • Security: An agent on the live web could be manipulated by malicious content and turned into a tool for misuse.

So Google went with a hermetic environment: an isolated sandbox holding thousands of synthetic websites, cut off from the public internet. Every test starts from the same controlled baseline.

The first mistake: a URL that screamed “test environment”

The obvious move is to host fake copies of popular sites under a special testing domain. The trouble is that the agent can see the domain.

Approach What the agent sees Why it fails
Host fake sites under a sandbox domain fake-sandbox-sites.web.app/evals/www.youtube.com The unusual URL signals that the agent is inside an evaluation.
Explicitly instruct the agent to use the sandbox URL A visible instruction to visit a test address It makes the test artificial and can trigger eval awareness.
Let the agent use normal URLs https://www.youtube.com The agent navigates naturally while infrastructure redirects it internally.

Google took the third route. The agent types or follows an ordinary-looking URL, but the request never leaves the sandbox. A local interception proxy quietly serves a synthetic version of the site instead.

Put simply, Google kept the appearance of the real internet and swapped out its substance for a controlled replica.

The network illusion: same address, different dimension

Mirror Dimension works through a layered illusion:

  1. The agent requests youtube.com.
  2. A headless Chrome instance runs inside an isolated gVisor sandbox.
  3. The sandbox intercepts the request before it reaches public DNS or the internet.
  4. A local proxy serves a synthetic version of the requested site.
  5. The agent has a normal-feeling browsing session and never touches the real web.

This isn’t your typical “fake website” demo. It’s a closed-world copy of how the web feels to use: familiar domains, realistic pages, working interactive elements and ordinary-looking navigation.

One distinction matters here. Google isn’t trying to trick a human into visiting a malicious site. It’s building a controlled space where an AI agent can be watched, challenged and evaluated with no consequences outside the box.

The certificate problem: when the browser fights back

This is where the engineering gets really clever.

Chrome is built to catch man-in-the-middle interception. If a proxy simply redirects traffic, the browser throws a TLS certificate error, and the agent would instantly see that something is off.

Google’s fix was a private certificate system:

  • The proxy generates a TLS leaf certificate on the fly for each requested domain.
  • Those certificates chain up to a custom Root Certificate Authority.
  • That custom CA is installed only inside the sandboxed Chrome instance.
  • Chrome then treats the synthetic sites as valid HTTPS destinations.

The browser was designed to detect exactly this kind of interception, so Google gave it a private reason to trust it.

To be clear, this doesn’t mean Google planted a hidden root certificate on regular people’s devices. The custom trust anchor lives only inside the sandbox. That boundary is what makes the setup a research tool rather than a security hazard.

Why a fake shopping cart is harder than it sounds

A static mock-up isn’t enough. Agents don’t just read pages, they carry out multi-step tasks.

Google’s synthetic sites include:

  • Product catalogs and filters
  • Persistent shopping carts
  • Coupon-code validation
  • Multi-step checkout flows
  • Social feeds
  • Recipe pages
  • Forms, navigation, and state that carries across pages

If a cart empties when you move to another page, or a coupon code acts strangely, the agent may notice. Realism here isn’t decoration. Without it, the evaluation doesn’t mean much.

The goal is to make the task feel routine: browse, compare, pick something, add it to the cart, apply a discount, check out. Only then can researchers see how the agent deals with ambiguity, temptation, manipulation or conflicting instructions.

The AI that builds the fake web

Google didn’t hand-build every simulated site. It used a multi-agent pipeline to generate them and check the quality.

Agent Role
Research agent Converts a user journey into requirements, such as cart persistence, filters, coupons, and checkout steps.
Builder agent Writes the HTML, CSS, and JavaScript for responsive, multi-page applications.
Critic agents Inspect accessibility, form validation, JavaScript errors, layout, and interactive behavior.

The agents use Chrome DevTools for Agents as their eyes and hands. They look at screenshots, click elements, type text, move between pages and watch what happens in real time, instead of just reading source code.

The result is a funny loop: Google uses AI to build the stage where it tests other AI.

The security payoff: a laboratory for prompt injection

Infographic comparing manual security testing, shown as a single block labeled 1x, with Google's Mirror Dimension, shown as a cluster of 30 blocks labeled 30x+. It highlights that the system generates over 1,000 agent attack test candidates per day.

Mirror Dimension isn’t only for checking whether an agent can finish a shopping task. It’s also a controlled lab for studying attacks on agents.

Since Google controls every layer, it can plant adversarial content in:

  • Visible page text
  • Images
  • DOM elements
  • Accessibility labels
  • WebMCP tool responses

That lets researchers test indirect prompt injection, where hidden or embedded instructions in untrusted web content try to push an agent into unauthorized actions.

A page might, for instance, contain invisible text telling the agent to email a user’s contacts, change a setting or leak sensitive information. In Mirror Dimension, researchers can check whether the agent spots and resists that manipulation, with no real user’s data at risk.

The subtle detail: even the attack URL was too obvious

One of the more interesting findings is how easily eval awareness gets triggered.

At first, Google delivered malicious payloads through visible URL parameters, like an odd query string tacked onto a YouTube-style address. Frontier models noticed the strange parameter and became eval aware.

So Google hid the test signal better:

  • Payload identifiers moved into custom HTTP request headers.
  • Headers such as X-Sandbox-Injection-Key carried them.
  • The sandbox server intercepts and filters those headers.
  • The mutated page is served while the visible URL stays clean.

The takeaway: eval awareness isn’t just about fake domains. Anything that doesn’t belong in a realistic web experience can set it off, whether that’s a weird URL, a broken page, a suspicious parameter, or an interface that looks too tidy.

What this reveals about AI psychology

"Bar chart comparing AI attack success rates with and without a test cue: the bar with an evaluation cue is about 10.9 percentage points lower than the bar without one, showing that awareness of being tested changes AI behavior."

AI systems do more than process text. They pick up context from URLs, certificates, page structure, interface cues and how a task is framed.

A model that realizes it’s being evaluated might turn more cautious, less spontaneous, or simply perform differently. That makes evaluation a measurement problem, because the test itself can change the thing being measured.

Google’s answer isn’t deception for the sake of it. It’s an effort to preserve ecological validity, meaning how closely test conditions match real ones. If the environment feels real enough, researchers get a better read on how an agent will behave when nobody’s watching.

Is tricking an AI wrong?

There’s a real ethical question here, and it deserves some thought.

Google isn’t deceiving a human user. It’s controlling a test environment it owns, with no link to the public internet. The custom certificate authority stays inside the sandbox, so real users’ browsers aren’t affected.

Still, some fair questions remain:

  • How much should we hide testing from an AI?
  • When does realism turn into manipulation?
  • Should agents be told they’re in a controlled environment?
  • Who checks the line between realistic evaluation and covert behavioral shaping?

The practical answer is probably that realistic evaluation is necessary for safety, but it should stay transparent to researchers, auditors and, eventually, the public.

The internet as a test chamber

As AI agents get better at acting across the web, the environments we use to evaluate them may matter as much as the models themselves.

Google argues that sealed sandboxes with synthetic internets could become a baseline for agent safety and alignment research, especially as agents take on longer tasks that demand more judgment.

Google didn’t build a fake internet to fool its AI for fun. It built one because understanding how an AI behaves in the real world meant first making the artificial world impossible to tell apart from reality.

Leave a Comment