Android CLI Benchmark: Testing Google's 70% Token Claim

Google says its new Android CLI cuts LLM token use by more than 70%. Google also says tasks finish 3x faster. Both numbers come from Google’s own internal tests. Nobody outside Google has published a measurement.

So I made one. I installed Android CLI 1.0. I built a real project. I counted every character an AI agent must read.

Here is the short answer. The token claim is true. I measured 87%. But the tool only helps with some tasks, not all of them. I also found a bug that matters more than the token number.

A quick word on “tokens”. An LLM does not read text as letters. It splits text into small pieces called tokens. You pay per token. So less text means less cost.

What Android CLI actually is

Many articles get this wrong, so let us be clear.

Android CLI is not adb. It is not gradlew. It is a new command-line tool from Google. It became stable at version 1.0 on May 19, 2026, at Google I/O.

The tool lets AI coding agents use the Android toolchain directly. An “agent” here means a tool like Claude Code, Codex, Gemini, or Cursor. You do not need to open Android Studio.

The 70% and 3x numbers come from Google’s announcement. InfoQ confirmed these come from internal experiments, not from public benchmarks.

There is an older tool with the same name in the legacy SDK. Check your version first:

android --version

If you see 0.7.x, you have the April preview. That is the new CLI, but an old build of it. Update it:

android update

My machine had 0.7.15232955. After the update it had 1.0.15985488. All numbers below come from 1.0.

My test setup

Everything ran on one machine:

• macOS on Apple Silicon (arm64)
• Java 23.0.2
• Android CLI 1.0.15985488
• Android SDK platform 36
• A running emulator (sdk_gphone64_arm64)

I created a new project:

android create --name "NoteBench" -o ./NoteBench empty-activity

This took 2.2 seconds. That includes downloading the android-36 platform.

One thing surprised me. The template is called “empty-activity”, but it is not empty. It creates Navigation.kt, a MainScreenViewModel, a DataRepository, and a Compose theme. It uses Navigation 3. This is a real app structure, not a blank file.

How I measured

I could not copy Google’s exact test. Google compared an agent inside Android Studio against an agent using Android CLI. I do not have that setup.

So I measured the idea behind it. How many characters must an agent read to finish the same task, with and without the CLI?

I counted characters, not tokens. I had no API key for exact token counting.

This has one limit. Characters are a good guide, but not perfect. Repeated XML text compresses well into tokens. Dense JSON with many quotes and brackets does not compress as well. This works against the CLI in my numbers. The XML side is probably a bit cheaper in tokens than its character count suggests.

So please treat my numbers as a direction, not an exact value. The direction is clear. A 97% gap does not disappear because of tokenizer differences. But do not quote “87%” as an exact token number. Exact counts change from model to model.

Here is each task.

Task 1: Understand the project

An agent needs to know the module layout and the build variants.

With the CLI, one command:

android describe

Output: 1,248 characters.

Without the CLI, the agent reads the build files:

That is an 82.5% reduction.

But the real saving is bigger than this. Here is why. android describe prints full folder paths, and it prints them five times. My test project used a very long path of 133 characters. So a few hundred characters were only my folder name.

I ran the same output with a normal path like ~/dev/NoteBench. It dropped to 703 characters. That is a 90.1% reduction. A shorter project path makes the CLI look better. I am showing the number I really measured, not the nicer one. But you should know the error points in the tool’s favour.

Watch out for one more thing. The output of describe changes after you build. Before a build it says Output listing file does not exist on disk. After a build it prints the APK path. Same command, different size. If you test this yourself, build first. Otherwise you compare two different things.

Now the important limit. android describe gives you modules and variants. It does not give you dependencies or versions. If your agent must add a library, it still opens libs.versions.toml. So describe does not replace reading build files. It answers one question only.

Task 2: Deploy and launch the app

With the CLI:

android run --apks app/build/outputs/apk/debug/app-debug.apk

Output: 119 characters, in 1.3 seconds.

Standard Install: Sent 11.35 MB (1 APK)
Launching com.example.notebench.MainActivity
Activation completed successfully

Note that it found MainActivity by itself.

Without the CLI, you need two commands:

adb install -r app/build/outputs/apk/debug/app-debug.apk
adb shell am start -n com.example.notebench/.MainActivity

The output is 97 characters. That looks smaller than 119. But this comparison is not fair. To write the second command, the agent must already know the activity name. To learn it, the agent reads the manifest. That is 785 more characters.

So the real total is 882 characters against 119. That is an 86.5% reduction.

Task 3: Inspect the UI on screen

This is the biggest saving.

With the CLI:

android layout

Output: 63 characters. Clean JSON:

[{"text":"Hello Android!","center":"[182,215]","key":3506402}]

Without the CLI:

adb shell uiautomator dump /sdcard/w.xml
adb shell cat /sdcard/w.xml

Output: 2,969 characters of XML. Every node repeats many attributes. You get checkable, checked, clickable, enabled, focusable, focused, scrollable, long-clickable, password, and selected. Most of them do not matter for the task.

That is a 97.9% reduction.

But look at that JSON again. android layout returned only the text node. It removed the whole view tree. Your agent cannot see a container, a resource ID, or a scroll position. So the output is not only shorter. It is incomplete.

I checked if a flag brings the tree back. android layout accepts -p/--pretty and -o/--output. I tested on the same screen:

--pretty only adds spaces and line breaks. It returns the same single node. No flag returns the full tree. The CLI removes the data before printing, not while printing.

One flag is very useful, though. --diff returns only the elements that changed since the last call. This is good for an agent that watches the screen react.

For most cases this trade is fine. An agent usually asks “what does the screen say?” But you should know the data is removed, not summarised.

The totals

87% is higher than Google’s claim of “more than 70%”.

I did not test the “3x faster” claim. That test needs one agent running against another agent, under controlled conditions. I do not have that setup. Anyone who quotes 3x from personal feeling is guessing. I prefer to report one number I tested properly. That is better than two numbers I only tested partly.

Where the CLI does not help

This part changes how you should use the tool.

I expected build output to use the most tokens. That was wrong. I ran a cold build:

./gradlew assembleDebug

It took 41 seconds and printed only 1,689 characters. Modern AGP with the configuration cache is already quiet. The old idea that “Gradle prints too much” is no longer true.

Then I broke the build on purpose. I added a Button without importing it. This is the most common mistake an AI agent makes. The failed build printed 2,687 characters. About 35 lines were UP-TO-DATE task messages before the real error.

To be fair to Gradle, a good agent adds -q:

./gradlew assembleDebug -q

This drops the same failure to 1,067 characters. That is 60% less. It also puts the compiler error on the first line:

e: .../MainScreen.kt:39:5 Unresolved reference 'Button'.
e: .../MainScreen.kt:39:29 @Composable invocations can only happen
   from the context of a @Composable function

If you take one practical tip from this article, take this one. Add -q to your agent’s build command. It is free. (--console=plain changed nothing in my test. Same 1,067 characters.)

So how does Android CLI help here? It does not. There is no android build command.

Run android --help and you get these commands: create, describe, docs, emulator, info, init, layout, run, screen, sdk, skills, update. Compiling stays with Gradle.

This matters a lot. Agents spend most of their time fixing compile errors. The loop is simple. Write code, build, read error, fix, build again. Android CLI does not touch this loop.

So the 87% saving is real. But it applies to setup, deployment, and screen inspection. It does not apply to the part where you spend the most money.

The bug: exit code 0 on failure

This is the most important thing I found. I have not seen any other article mention it.

First, a definition. Every command returns an “exit code” when it ends. 0 means success. Any other number means failure. Scripts and agents use this number to decide what to do next.

Android CLI 1.0 returns exit code 0 when a command fails.

android run
# Error: No apks specified. Use --apks <path1,path2,...>
# Component check failed
echo $?
# 0

This happens more than once. I reproduced it in three ways:

# 1. Point at an APK that does not exist
android run --apks /tmp/does-not-exist.apk
# Error: FileNotFoundException ... Component check failed
echo $?   # 0

# 2. Run describe outside an Android project
cd /tmp/notaproject && android describe
# Error: gradlew not found
echo $?   # 0

Each command prints a clear error for the human. Each command reports success to the shell.

For a person, this is a small problem. You read the error and you understand. For an agent, this is a real problem. Agents check the exit code. Here is the common pattern:

android run --apks app.apk && echo "deployed" || echo "failed"

This prints deployed even when the deploy failed. The agent then believes the app is running. It moves to the next step. Later it becomes confused, because nothing works.

Until Google fixes this, do not trust the exit code. Check the text output instead:

# Check the error text, not the exit code
out=$(android run --apks app.apk 2>&1)
case "$out" in
  *"check failed"*|*"Error:"*) echo "failed: $out"; exit 1 ;;
  *) echo "deployed" ;;
esac

Add a guard like this if you use Android CLI in CI or in an agent loop.

Skills install for every agent by default

Android CLI also ships “skills”. A skill is a set of instructions that teaches an agent one Android workflow. Real examples include perfetto-trace-analysis, r8-analyzer, agp-9-upgrade, and migrate-xml-views-to-jetpack-compose.

The skills are well made. The XML-to-Compose skill has 52,822 characters in 8 files. But the agent does not load all of it. The main file is only 5,434 characters, which is 10.3% of the total. The rest loads only when the agent needs it. This design saves many tokens.

Now look what happens when you install one:

android skills add migrate-xml-views-to-jetpack-compose

It installed into eleven folders:

~/.claude/skills/
~/.codex/skills/
~/.cursor/skills/
~/.copilot/skills/
~/.gemini/skills/
~/.gemini/config/skills/
~/.gemini/antigravity/skills/
~/.gemini/antigravity-cli/skills/
~/.hermes/skills/
~/.config/opencode/skills/
~/.qwen/skills/

There is no question and no warning. By default it writes settings for every AI coding tool on your machine.

You can limit this, but you must ask for it:

# Install for one agent only
android skills add migrate-xml-views-to-jetpack-compose --agent claude

# Or install inside one project instead of your home folder
android skills add migrate-xml-views-to-jetpack-compose --project ./NoteBench

Please use one of these flags. If you use several agents, the plain command changes all of them at once. android skills remove cleans up correctly, so you can undo it. But the safe option should have been the default.

Should you use it?

Yes, if you know what it does and what it does not do.

Use Android CLI for creating projects, understanding module structure, deploying to a device, and reading the screen. The savings there are large and real. android layout alone is worth the install if your agent works with the UI.

Do not expect help with compiling or with reading build errors. Gradle still does that. And that is where most of your tokens go.

Add a guard for exit codes until Google fixes the bug.

The headline number is honest. Google said more than 70%. On the tasks the tool covers, I measured 87%. But the tool covers a smaller part of your work than the headline suggests. The biggest gap is the build-and-fix loop, and that is where agents spend most of their time.

Related Articles

• Android CLI: Build Android Apps 3x Faster With Any AI Agent — what the tool is and every command, if you are new to it
• What Are AI Coding Agents? — the basics of agents like Claude Code and Codex
• Agentic AI Workflows in 2026 — how agent loops work in practice
• Android App Functions — another new Android API tested on a real device

I took all measurements on 8 August 2026. Setup: Android CLI 1.0.15985488, macOS arm64, Java 23.0.2, Android SDK 36. I counted characters with wc -c. Please repeat the commands above. I would like to know if your numbers are different.