CTS and VTS
CTS · VTS · GTS · STS · CTS-on-GSI
120 failures.
One root cause.
The test_result.xml says FAIL 120 times across 22 builds. logcat.ai reads the shape instead of the count: Delta lines the runs up build by build, each failure is correlated with the device log from its own run, the block is sorted into device config, test bug or framework regression, and the AOSP Gerrit patch is surfaced if one already exists.
24 modules down · 64 builds across, every second one
b64
Modules down, builds across, every failure a punched hole: the shape of the holes is the diagnosis, and the count never was.
Illustrative. Synthetic data.
Solutions / CTS and VTS testing
CTS · VTS · GTS · STS · CTS-on-GSI
120 failures.
One root cause.
The test_result.xml says FAIL 120 times across 22 builds. logcat.ai reads the shape instead of the count: Delta lines the runs up build by build, each failure is correlated with the device log from its own run, the block is sorted into device config, test bug or framework regression, and the AOSP Gerrit patch is surfaced if one already exists.
module / build
b01b09b17b25b33b41b49b57b64
Modules down, builds across, every failure a punched hole: the shape of the holes is the diagnosis, and the count never was.
Illustrative. Synthetic data.
02 · Three worlds
Illustrative. Synthetic data.
Three disconnected worlds. One massive workflow problem.
Every CTS/VTS failure forces engineers to manually bridge three completely separate information sources. Today, that bridging is done with grep, timestamps, and hours of Gerrit archaeology across 2M+ test cases.
World 1 · Test results
The CTS XML says FAIL. It gives you a test name, maybe a stack trace. No device context. No system state. No explanation of what actually happened on the device during the test.
<Test result="fail"
name="testAv1Decode">
<StackTrace> IllegalArgumentException:
no decoder for video/av01
name="testAv1Decode">
<StackTrace> IllegalArgumentException:
no decoder for video/av01
World 2 · Device logs
The 100MB+ bugreport contains the answer, buried somewhere in logcat, dmesg, and dumpsys output. You have to manually correlate timestamps between the XML failure and the device log to find the relevant events.
14:03:12.402 W CodecConfig
no entry for video/av01
in media_codecs_vendor.xml
correlated: 6 tests, 90 ms window
no entry for video/av01
in media_codecs_vendor.xml
correlated: 6 tests, 90 ms window
World 3 · Upstream fixes
AOSP Gerrit may already have a patch that fixes your exact failure. But finding it means manually searching commit history, reading changelogs, and cross-referencing test names with code changes. Per failure.
Change 0000000 · MERGED
media: register av01 decoder
in vendor codec config
relevance: high · ready to cherry-pick
media: register av01 decoder
in vendor codec config
relevance: high · ready to cherry-pick
logcat.ai bridges all three worlds automatically. Upload the XML and the logcat. Get correlated root causes with Gerrit patches in minutes, not hours.
03 · Before vs after
Illustrative. Synthetic data.
Two hours per module, or five minutes.
The workflow transformation for every failing CTS/VTS module. One hole is five minutes.
Before · manual investigation
Total time: 2+ hours per module
01
Run CTS suite, wait for completion
02
Grep XML for FAIL results across modules
03
Open logcat, manually search for timestamps matching each failure
04
Read through thousands of log lines looking for relevant errors
05
Manually search Gerrit for each failure pattern
06
2+ hours per module, repeated across dozens of modules
After · logcat.ai
Total time: 5 minutes per module
01
Upload CTS XML + device logcat
02
logcat.ai correlates every failure with device state automatically
03
Root causes categorized: device config, test bug, framework regression
04
Gerrit patches surfaced with status, links, and relevance
05
5 minutes per module, regardless of failure count
04 · What we detect
Illustrative. Synthetic data.
Four shapes of failure.
AI-powered categorization of test failure root causes. Each category has a shape on the card, and the shape is what gets read.
a row that fails on every build
Test code bugs
Missing capability guards, incorrect resource paths, broken assumptions in AOSP test code, with Gerrit patches that fix them.
a block: some builds, some modules
Device configuration gaps
Missing codecs, unconfigured permissions, absent hardware capabilities that cause failures unrelated to code quality.
holes from one build onward
Framework regressions
Android framework behavior changes between versions that break test expectations. Delta pinpoints the version where the regression appeared.
a column: one run, every module
Environment issues
Missing test media files, harness misconfiguration, incomplete test setup: issues that waste debugging time but have simple fixes.
05 · Supported test suites
Illustrative. Synthetic data.
Five suites, one card.
Full coverage across the Android certification ecosystem. Upload the test_result.xml from any of them alongside the device logcat from the same session.
CTS
Compatibility Test Suite
VTS
Vendor Test Suite
GTS
Google Test Suite
STS
Security Test Suite
CTS-on-GSI
CTS on Generic System Image
06 · Delta
Illustrative. Synthetic data.
Two cards, one difference.
Delta correlates CTS/VTS result XML with device logcat to connect test outcomes with runtime evidence. Compare passing vs failing builds to isolate regressions: the holes both builds share are noise, the holes only the failing build has are the regression.
b28 · passing build
12 modules × 43 tests, sampled
12 failures. All flaky, all known.
Delta
+20 new
0 fixed
12 shared
0 fixed
12 shared
b41 · failing build
same 12 modules, same 43 tests
32 failures. 20 are new, and they sit in three rows: a framework regression.
Subtract the passing card from the failing one and only the regression is left: Delta does the subtraction, then correlates the new holes with the logcat from b41.
Learn more about Delta07 · Use cases
Illustrative. Synthetic data.
Who reads the card.
How teams use CTS/VTS analysis to accelerate certification.
01
OEM certification
Pass CTS/VTS for device certification. Quickly identify whether failures are device issues or known AOSP test bugs, with Gerrit patches ready to cherry-pick.
a row: known test bug, patch ready
02
ODM validation
Validate custom ROMs and BSP integrations against CTS requirements across multiple device variants. Delta compares results across variants to isolate BSP-specific failures.
two columns: one variant, every module
03
QA regression testing
Catch regressions introduced by firmware updates before they reach certification. Delta compares pre- and post-update CTS runs to flag new failures instantly.
right half only: after the update
08 · Analyzers
Part of the logcat.ai platform
Analyzers behind the triage.
Every artifact a failed test drops has its own analyzer to drill into. CTS/VTS analysis uses the same Deep Research and Delta engines that power bugreport analysis, dmesg debugging, and cross-build comparison across the entire logcat.ai platform.
Bugreport Analyzer
Crashes, ANRs, and 30+ sections from an Android bugreport.
Logcat Analyzer
Root cause from a raw logcat stream, cited to the line.
Dmesg Analyzer
Kernel panics, oopses, and driver faults from dmesg.
Bundle Analysis
Cross-file correlation across a whole diagnostic archive.
Perfetto Analyzer
Jank, startup, and scheduling from a Perfetto trace.
Ramdump Analyzer
Crash root cause from a kernel ramdump, no logs needed.
09 · FAQ
Frequently asked questions.
Common questions about CTS/VTS test analysis.
01
What CTS result formats do you support?
XML result files from CTS, VTS, GTS, STS, and CTS-on-GSI. Upload the test_result.xml alongside the device logcat from the test session.
02
How does Gerrit search work?
Deep Research cross-references test failures against AOSP's public commit history to determine whether a fix already exists. If a relevant patch is found, merged or in review, it surfaces in the analysis report with status, authorship, and a direct link. You get the answer without the manual Gerrit archaeology.
03
Can I upload just the CTS XML without logcat?
Yes. You'll get test result analysis, but without logcat correlation the root cause analysis will be limited to what's in the XML (failure messages and stack traces). For full root cause analysis, upload both the XML and the device logcat.
04
Does this work for custom test suites?
Currently optimized for standard Android test suites (CTS/VTS/GTS/STS). Custom test frameworks can be uploaded as unstructured text.
05
How large can the test result files be?
Up to 250MB per file, and far higher for the formats that are large by nature. Large CTS result XMLs with thousands of test cases are supported.
10 · Get started
Illustrative. Synthetic data.
Stop bridging three worlds manually.
Upload your CTS/VTS results and device logs. Get correlated root causes with Gerrit patches, in minutes, not hours.