Skip to main content
The Dark Art of Postmortem DebuggingRead

Solutions / Device manufacturers

The call dropped and the phone rebooted. The cause was in a memory dump.

Your devices leave bugreports, Perfetto traces, modem captures, kernel logs, memory dumps and your own system-app logs. logcat.ai reads them together and follows a failure through every layer to the line that caused it.

One field failure, traced across six kinds of fileIllustrative. Synthetic data.
  1. What the customer saw
    Your app logs
    InCallUi: call ended unexpectedly
  2. Telephony loses the radio
    Bugreport
    RILJ: RADIO_NOT_AVAILABLE, modem restarting
  3. Modem crashes
    Modem capture
    modem fatal: shared-memory channel corrupt
  4. Kernel reboots the phone
    Kernel log
    subsys-restart: resetting the SoC, modem crashed
  5. Root cause, found in memory
    Memory dump
    smem_tx: write past FIFO end, length never checked
App
Framework
Modem
Kernel
Kernel memory
Your app logs
dialer.log
Bugreport
logcat, dumpsys
Perfetto trace
threads, slices
Modem capture
baseband log
Kernel log
dmesg, last_kmsg
Memory dump
ramdump
What the customer saw
InCallUi: call ended unexpectedly
Telephony loses the radio
RILJ: RADIO_NOT_AVAILABLE, modem restarting
Modem crashes
modem fatal: shared-memory channel corrupt
Kernel reboots the phone
subsys-restart: resetting the SoC, modem crashed
Root cause, found in memory
smem_tx: write past FIFO end, length never checked
Every file is read. Each box cites its evidence, and the green line is the path from what the customer saw to the root cause.Fix: bound the write in the shared-memory driver.

Upload the whole folder the device came back with.

Field teams pull more than a bugreport: Perfetto traces, modem captures, the previous boot's kernel log, memory dumps, and logs your own system apps and vendor services write in formats nobody else has seen. No single file covers every layer. Read together, they do.

Your own text log formats work on the first upload, with no parser to write.

field_return_0412.7z10 files from one failing device
FileWhat it isAppFrame­workModemKernelMemory
bugreport-kt7_eu.zipBugreport, evidence for App, Framework, Kernel
dialer.logYour dialer app, evidence for App
telephony_vendor.logYour telephony service, evidence for Framework, Modem
call.perfetto-tracePerfetto trace, evidence for App, Framework, Kernel
modem_0412.qmdl2Modem capture, evidence for Modem
dmesg.txtKernel log, evidence for Kernel
last_kmsgPrevious boot, evidence for Modem, Kernel
ramdump_0412.7zMemory dump, evidence for Kernel, Kernel memory
tombstone_03Native crash, evidence for Framework
anr_traces.txtANR stacks, evidence for App, Framework
Read togetherevery layer covered

Illustrative bundle. Synthetic data. Filled cells mark the layers a file holds evidence for.

Hard bugs stop waiting for the two engineers who can read every layer.

Today a cross-layer failure escalates to the few people who can read all of these files, and the rest of their queue waits. With logcat.ai the device never leaves the field, and every file is read at once instead of one person reading them in turn.

Today
  1. Customer ships the device back
  2. Days in transit to the lab
  3. Reproduce on the bench
  4. Grep the bugreport
  5. Escalate to a senior engineer
  6. Correlate every file manually
  7. Fix, then validate again
With logcat.ai
Upload what the field pulled and get a cited root cause back, for a bugreport usually in minutes.
1 to 3 weeksper field failure today
Minutes to hoursto a cited root cause

Then find the build that introduced it.

A bugreport names the exact build it came from. Delta compares two builds of the same device and lists what changed between them, so you know which firmware to fix and which units to patch.

FW-36, working
reference/kt7_eu/kt7:15/KX0A.000000.000/00001:user/release-keys
FW-37, call reboots
reference/kt7_eu/kt7:15/KX0B.000000.000/00002:user/release-keys
Modem crashes0 to 3All three during a call, none on FW-36
Reboots0 to 3Each one follows a modem crash within a second
Shared-memory errors0 to 3Logged by the kernel just before each modem crash
Camera, wifi, power, storageunchangedNothing else moved between the builds
Illustrative. Synthetic data. Both fingerprints are invented.How Delta works

Questions device teams ask

What can we upload?

Bugreports, logcat, dmesg and last_kmsg, Perfetto traces, modem captures, tombstones and ANR traces, kernel memory dumps, and the logs your own system apps and vendor services write. Upload one file, or the whole folder as an archive. A folder holding a memory dump, trace or modem capture is large, so send it as a .7z to keep the larger size allowance.

Do our own log formats need a parser?

No. logcat.ai works out the format of a text log it has not seen before when you upload it, so your system-app and vendor logs are read alongside the standard ones.

How large a capture can you process?

We routinely process bugreports over 100 MB with millions of log lines, and archives of many files. A bugreport typically completes in minutes.

Can you compare across firmware versions?

Yes. Delta compares captures from different builds side by side and lists the new crashes, changed behavior, performance regressions and driver failures between them.

Do you support non-Android devices?

Yes. We read kernel logs from any embedded Linux device, including Yocto, Buildroot and OpenWrt, alongside whatever else the device writes. Full bugreport analysis needs Android, AOSP or a custom build.

Is my log data secure?

Your log data is encrypted in transit (TLS 1.3) and at rest (AES-256), processed in your own tenant, and never used to train AI models. Files are deleted after 90 days, or immediately from your dashboard. Enterprise plans offer on-premise deployment for regulated or air-gapped environments.

What's the setup process?

There's no setup required. Upload through the web interface and get results in minutes. Modem capture and memory dump analysis need to be enabled on your account; email sales@logcat.ai to turn them on. For enterprise deployments, we offer dedicated cloud instances with SSO/OIDC, custom retention, and API access.

Bring a field failure your team already chased.

Send the folder that came back with it and see the trace logcat.ai finds. Enterprise plans run on a dedicated instance or on premises, with SSO, custom retention and API access.