Skip to main content
The Dark Art of Postmortem DebuggingRead
Post-mortem report

The device wrote nothing down.
Its memory did.

A hard crash dumps the phone's RAM: gigabytes of frozen memory and no log files. logcat.ai treats the image as a site to dig rather than a file to read: it recovers the device's own dmesg and logcat, the screen it was showing, the reset syndrome and what every CPU was running, then names the root cause.
Exhibit 1Plan of the image
12 × 16 sq
Seven finds, all dug out of the same frozen memory. None was read from a log, because the device never wrote one.
ABCDEFGHIJKL12345678910111213141516IMEMDDR1234567
Exhibit 3Register of finds
sq · stratum
1Hardware register captureC20
2Reset syndromeE2I
3Device identityH4I
4Kernel logA13II
5Per-CPU stateD11II
6Android logcatG14III
7ScreenshotJ150
Strata 0 and I from any dump; II and III with a matching vmlinux. Illustrative. Synthetic data.
Post-mortem report · Kernel ramdump
Site RD-0000 · Report no. 0000

The device wrote nothing down.
Its memory did.

Abstract
A hard crash dumps the phone's RAM: gigabytes of frozen memory and no log files. logcat.ai treats the image as a site to dig rather than a file to read: it recovers the device's own dmesg and logcat, the screen it was showing, the reset syndrome and what every CPU was running, then names the root cause and cites the evidence.
Reading the plan
Each square is 32 MiB of the dumped image, six gigabytes in all. Hatched squares were dug: the regions the finds came out of. Filled squares were not in the dump.
Reading the section
Depth is the number of decoding layers between the raw bytes and the find. Strata 0 and I come out of any dump. Strata II and III open once a matching vmlinux is on file, and stay open for every later dump of that build.
Method
Nothing here was read from a file. Every find was recovered from the image. Struct offsets and type sizes are resolved from your kernel image's debug information, so the dig is the same across kernel builds.
Illustrative. Synthetic data. No real device, build or address appears on this page.
Exhibit 1Plan of the image
24 × 8 squares · 1 sq = 32 MiB · north is low address
The seven things this report is built on were all dug out of the same frozen memory. None of them was read from a log, because the device never wrote one.
ABCDEFGHIJKLMNOPQRSTUVWX12345678IMEMDDRnot dumped1REG2RESET3IDENTITY4DMESG5PER-CPU6LOGCAT7SCREEN
Exhibit 2Section through the finds
Depth = decoding layers below the raw bytes
surface
bottom of imagestrata II and III need a matching vmlinux0 · Raw bytes as dumpedI · Fixed structuresII · Kernel object modelIII · Compressed heap
1234567
Exhibit 3Register of finds
7 finds · 4 from any dump · 3 with a kernel image
No.FindSquareStratumRecovered fromNeeds
1Hardware register captureC20the SoC's own latched values, in the IMEM blobany dump
2Reset syndromeE2Iabort and watchdog state: NSWDogBite ×6, SWDogBark ×3any dump
3Device identityH4Iproperty area and kernel banner: reference handset, Android 15any dump
4Kernel logM5IIthe printk ring buffer, decoded in place: the full dmesgkernel image
5Per-CPU stateP3IIeach core's runqueue at the freeze, and every task's stackkernel image
6Android logcatS6IIIlogd's heap, with its zstd chunks decompressedkernel image
7ScreenshotV70the framebuffer on screen at the crash, from raw memoryany dump
Finds 4 to 6 become searchable logs of their own, each with its own dashboard. Site, squares and values are synthetic.
Section 02A real dig: one dump, one question
Deep Research · kernel ramdump
The site
Bundle: QSM560DR-B00-000-2000-01-01-ramdump.7z
Device: Quectel QSM560DR (QCM6490), Android 16
Kernel: 6.12.69-android16-6 (debug)
Input: frozen RAM image, dmesg and logcat recovered from memory
The question
Reconstruct the device's live state from this ramdump: what was on the screen, which apps and services were running, and what the last userspace logcat and kernel dmesg lines show.
Provenance
Real Deep Research · kernel ramdump · device identity substituted. A selection of the run's steps is shown.
Watch it work on a crash.
One dump, one question, from a device that wrote nothing to disk. The investigation rebuilds the whole live state from frozen memory: what was on screen, which tasks were running, the last log lines, and the fatal interconnect error the dump recorded.
Exhibit 4Field log of the dig
step · result · confidence
01
Read the NoC error record and reset tokens
fatal GEM_NOC error at 0x1d92000, NOCERR latched 10 times
low
02
Reconstruct the on-screen state
lockscreen credential just verified, mid-unlock
med
03
Enumerate running tasks from the task list
163 resident; SurfaceFlinger on CPU0; two in D state
high
04
Recover the last userspace logcat from logd in memory
keystore2 on_device_unlocked, vold CE key at 08:10:47.4
high
05
Recover the last kernel dmesg lines
qca6750 HIF: unable to wakeup PCIe endpoint, at 66.097 s
high
06
Inspect the UFS controller and its workqueue
REQ_CLKS_OFF with gate work in flight; Keymaster abort 52 ms before
high
07
Place the fault and name the leading hypothesis
a refused write on the storage-encryption path, by elimination
high
16:10 · 0:51The full reconstruction, on video
From a frozen memory image: the on-screen state, the running tasks, the last logcat and dmesg lines, and the fatal interconnect error the dump recorded.
Finding
A fatal interconnect error fired mid-unlock; the device was awake and unlocking when it hit.
Nothing survived on disk, so the whole live state was rebuilt from the RAM image. The lockscreen credential had just been verified, 163 tasks were resident, SurfaceFlinger was running on CPU0, and the last kernel line is a Wi-Fi PCIe wake failure at 66 s. The NoC error table holds a fatal record with a refused write on the storage-encryption path; the run reaches that reading by elimination and says so. Every line of this came out of memory, reconstructed rather than read from a log file.
Next step named: decode the NoC master port against the vendor table to confirm or refute the storage-path reading. Cited across the reset tokens, the NoC error table, the recovered dmesg and logcat, the task list and the per-CPU state.
Section 03The forensic floor
No kernel image required
The floor
Strata 0 and I of the section: what comes out of any dump with no kernel image at all. Identity, reset reason, bus faults, the hardware register capture and the screenshot.
Severity
Red is critical, amber is a warning. Everything else is context, and context is kept, because a hang and a panic look the same until the screen is recovered.
Illustrative. Synthetic dump; values invented.
Before it answers, it reads the frozen memory.
Every dump is first reduced to a recovered identity and a set of forensic findings (reset reason, interconnect faults, per-CPU state): the deterministic floor that stands even with no kernel image.
Exhibit 5Recovered identity
read from DRAM
Device
reference handset
recovered from: recovered property area (DDR)
Android
15 · API 35
recovered from: ro.build.version props
Build
userdebug
recovered from: build props
SoC
reference-class
recovered from: hardware / SoC prop
Kernel
6.6-android15
recovered from: kernel banner in DRAM
Reset reason
watchdog reset
recovered from: IMEM abort log
Exhibit 6What it recovers
from frozen memory
Reset
critical
Hardware watchdog reset: the kernel stopped petting the watchdog. Read from the abort log in IMEM.
Task state
critical
A kernel worker blocked on a mutex, a soft-lockup. Read from the per-CPU runqueues and task list.
Driver
warn
The mutex is held by a driver stuck in its runtime-resume path. Resolved from the loaded-module map.
Screen
info
The recovered framebuffer shows the UI frozen mid-refresh: a hang, not a kernel panic.
Logcat
info
The reconstructed userspace log shows the foreground app blocked on a call that never returned.
Collection
good
Full memory was captured after the watchdog fired, so the evidence is complete and self-consistent.
Section 04Figures from the dig
Drawn from the register, not from a log
Reading the figures
Each bar is a count of things recovered from the image, drawn to the same scale. A watchdog bite dominates the reset signatures; four of the seven findings are critical.
Traceability
Every value on this page traces back to the memory it came from: the square on the plan and the stratum in the section.
Illustrative recovered output. Synthetic dump.
Charts from a dump.
Auto-generated from the recovered evidence: the reset signatures (a watchdog bite dominates) and the finding mix, every value traced back to the memory it came from.
Fig. 1Reset / abort signatures
count · from IMEM
NSWDogBite
6
HANG_DETECT
3
SWDogBark
3
SGIReset
2
XBLSdi
1
Fig. 2Findings by severity
count · 7 findings
critical
4
warning
2
info
1
Section 05Why the site is hard to dig
Field notes
The site
A ramdump is frozen physical memory, not a log. The answer is in kernel structs, hardware register banks and the reset syndrome, none of it readable without the exact kernel debug image.
By hand
Five jobs, each measured in hours, each done again for the next dump.
Why ramdumps are hard to read.
The device never wrote a log. What it left is the whole machine, frozen, and the machine does not label its own memory.
Note 01
No logs
The device crashed before it could write anything to disk. The evidence is only in the RAM it dumped.
Note 02
Raw memory
Gigabytes of DDR and IMEM blobs. The process list, the reset reason and the fault address are buried in kernel structures, not text.
Note 03
Symbol dependence
Reading the object model needs the matching vmlinux; hand-walking a multi-gigabyte dump takes hours and expert knowledge.
Note 04
No faulting backtrace
Bus and TrustZone faults are caught after the access retired. There is often no stack pointing at the culprit.
Exhibit 7The real cost of reading a ramdump by hand
per dump
01Standing up a debugger against a multi-gigabyte dump and the right kernel image.
02Hand-walking kernel structs to recover the process list and reset reason.
03Decoding the reset syndrome and the hardware error logs by hand.
04Reconstructing what the device was doing with no logs and no backtrace.
05Cross-referencing the crash class against upstream kernel fixes yourself.
Section 06Method
Four ways the dig differs
Section, key
0 · RAW BYTESregisters, screenI · FIXED STRUCTURESidentity, resetII · KERNEL OBJECTSdmesg, CPUs, stacksIII · COMPRESSED HEAPlogcat
FLOORVMLINUXbottom of image
Method
It reads the object model straight out of frozen memory: no hardcoded offsets, everything resolved from your kernel image.
What logcat.ai does differently.
The dig goes as deep as the evidence allows. Without a kernel image it stops at the floor and still names the reset reason; with one, every stratum opens.
M1
Recovers the device's own logs and screen
Reconstructs the kernel dmesg and the full Android logcat from memory (decompressing logd's zstd chunks along the way) and carves the framebuffer to recover the screenshot that was on-screen when it crashed.
Strata II, III and 0
M2
A forensic floor with no kernel image
Device identity, reset reason, bus faults and the SoC's own hardware register capture come out of every dump with no vmlinux at all. Upload a matching kernel image and the deep object-model tables activate.
Strata 0 and I
M3
Zero hardcoded offsets
Every struct field, enum and type size is resolved from your exact vmlinux's debug info, so extraction stays correct across kernel builds and Android versions: process list, per-CPU runqueue, kernel stacks, drivers, binder graph, per-process memory.
Strata II and III
M4
Ask the dump in plain language
A research agent drills the dump through a live kernel session (read the log, resolve an address to a driver, dump bytes) grounded in the deterministic evidence, and correlates the crash class against upstream fixes.
Every stratum
Section 07Everything in the image is on the table
What we read
Reading the plan
The same site as Exhibit 1, dug wider. Twelve finds plotted by square; the stratum column in the register says how deep each one sits.
Not a limit
The examples are a sample of the breadth. If it was in RAM, it is recoverable: kernel and driver state, the logs the device never got to write, even the screen it was showing.
Illustrative. Synthetic squares and values.
A dump is the whole machine, frozen.
A ramdump is not a log: it is the device's entire memory at the instant it died. If it was in RAM, it is recoverable. The plan below is a sample of what comes out, not a limit.
Exhibit 8Plan of the image, twelve finds
24 × 8 squares · 1 sq = 32 MiB
ABCDEFGHIJKLMNOPQRSTUVWX12345678IMEMDDR123456789101112
Exhibit 9A sample of what comes out
find · square · stratum
1
Device identityH4
Model, build, security patch and kernel, from the property area and the kernel banner.
I
2
Reset reasonE2
The abort / reset syndrome and watchdog state: why the device actually went down.
I
3
Kernel logM5
The full dmesg, decoded from the kernel's printk ring buffer in memory.
II
4
Android logcatS6
The complete userspace log, carved from logd's heap and zstd-decompressed.
III
5
ScreenshotV7
The framebuffer that was on-screen at the crash, scanned out of raw memory.
0
6
Per-CPU stateP3
What every core was running at the freeze, from each CPU's runqueue.
II
7
Kernel stacksO5
The backtrace of every task, walked from the frames saved in memory.
II
8
Loaded driversJ1
The module map, so a raw address resolves to the driver behind it.
II
9
Binder IPCT8
The in-flight cross-process calls: who was waiting on whom when it hung.
III
10
Per-process memoryD7
Each process's memory map and footprint, from the kernel's own tables.
II
11
Hardware registersC2
The SoC's own register capture: the values it latched at the crash.
0
12
Anything elseW4
Network sockets, thermal state, keystore, timers: if it was in memory, it is on the table.
II
Section 08How a crash dump reaches you
Three cases
Arrival
A crash dump is written when the device could not stay up long enough to tell you why. Three of those.
How a crash dump reaches you.
Case 01
A device that crashed with no logs
It froze or rebooted and there is nothing in logcat: just the RAM it dumped. The investigation above rebuilt the whole live state (what the device was doing, which tasks were resident, the last kernel line, the fatal interconnect record) out of that frozen memory.
Case 02
A reboot or hang with no backtrace
A silent reboot, a watchdog bite, a freeze with no panic and no stack. The analysis decodes the reset syndrome and the per-CPU state, and ties the crash to the driver and the code behind it.
Case 03
A dump you need to see, not walk by hand
You do not want to hand-walk a multi-gigabyte dump in a debugger. The dashboard reads the dump's structure and draws what matters: identity, reset signatures, per-CPU state, top memory and the recovered screenshot.
Section 09Part of the logcat.ai platform
Related pages
The same dig, on other evidence. A recovered dmesg or logcat becomes a first-class log with its own dashboard, so these pages apply to what the dump gives back.
Section 10Frequently asked questions
Kernel ramdump analysis, answered
Questions
Six questions the practitioners ask before the first upload. Answers in the order they come up.
Frequently asked questions.
Q.01
What ramdump formats are supported?
Qualcomm kernel ramdumps: the directory of DDR / IMEM region blobs plus the collector manifests (load.cmm, dump_info.txt), typically shipped as a .7z. A ramdump is recognized by its contents, not its extension, so the archive format is flexible.
Q.02
Do I need the kernel image (vmlinux)?
No. A forensic floor (identity, reset reason, bus faults, hardware register capture, screenshot) comes out of any dump with no kernel image. Uploading a matching vmlinux once activates the deep object-model tables (process list, stacks, drivers, logs) and symbolizes every future dump of that same build automatically.
Q.03
It really recovers the logs and screenshot?
Yes. The kernel dmesg is decoded from the printk ring buffer, the Android logcat is carved from logd's heap (decompressing its zstd chunks), and the on-screen framebuffer is scanned out of physical memory, all from a device that wrote nothing to disk. The recovered logs become first-class and searchable, each with its own dashboard.
Q.04
How does it stay correct across kernel versions?
Nothing is hardcoded per build. Every struct offset, enum and type size is resolved from your exact vmlinux's debug information, so the same extraction works across kernel builds and Android versions.
Q.05
Can I ask questions about the dump?
Yes. A research agent drives a live kernel session over the dump (reading the log, resolving addresses to drivers, dumping memory) grounded in the deterministic evidence, and correlates the crash class against known upstream fixes.
Q.06
Is my dump data secure?
Your dump data is encrypted in transit (TLS 1.3) and at rest (AES-256), processed in your own tenant, and never used to train AI models. Kernel images you upload are stored per tenant and reused only for your dumps. Files are deleted after 90 days, or immediately from your dashboard. Enterprise plans offer on-premise deployment for regulated or air-gapped environments.
Section 11End of report
Next step
Bring the site
A Qualcomm kernel ramdump, as the .7z the collector wrote. A matching vmlinux is optional and opens every stratum.
Bring us a dump nothing else can read.
Upload a Qualcomm kernel ramdump (the memory a crashed device left behind) and get its identity, reset reason, recovered logs and screen, and a cited root cause, without ever opening a debugger.
Post-mortem report · Kernel ramdump · End