Module 13 — Forensic Validation, Evidence Handling & Vendor Claim Reproduction

Por

Purpose & scope: This module gives you a complete, auditable workflow to (1) validate integrity- or security-bypass claims, (2) collect forensic-grade evidence from Android app tests, and (3) produce reproducible results that hold up under scrutiny. It covers artifacts to demand from third parties, chain-of-custody, time synchronization, redaction/PII hygiene, structured logging, replay-safe lab procedures, and the final reporting package.
Ethics & legality: All procedures must be executed within written authorization and against targets in scope. Never exfiltrate production user data. Prefer lab/test accounts and intentionally vulnerable builds when demonstrating techniques.

13.0 Learning objectives

By the end of this module you will be able to:

  • Define what evidence is required to substantiate a mobile security claim (e.g., “Play Integrity bypass,” “pinning disabled,” “root detection defeated”).
  • Collect and preserve forensic artifacts with hashes, timestamps, and environment manifests.
  • Reproduce third-party findings in a controlled lab, isolating variables (app version, device, kernel, tooling versions).
  • Validate server-side truth (nonce usage, attestation/token verification, authorization outcomes) against captured client traces.
  • Produce a final, audit-ready report with clear pass/fail verdicts and remediation/prioritization.

13.1 Evidence taxonomy — what “proves” a claim

Claim typeMinimum artifacts to accept as evidence
Attestation/Integrity bypassRaw attestation token(s) (base64/JWT/CBOR), issued nonce(s) and timestamps, server verification logs (accept/deny reasons + trace IDs), device profile (model, OS, kernel, boot state), APK hash and signing cert digest.
TLS pinning disabledProxy/PCAP showing app’s protected endpoints intercepted in a lab, app version & hash, device trust store state, note of any runtime instrumentation used, and server logs if requests reached backend.
Root/emulator/anti-debug bypassLogcat traces, proof of app behavior with/without checks (screens/video), methodology (e.g., Frida script or Objection commands) used only in lab, APK hash, device state snapshot.
Authorization/IDORFull request/response pairs, account IDs used (test accounts), server decision logs, proof no out-of-band access occurred, and redacted payloads.
Local secret exposureFiles/DB dumps from app private storage (lab device), SHA-256 of files, demonstration that data is readable without keystore decryption (if that’s the claim).

Golden rule: client-side screenshots alone are not sufficient. You need artifacts that can be re-validated server-side and reproduced.

13.2 Chain of custody & integrity of evidence

  1. Case ID & clock sync
  • Assign a unique Case ID (e.g., CASE-YYYYMMDD-XXX).
  • Sync lab hosts to NTP. Record UTC timestamps everywhere.
  1. Hash everything
  • Before analysis: sha256sum app.apk > app.apk.sha256.
  • For each artifact (pcap, mitm log, logcat, screenshots, scripts), compute SHA-256 and store alongside.
  1. Immutable storage
  • Store originals in a write-once or append-only location (e.g., S3 bucket with object lock, or a vault).
  • Work on copies in a work directory.
  1. Evidence register (CSV/JSON)
    Include: filename, SHA-256, creator, timestamp (UTC), tool version, brief description, related Case ID.

13.3 Environment manifest (make findings reproducible)

Record these before running tests:

  • App identity: package name, versionName/versionCode, APK SHA-256, signing cert SHA-256.
  • Device: model, Android version, security patch level, kernel version, bootloader state (locked/unlocked), verified boot state.
  • Lab state: emulator vs real, frida-server version, objection version, proxy (Burp/mitm) version, CA trust store changes.
  • Network: proxy IP:port, whether traffic is routed through proxy, VPN state.
  • Time: timezone (UTC), sync source, test window start–end.

Template (YAML):

case_id: CASE-2025-09-27-001
app:
  package: com.bank.app
  version_name: 12.3.0
  version_code: 123000
  apk_sha256: "..."
  signer_cert_sha256: "..."
device:
  type: emulator   # or real
  model: Pixel_7_API_34
  android_version: 14
  kernel: 5.10.0
  bootloader_locked: true
  verified_boot: green
lab:
  frida_tools: 15.3.0
  frida_server: 15.3.0
  objection: 1.11.0
  proxy: burp-2024.7
  ca_installed: user_store
time:
  tz: UTC
  start: 2025-09-27T13:05:00Z
  end: 2025-09-27T14:15:00Z

13.4 Artifact capture playbooks (step-wise)

13.4.1 Logcat capture (structured)

adb logcat -v threadtime | tee "logcat_CASE-..._$(date -u +%Y%m%dT%H%M%SZ).log"
  • Start before launching app; stop after reproducing behavior.
  • Hash the log; note relevant timestamps.

13.4.2 Proxy capture

  • Set system proxy on the device/emulator, install lab CA (user store).
  • In Burp/mitmproxy, save the entire session (.burp or .mitm).
  • Export specific flows (JSON/HTTP history) to correlate with server logs.

13.4.3 Packet capture

On the proxy box or a span port:

sudo tcpdump -i any -s 0 -w CASE-...-api.pcap host api.example.com

13.4.4 Attestation tokens & nonces

  • Capture nonce issuance (/attest/nonce) responses and token submission (/attest/verify).
  • Save raw token strings (base64/JWT), request bodies, and server responses.
  • Record server verification trace IDs and decisions.

13.4.5 App storage snapshot (lab devices only)

adb shell "run-as com.bank.app ls -l /data/data/com.bank.app/"
adb shell "run-as com.bank.app cat /data/data/com.bank.app/shared_prefs/<file>.xml" > prefs.xml
  • If run-as is unavailable, use a test build with debuggable flag or a lab-rooted device.
  • Never pull production user data.

13.5 Redaction & privacy hygiene

  • Never include real credentials, PII, or full tokens in distributed reports.
  • Store raw artifacts encrypted and redact public copies:
    • Replace values with ***REDACTED*** or hashed surrogates.
    • Use search/replace scripts to redact token values while preserving lengths.
  • Keep a mapping file (encrypted) that links redacted markers to originals if needed for audit.

13.6 Server-side truth: validating what actually happened

Why this matters: Many claims hinge on server behavior (e.g., “server accepted a request from a repackaged app” or “server trusted a bad attestation”). You must compare client traces with backend logs.

13.6.1 What to request from backend teams

  • Nonce issuance log: nonce ID, bound account/session, issue time, TTL.
  • Attestation verification log: decision, parsed claims (package, cert digest, flags), token timestamp, trace ID.
  • Auth/authorization logs: request route, principal, scopes/roles, decision and reason.
  • TLS termination logs (if relevant): SNI, cert pin status (if validated server-side).

13.6.2 Cross-checks to perform

  • Nonce single-use: ensure each nonce in artifacts appears exactly once in verification logs within TTL.
  • Package / cert digest: token’s identity fields match release manifest.
  • Decision fork: if client shows “success” but server denial in logs → the client may be showing stale UI or offline state.
  • Repackaging: calculate APK signing cert digest from provided APK; compare to server-expected digest.

13.7 Reproduction methodology (deterministic & safe)

  1. Reset environment: new emulator snapshot or wiped lab device.
  2. Install same APK (exact hash).
  3. Match toolchain versions (Frida/Objection/proxy).
  4. Follow vendor’s steps verbatim; record screen and console.
  5. Collect artifacts again (logcat, PCAP, proxy logs, tokens, nonces, server trace IDs).
  6. Stop at the same observable point (e.g., “funds transfer succeeded”).
  7. Compare results line-by-line with vendor artifacts.

If your server validation denies what vendor claims as “accepted,” you have a discrepancy to resolve: request their server logs or confirm they actually hit your production/test backend.

13.8 Attestation-specific validation (deep)

13.8.1 Play Integrity / SafetyNet style tokens

  • Verify JWT/CBOR signature chain to Google/Play root.
  • Confirm nonce equals server-issued nonce (exact bytes).
  • Check appPackageName and apkCertificateDigestSha256.
  • Interpret flags: ctsProfileMatch, basicIntegrity, evaluation type.
  • Compare token timestampMs to server receipt time (skew window, e.g., ±120s).

13.8.2 Key Attestation

  • Verify the X.509 chain to the device Keymaster root.
  • Parse attestation extension, validate challenge against nonce.
  • Ensure TEE/StrongBox claims make sense for the device model; record OS/patch level claims if present.
  • If using device binding: confirm the public key thumbprint on server matches the device key used later.

Evidence to keep: full token (encrypted at rest), parsed claims (JSON), decision, reasons, and trace ID.

13.9 Pinning/transport verification (network)

  • Use proxy/PCAP to show handshake and decrypted HTTP only when acceptable in lab.
  • If pinning is claimed “bypassed,” capture both:
    1. baseline (pinning active → failed proxy interception), and
    2. altered state (lab bypass technique → interception succeeds).
  • Ensure the intercepted host is actually the protected API (not a public, unpinned CDN).
  • Server logs should show matching requests; if not, the app could be talking to a mock or alternate host.

13.10 Structuring the final verdict (per claim)

For each claim:

  • Claim summary (verbatim).
  • Scope & environment (manifest + environment YAML).
  • Reproduction steps (numbered).
  • Evidence list (artifacts with hashes).
  • Server validation results (trace IDs, reasons).
  • Outcome: Confirmed / Partially confirmed / Not reproduced.
  • Root cause (if confirmed) and business impact.
  • Remediations (prioritized).
  • Owner & due date (who must fix, when).

13.11 Severity & prioritization rubric (mobile-specific)

SeverityTypical conditions
CriticalServer accepts high-risk action (e.g., money movement) from device/app failing attestation or from repackaged app; or pinning failure enables MitM of credentials/tokens in production.
HighAuthorization bypass/IDOR of sensitive PII; plaintext long-lived tokens over TLS (leak risk high).
MediumClient-only check bypass with no server acceptance; sensitive data logged locally but requires physical access.
LowNon-sensitive debug info leakage; cosmetic or easily mitigated UX issues.

Tie severity to verified server behavior and real exploitability, not only client screenshots.

13.12 Reporting package (audit-ready)

Deliver a single, versioned package:

  • /report/ — PDF/Markdown executive summary + technical deep dive.
  • /evidence/ — raw artifacts (pcap, burp/mitm logs, logcat, screenshots) + .sha256 files.
  • /manifest/ — environment YAML, APK, cert fingerprints, tool versions.
  • /parsers/ — optional scripts used to parse tokens/logs.
  • /appendix/ — redaction log, glossary, references.

Include a README with Case ID, contact, and verification instructions.

13.13 Example CLI snippets (ready to adapt)

  • APK signer info
apksigner verify --print-certs app.apk | tee signer.txt
sha256sum app.apk > app.apk.sha256
  • TLS peek (server cert)
echo | openssl s_client -connect api.example.com:443 -servername api.example.com 2>/dev/null | openssl x509 -noout -text > server_cert.txt
  • Attestation token pretty-print (JWT)
python3 - <<'PY'
import sys, json, base64
t = sys.stdin.read().strip().split('.')
hdr = json.loads(base64.urlsafe_b64decode(t[0] + '=='))
pl  = json.loads(base64.urlsafe_b64decode(t[1] + '=='))
print(json.dumps({"header":hdr,"payload":pl}, indent=2))
PY
  • Evidence register (create/update)
printf '%s,%s,%s,%s\n' "CASE-..." "file.pcap" "$(sha256sum file.pcap|cut -d' ' -f1)" "$(date -u +%FT%TZ)" >> evidence.csv

13.14 Communication plan (stakeholders & vendors)

  • Intake: acknowledge receipt of claim, share your evidence requirements checklist and target SLA.
  • During analysis: provide Case ID, request missing artifacts, schedule a live reproduction session if needed.
  • Closure: deliver verdict (with report), remediation plan, and offer a re-test window.
  • Disagreement handling: require server logs or allow supervised lab access to reproduce; document unresolved items with rationale.

13.15 Common pitfalls & how to avoid them

  • Accepting client screenshots without server logs → always cross-verify.
  • Missing nonce linkage → store and match nonces to a session/user; deny replays.
  • Tool version drift → lock Frida/Objection/Proxy versions in the manifest.
  • Unredacted reports → automate redaction; keep raw data encrypted offline.
  • Scope creep → tie every test to written authorization and Case ID.
  • No time sync → NTP drift can invalidate token timestamps; always log UTC.

13.16 Labs (authorized, reproducible)

Lab 13-A — Attestation claim validation

  • Given vendor artifacts (nonce, token, success screenshot), re-run in lab.
  • Parse token; verify signature & claims; compare nonce to server logs.
  • Verdict should hinge on server decision and token validity.

Deliverables: parsed token JSON, server trace ID with accept/deny reason, side-by-side timeline.

Lab 13-B — Pinning claim validation

  • Baseline: show TLS interception fails with pinning enabled.
  • Lab bypass attempt (authorized): demonstrate interception succeeds and confirm requests hit real backend (server logs).
  • Document techniques and scope limitations.

Deliverables: two PCAPs (before/after), Burp project file, server request IDs.

Lab 13-C — Repackaging denial

  • Re-sign lab APK; attempt login/attestation.
  • Server must deny due to cert digest mismatch; capture denial reason.

Deliverables: APK hashes (original vs modified), server denial logs.

13.17 Templates (drop-in)

13.17.1 Evidence request (to vendors)

  • APK + SHA-256
  • Device profile (model, OS, kernel, bootloader state)
  • Tooling versions (Frida/Objection/Proxy)
  • Logcat, Proxy/Burp project, PCAP
  • Attestation tokens + nonces + timestamps
  • Steps to reproduce (numbered)

13.17.2 Final verdict block (per finding)

Finding ID: ATT-001
Claim: Server accepts requests from devices failing ctsProfileMatch
Verdict: NOT REPRODUCED
Reason: Server-side verification log (TRACE: att-2025-09-27-001) shows nonce mismatch → token rejected. Client screenshot captured only pre-validation UI.
Impact: None in production.
Remediation: N/A; add clearer user-facing error on attestation failure.

13.18 Deliverables from Module 13

  • Forensic SOP (this module as an internal standard).
  • Environment manifest template (YAML) and evidence register (CSV/JSON).
  • Redaction toolkit (scripts/patterns) and report skeleton (Markdown/PDF).
  • Checklists: vendor intake, backend log requests, attestation verification steps.
  • Training lab: packaged VM/emulator snapshot with mock backend for team drills.

13.19 “At a glance” cheat sheet

  • If it isn’t logged server-side, it didn’t happen.
  • Nonce is the anchor — single-use, short TTL, matched to session.
  • Hash & timestamp everything (UTC).
  • Redact aggressively in shared outputs.
  • Reproduce deterministically: same APK, same tools, fresh environment.
  • Verdicts are binary (confirmed/partial/not reproduced) with reasons and trace IDs.