M. A. Hakbilen · research notes

ML architecture researcher

Mehmet Arda
Hakbilen

( kael valen )

I study why sequence architectures are built the way they are, rebuilding them from scratch to test which assumptions hold outside language.

Now running

TRAINSCOPE: LOSS-SPIKE FLIGHT RECORDER

CUSUM change-point detection (5-20s lead)
280+ noise scenarios tested · WandB auto-attach

pypi.org/project/trainscope →
01

Research

::

Loss dynamics, sequence architectures, and empirical limits.

My main focus is trainscope: a post-mortem flight recorder for LLM training loss spikes. Its CUSUM change-point detector catches gradual loss drift 5-20 steps before it turns into a full spike, validated with 0.0% false positives across 280+ noise scenarios and 100% sensitivity for 0.10σ-0.50σ drift. When failure occurs, it reconstructs which layer moved first, whether from activation kurtosis or gradient explosion.

Trainscope grew out of work on ENGRAM, a hybrid sequence backbone interleaving Mamba-2 SSD blocks with Gated Delta Rule blocks (3:1 ratio). It applies identical hyperparameters to 12-lead ECG, spoken commands, and sequential images, verified end-to-end across 312 numerical equivalence and streaming tests.

Earlier, I closed out NOESIS, testing if Titans-style associative memory matrices could inject continual learning directly into a frozen LLM manifold. The result was negative: frozen manifolds cannot assimilate untrained vector injections without distortion (0% recall on a 50-fact benchmark; full analysis in FINDINGS.md).

System Architecture & Flight Recorder

Simulate LR Surge:

Loss ℒ(t) Kurtosis κ(t) CUSUM S_k
10.05.01.0Kurtosis Fire (16.7s lead)CUSUM Fire (9.7s lead)s10s25s43s50s56s60
Step 43 · KURTOSIS ALERT (16.7 steps early warning)Loss: 2.12

Mechanism Story: Block 14 activation kurtosis spike (7.84 > 3.5 margin)

CUSUM S_k: 0.28σKurtosis: 7.8Storage: Arrow IPC
Per-Block Gradient L2 NormStep 43
Layer 2
0.25
Layer 8
0.41
Layer 14
1.85
Layer 18
0.38

Pipeline validation: PTB-XL super-diag, val macro-AUC

configauc
ENGRAM hybrid (SSD + GDR)0.8908
Gated DeltaNet only0.8906
ENGRAM legacy (S4D + GDR)0.8882
Mamba-2 only (SSD)0.8836
ResNet1D0.8828
small Transformer0.8769
02

Projects

::
engram
active
modality-portable SSD + Gated Delta Rule sequence backbonePyTorch
trainscope
PyPI
loss-spike flight recorder for LLM training runsFastAPI · React
latch-lang
crates.io
a programming language of my ownRust
connor
public
terminal-native CI/CD runner (DAG execution, no YAML)Rust
weave
active
local-first, plugin-based productivity systemTauri · React · Rust
beyond_transformer
superseded
PULSE: predecessor to ENGRAM, kept as design recordPyTorch
nanonet
archived
monitoring & control for distributed services, ~70k linesGo · Rust · TS

all projects above are original work (no forks). full index on github.

03

Toolbox

::
machine learningPyTorch · Triton · CUDA
systemsRust · Go · C++
webTypeScript · React · Next.js
environmentNixOS · Docker · Git

laptop runs NixOS. ENGRAM trains on a single RTX 5060: if the architecture does not fit in 8 GB, the architecture changes.

04

Contact

::

Research chatter, collaboration, or just to argue about state-space models.