Skip to content
M. A. Hakbilen

ML architecture researcher

Mehmet Arda
Hakbilen

( kael valen )

I study why sequence architectures are built the way they are, rebuilding them from scratch to test which assumptions hold outside language.

0.0%
false positives across 280+ held-out noise scenarios
5–20
steps of early warning before a loss spike
100%
detection for 0.10σ–0.50σ drift
76.77
CERATA accuracy vs 59.30 for the v1 expert bank
01

Research

Loss dynamics, sequence architectures, and empirical limits.

My main focus is trainscope: a post-mortem flight recorder for LLM training loss spikes. Its CUSUM change-point detector catches gradual loss drift 5-20 steps before it turns into a full spike, validated with 0.0% false positives across 280+ noise scenarios and 100% sensitivity for 0.10σ-0.50σ drift. When failure occurs, it reconstructs which layer moved first, whether from activation kurtosis or gradient explosion.

Where trainscope watches training runs, CERATA asks what a frozen model can still learn after deployment. Learning is an API call: a FAST key-value write for a single item, a MEDIUM closed-form edit from float64 sufficient statistics, and a SLOW consolidation into frozen experts. Every write and forget is guarded: locality, reversibility, order invariance, zero-parameter routing. Forget is an exact downdate. On frozen ViT-B/16 features (CIFAR-100, 20 tasks), its closed-form readout beats the v1 expert bank with zero trained parameters: 76.77 vs 59.30.

Earlier, I closed out NOESIS, testing if Titans-style associative memory matrices could inject continual learning directly into a frozen LLM manifold. The result was negative: frozen manifolds cannot assimilate untrained vector injections without distortion (0% recall on a 50-fact benchmark; full analysis in FINDINGS.md).

System architecture & flight recorder

Simulate LR surge:

Loss ℒ(t) Kurtosis κ(t) CUSUM S_k
10.05.01.0Kurtosis Fire (16.7s lead)CUSUM Fire (9.7s lead)s10s25s43s50s56s60
Step 43 · KURTOSIS ALERT (16.7 steps early warning)Loss: 2.12

Mechanism story: Block 14 activation kurtosis spike (7.84 > 3.5 margin)

CUSUM S_k: 0.28σKurtosis: 7.8Storage: Arrow IPC
Per-block gradient L2 normStep 43
Layer 2
0.25
Layer 8
0.41
Layer 14
1.85
Layer 18
0.38

telemetry is simulated; event names stay in English

Learning after deployment: CIFAR-100, 20 tasks, frozen ViT-B/16

configaccuracy (%)
  • oracle routing (ceiling)97.46
  • cerata: closed-form ridge, 0 params76.77
  • per-task expert bank70.58
  • NCM prototype, training-free70.34
  • iCaRL64.94
  • v1 PAL-MoE59.30
02

Projects

showing 7 of 7 projects

all projects above are original work (no forks). full index on github.

03

Toolbox

machine learning
PyTorch · Triton · CUDA
systems
Rust · Go · C++
web
TypeScript · React · Next.js
environment
NixOS · Docker · Git

laptop runs NixOS. CERATA runs on a single RTX 5060: if the architecture does not fit in 8 GB, the architecture changes.

04

Contact

Write to me at mehmetardahakbilen2005@gmail.com

Research chatter, collaboration, or just to argue about state-space models.