Research
::Loss dynamics, sequence architectures, and empirical limits.
My main focus is trainscope: a post-mortem flight recorder for LLM training loss spikes. Its CUSUM change-point detector catches gradual loss drift 5-20 steps before it turns into a full spike, validated with 0.0% false positives across 280+ noise scenarios and 100% sensitivity for 0.10σ-0.50σ drift. When failure occurs, it reconstructs which layer moved first, whether from activation kurtosis or gradient explosion.
Trainscope grew out of work on ENGRAM, a hybrid sequence backbone interleaving Mamba-2 SSD blocks with Gated Delta Rule blocks (3:1 ratio). It applies identical hyperparameters to 12-lead ECG, spoken commands, and sequential images, verified end-to-end across 312 numerical equivalence and streaming tests.
Earlier, I closed out NOESIS, testing if Titans-style associative memory matrices could inject continual learning directly into a frozen LLM manifold. The result was negative: frozen manifolds cannot assimilate untrained vector injections without distortion (0% recall on a 50-fact benchmark; full analysis in FINDINGS.md).
Simulate LR Surge:
Mechanism Story: Block 14 activation kurtosis spike (7.84 > 3.5 margin)
Pipeline validation: PTB-XL super-diag, val macro-AUC
Projects
::all projects above are original work (no forks). full index on github.
Toolbox
::laptop runs NixOS. ENGRAM trains on a single RTX 5060: if the architecture does not fit in 8 GB, the architecture changes.
Contact
::Write to me at mehmetardahakbilen2005@
Research chatter, collaboration, or just to argue about state-space models.