Research
Loss dynamics, sequence architectures, and empirical limits.
My main focus is trainscope: a post-mortem flight recorder for LLM training loss spikes. Its CUSUM change-point detector catches gradual loss drift 5-20 steps before it turns into a full spike, validated with 0.0% false positives across 280+ noise scenarios and 100% sensitivity for 0.10σ-0.50σ drift. When failure occurs, it reconstructs which layer moved first, whether from activation kurtosis or gradient explosion.
Where trainscope watches training runs, CERATA asks what a frozen model can still learn after deployment. Learning is an API call: a FAST key-value write for a single item, a MEDIUM closed-form edit from float64 sufficient statistics, and a SLOW consolidation into frozen experts. Every write and forget is guarded: locality, reversibility, order invariance, zero-parameter routing. Forget is an exact downdate. On frozen ViT-B/16 features (CIFAR-100, 20 tasks), its closed-form readout beats the v1 expert bank with zero trained parameters: 76.77 vs 59.30.
Earlier, I closed out NOESIS, testing if Titans-style associative memory matrices could inject continual learning directly into a frozen LLM manifold. The result was negative: frozen manifolds cannot assimilate untrained vector injections without distortion (0% recall on a 50-fact benchmark; full analysis in FINDINGS.md).
System architecture & flight recorder
Simulate LR surge:
Mechanism story: Block 14 activation kurtosis spike (7.84 > 3.5 margin)
telemetry is simulated; event names stay in English
Learning after deployment: CIFAR-100, 20 tasks, frozen ViT-B/16
- oracle routing (ceiling)97.46
- cerata: closed-form ridge, 0 params76.77
- per-task expert bank70.58
- NCM prototype, training-free70.34
- iCaRL64.94
- v1 PAL-MoE59.30
Projects
showing 7 of 7 projects
- 01active
cerata
learning after deployment on a frozen model: closed-form edits, exact forgetting
PyTorch(open) - 02PyPI
trainscope
loss-spike flight recorder for LLM training runs
FastAPI · React(open) - 03crates.io
latch-lang
a programming language of my own
Rust(open) - 04public
connor
terminal-native CI/CD runner (DAG execution, no YAML)
Rust(open) - 05active
weave
local-first, plugin-based productivity system
Tauri · React · Rust(open) - 06superseded
beyond_transformer
PULSE: early sequence-architecture prototype, kept as design record
PyTorch(open) - 07archived
nanonet
monitoring & control for distributed services, ~70k lines
Go · Rust · TS(open)
all projects above are original work (no forks). full index on github.
Toolbox
- machine learning
- PyTorch · Triton · CUDA
- systems
- Rust · Go · C++
- web
- TypeScript · React · Next.js
- environment
- NixOS · Docker · Git
laptop runs NixOS. CERATA runs on a single RTX 5060: if the architecture does not fit in 8 GB, the architecture changes.
Contact
Write to me at mehmetardahakbilen2005@
Research chatter, collaboration, or just to argue about state-space models.