The question behind the work.
How can an autoregressive transformer be trained and decoded efficiently while preserving the same logits as full-prefix inference?
Dataset & environment
To be selected: a small, licensed public text corpus with documented train/validation split.
From the idea to the method.
- Implement a small decoder-only transformer from scratch in PyTorch.
- Train a next-token predictor on a small public text corpus with masked attention.
- Implement a per-layer key/value cache for incremental autoregressive inference.
- Verify cached-vs-uncached logits to within numerical tolerance.
- Measure memory, tokens-per-second and generation latency with consistent settings.
What was implemented.
- Planned implementation only. No completed code or experiments have been claimed.
How the work is evaluated.
Planned; no implementation, model, results or benchmarks are available yet.
What is documented.
Not started. This is a technical roadmap, not a completed project.
A foundation for the next experiment.
- Next milestones: baseline, trained checkpoint, numerically validated cache, reproducible benchmark.