Project overview
What it's about
Instead of relying only on benchmark leaderboards, I run my own repeatable tests to compare how different models behave for development-focused tasks.
Areas I'm working on
- Trying different model families and quantization levels
- Testing code generation in C#, Python, JavaScript, Go and SQL
- Tracking response quality, speed and incomplete generations
- Learning the limits of local GPU memory and inference settings
Where it's going
The experiments are ongoing. Any numbers or rankings I eventually publish will be tied to a specific model, test setup and date rather than presented as universal results.