Run Your Local AI Harness 2x Faster w/ MTP 🤯 | OpenCode, Qwen, Gemma TESTED

Run Your Local AI Harness 2x Faster w/ MTP 🤯 | OpenCode, Qwen, Gemma TESTED

by xCreate · View the channel on PSI

Applied Sciences & Engineering AI — Models & Research UYQArtificial intelligence (AI)UYQFGenerative artificial intelligence
🎙 xCreate 👥 26K 📅 May 25, 2026 ⏱ 15 min 👁 7K 📄 tutorial 🧭 2026-09-16
Available in: English (current) Français

Keywords

Multi-Token Predictionspeculative decodinglocal LLMOpenCodeInferencer

Summary

The video tests Multi-Token Prediction (MTP) as a method to speed up local AI inference within coding harnesses such as OpenCode and Continue. The presenter uses the Inferencer app on an M3 Ultra 512GB system. He first demonstrates MTP with single-shot prompts on Qwen 3.6 27B, showing speed increases from 28 to 58 tokens per second when enabling MTP with a 4-bit quantized draft model. He then tests Gemma 4, achieving around 100 tokens per second with MTP enabled versus 50 without. He explains how to download MTP-optimized models from Hugging Face and configure the server, including enabling persistent prompt caching. He integrates the setup with OpenCode and Continue, showing that token generation rates remain above 100 tokens per second even when agents are actively editing files and making tool calls. The video concludes that MTP is effective not only for simple generation but also for agentic workflows with large system prompts. The presenter notes occasional reliability issues with file saving in OpenCode but attributes them to using the 4-bit quantization. Overall, the video provides practical guidance and real-world benchmarks for users wanting to accelerate local AI agents.

188 words

Critical Evaluation

Value of the Information & Strength of the Argument

The value lies in hands-on, quantitative demonstrations on a high-end system, offering real numbers for token throughput with and without MTP. The argumentation is straightforward: show the same model and prompt with MTP off versus on, and report the speedups. However, the evidence is limited to single runs, no standard deviation, and only one hardware configuration. The presenter relies on anecdotal observations (“I’ve seen it flash 113”) and does not pit against alternative speed-up techniques. Despite these shortcomings, the practical steps for configuring MTP are clearly explained, making the video useful for practitioners.

Scientific Rigor, Source Quality, Title Accuracy

Scientific rigor is moderate. The testing lacks controlled experimental design; factors like token diversity, prompt length, and temperature affect draft acceptance rates but are not controlled. Sources are almost exclusively the presenter’s own Inferencer app and models, with no external references to papers or independent benchmarks. The title accurately reflects content, as demonstrated speedups are close to 2x. The adequacy between title and content is good, but the rigor is hampered by potential conflict of interest (promoting Inferencer). No external sources are cited beyond links in the description, which point to the app and Hugging Face model pages.

206 words

Title / Content Match

The title accurately reflects the content: the video demonstrates that enabling MTP roughly doubles inference speed within agentic harnesses like OpenCode, with tests on Qwen and Gemma.

Quality & Reliability

7/10

The video provides concrete performance measurements and step-by-step configuration guides, but the testing methodology is limited to a single high-end system (M3 Ultra) without controlled repeats or statistical analysis. The content is largely vendor-specific to the Inferencer app, and lacks independent verification or citations to academic sources.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

Contribution & Novelties

The video contributes practical evidence that MTP acceleration works not only for direct prompt completion but also for agentic workflows where the harness repeatedly calls the model with large system prompts and tool-use patterns. It demonstrates that using a lower-bit (4-bit) MTP draft model can yield speed gains with minimal quality loss, provided the main model is also quantized. It also highlights the synergy of persistent prompt caching with MTP, achieving sustained high token rates in real coding tasks.

To go further:

  • Speculative decoding - Wikipedia — Background on the technique that MTP builds upon, explaining why draft models speed up generation.
  • Multi-token prediction (MTP) — The concept of training a model to predict multiple future tokens jointly, as implemented by Qwen and others. See the research paper “Better & Faster Large Language Models via Multi-token Prediction” (arXiv:2404.19737) for empirical evidence.
  • KV Cache - Wikipedia — Explanation of the cache mechanism that persistent prompt caching exploits to avoid recomputing system prompts.

161 words

Radar Profile

The scores are relatively balanced with information quantity and quality slightly above technical depth, reflecting a video that provides many real-world numbers but lacks deep algorithmic explanation. The reliability score is moderate, aligning with the informal testing methodology.

Reliability 7/10