Qwen3 mtp dense#1159

Merged

hiworldwzj merged 79 commits intomainfrom

qwen3_mtp_dense

Jan 4, 2026

Commits on Dec 30, 2025

qwen3_moe mtp

shihaobai
authored and
wangzaijun
committed
fix weight name

shihaobai
authored and
wangzaijun
committed
fix qwen3 fa3 mtp

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
fix rebase
wangzaijun
committed
mtp dense

shihaobai
authored and
wangzaijun
committed
mtp dense weight

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
fix

shihaobai
authored and
wangzaijun
committed
remove mtp norm

shihaobai
authored and
wangzaijun
committed
mtp dense

shihaobai
authored and
wangzaijun
committed
update

shihaobai
authored and
wangzaijun
committed
fix
wangzaijun
committed
fix
wangzaijun
committed
fix mtp mistral model
wangzaijun
committed
mistral mtp pre layer infer
wangzaijun
committed
fix pre layer mtp
wangzaijun
committed
fix mistral mtp weight load
wangzaijun
committed
fix
wangzaijun
committed
fix
wangzaijun
committed

Commits on Dec 31, 2025

fix mistral support fa3
wangzaijun
committed
fix weight
wangzaijun
committed
fix mtp_avg_token_per_step calcu
wangzaijun
committed
diverse_mode support mtp
wangzaijun
committed
fix init weights and init layers
wangzaijun
committed
fix init weights and init layers
wangzaijun
committed
rename mtp
wangzaijun
committed
fix cpu cache kv layer num
wangzaijun
committed
fix mem layer
wangzaijun
committed
fix bloom
wangzaijun
committed
fix added_mtp_kv_layer_num
wangzaijun
committed
fix token decode kernel for int32 overflow
wangzaijun
committed
fix mtp mode support
wangzaijun
committed

Commits on Jan 1, 2026

Commits on Jan 2, 2026

fix norm
hiworldwzj
committed
fix norm head tp
hiworldwzj
committed
fix norm
hiworldwzj
committed
fix bloom
hiworldwzj
committed
fix rmsnorm llama call
hiworldwzj
committed
fix pos embdiing
hiworldwzj
committed
fix starcoder
hiworldwzj
committed
fix rmsnorm call
hiworldwzj
committed
fix chatglm
hiworldwzj
committed
fix stablelm
hiworldwzj
committed
fix wte name
hiworldwzj
committed
fix rmsnorm
hiworldwzj
committed
fix
hiworldwzj
committed
fix
hiworldwzj
committed
fix
hiworldwzj
committed
fix mtp deepseek
hiworldwzj
committed
fix mtp mistral
hiworldwzj
committed
fix
hiworldwzj
committed
fix mtp deepseek
hiworldwzj
committed
fix
hiworldwzj
committed
fix all weights
hiworldwzj
committed

Commits on Jan 3, 2026

Commits on Jan 4, 2026

fix prefill dp banlance feature
wangzaijun
committed
add test model acc sh
wangzaijun
committed
fix
wangzaijun
committed
fix sh
wangzaijun
committed
add test_qwen2.sh
wangzaijun
committed
fix sh
wangzaijun
committed
fix sh
wangzaijun
committed
fix unittest
wangzaijun
committed
fix vitnorm params
wangzaijun
committed
fix vl weight
wangzaijun
committed