ModelScope on Producthunt daily

SkyReels-V2

Fri, 01 Aug 2025 15:37:00 +0800

SkyworkAI/SkyReels-V2

SkyReels V2: Infinite-Length Film Generative Model

📑 Technical Report · 👋 Playground · 💬 Discord · 🤗 Hugging Face · 🤖 ModelScope

Welcome to the SkyReels V2 repository! Here, you’ll find the model weights and inference code for our infinite-length film generative models. To the best of our knowledge, it represents the first open-source video generative model employing AutoRegressive Diffusion-Forcing architecture that achieves the SOTA performance among publicly available models.

🔥🔥🔥 News!!

Jun 1, 2025: 🎉 We published the technical report, SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers.
May 16, 2025: 🔥 We release the inference code for video extension and start/end frame control in diffusion forcing model.
Apr 24, 2025: 🔥 We release the 720P models, SkyReels-V2-DF-14B-720P and SkyReels-V2-I2V-14B-720P. The former facilitates infinite-length autoregressive video generation, and the latter focuses on Image2Video synthesis.
Apr 21, 2025: 👋 We release the inference code and model weights of SkyReels-V2 Series Models and the video captioning model SkyCaptioner-V1 .
Apr 3, 2025: 🔥 We also release SkyReels-A2. This is an open-sourced controllable video generation framework capable of assembling arbitrary visual elements.
Feb 18, 2025: 🔥 we released SkyReels-A1. This is an open-sourced and effective framework for portrait image animation.
Feb 18, 2025: 🔥 We released SkyReels-V1. This is the first and most advanced open-source human-centric video foundation model.

🎥 Demos

The demos above showcase 30-second videos generated using our SkyReels-V2 Diffusion Forcing model.

📑 TODO List

Technical Report
Checkpoints of the 14B and 1.3B Models Series
Single-GPU & Multi-GPU Inference Code
SkyCaptioner-V1: A Video Captioning Model
Prompt Enhancer
Diffusers integration
Checkpoints of the 5B Models Series
Checkpoints of the Camera Director Models
Checkpoints of the Step & Guidance Distill Model

🚀 Quickstart

Installation

# clone the repository.
git clone https://github.com/SkyworkAI/SkyReels-V2
cd SkyReels-V2
# Install dependencies. Test environment uses Python 3.10.12.
pip install -r requirements.txt

Model Download

You can download our models from Hugging Face:

Type	Model Variant	Recommended Height/Width/Frame	Link
Diffusion Forcing	1.3B-540P	544 * 960 * 97f	🤗 Huggingface 🤖 ModelScope
5B-540P	544 * 960 * 97f	Coming Soon
5B-720P	720 * 1280 * 121f	Coming Soon
14B-540P	544 * 960 * 97f	🤗 Huggingface 🤖 ModelScope
14B-720P	720 * 1280 * 121f	🤗 Huggingface 🤖 ModelScope
Text-to-Video	1.3B-540P	544 * 960 * 97f	Coming Soon
5B-540P	544 * 960 * 97f	Coming Soon
5B-720P	720 * 1280 * 121f	Coming Soon
14B-540P	544 * 960 * 97f	🤗 Huggingface 🤖 ModelScope
14B-720P	720 * 1280 * 121f	🤗 Huggingface 🤖 ModelScope
Image-to-Video	1.3B-540P	544 * 960 * 97f	🤗 Huggingface 🤖 ModelScope
5B-540P	544 * 960 * 97f	Coming Soon
5B-720P	720 * 1280 * 121f	Coming Soon
14B-540P	544 * 960 * 97f	🤗 Huggingface 🤖 ModelScope
14B-720P	720 * 1280 * 121f	🤗 Huggingface 🤖 ModelScope
Camera Director	5B-540P	544 * 960 * 97f	Coming Soon
5B-720P	720 * 1280 * 121f	Coming Soon
14B-720P	720 * 1280 * 121f	Coming Soon

After downloading, set the model path in your generation commands:

Single GPU Inference

Diffusion Forcing for Long Video Generation

The Diffusion Forcing version model allows us to generate Infinite-Length videos. This model supports both text-to-video (T2V) and image-to-video (I2V) tasks, and it can perform inference in both synchronous and asynchronous modes. Here we demonstrate 2 running scripts as examples for long video generation. If you want to adjust the inference parameters, e.g., the duration of video, inference mode, read the Note below first.

synchronous generation for 10s video

model_id=Skywork/SkyReels-V2-DF-14B-540P
# synchronous inference
python3 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 0 \
  --base_num_frames 97 \
  --num_frames 257 \
  --overlap_history 17 \
  --prompt "A graceful white swan with a curved neck and delicate feathers swimming in a serene lake at dawn, its reflection perfectly mirrored in the still water as mist rises from the surface, with the swan occasionally dipping its head into the water to feed." \
  --addnoise_condition 20 \
  --offload \
  --teacache \
  --use_ret_steps \
  --teacache_thresh 0.3

asynchronous generation for 30s video

model_id=Skywork/SkyReels-V2-DF-14B-540P
# asynchronous inference
python3 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 5 \
  --causal_block_size 5 \
  --base_num_frames 97 \
  --num_frames 737 \
  --overlap_history 17 \
  --prompt "A graceful white swan with a curved neck and delicate feathers swimming in a serene lake at dawn, its reflection perfectly mirrored in the still water as mist rises from the surface, with the swan occasionally dipping its head into the water to feed." \
  --addnoise_condition 20 \
  --offload

Note:

If you want to run the image-to-video (I2V) task, add --image ${image_path} to your command and it is also better to use text-to-video (T2V)-like prompt which includes some descriptions of the first-frame image.

For long video generation, you can just switch the --num_frames, e.g., --num_frames 257 for 10s video, --num_frames 377 for 15s video, --num_frames 737 for 30s video, --num_frames 1457 for 60s video. The number is not strictly aligned with the logical frame number for specified time duration, but it is aligned with some training parameters, which means it may perform better. When you use asynchronous inference with causal_block_size > 1, the --num_frames should be carefully set.

You can use --ar_step 5 to enable asynchronous inference. When asynchronous inference, --causal_block_size 5 is recommended while it is not supposed to be set for synchronous generation. REMEMBER that the frame latent number inputted into the model in every iteration, e.g., base frame latent number (e.g., (97-1)//4+1=25 for base_num_frames=97) and (e.g., (237-97-(97-17)x1+17-1)//4+1=20 for base_num_frames=97, num_frames=237, overlap_history=17) for the last iteration, MUST be divided by causal_block_size. If you find it too hard to calculate and set proper values, just use our recommended setting above :). Asynchronous inference will take more steps to diffuse the whole sequence which means it will be SLOWER than synchronous mode. In our experiments, asynchronous inference may improve the instruction following and visual consistent performance.

To reduce peak VRAM, just lower the --base_num_frames, e.g., to 77 or 57, while keeping the same generative length --num_frames you want to generate. This may slightly reduce video quality, and it should not be set too small.

--addnoise_condition is used to help smooth the long video generation by adding some noise to the clean condition. Too large noise can cause the inconsistency as well. 20 is a recommended value, and you may try larger ones, but it is recommended to not exceed 50.

Generating a 540P video using the 1.3B model requires approximately 14.7GB peak VRAM, while the same resolution video using the 14B model demands around 51.2GB peak VRAM.

Video Extention

model_id=Skywork/SkyReels-V2-DF-14B-540P
# video extention
python3 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 0 \
  --base_num_frames 97 \
  --num_frames 120 \
  --overlap_history 17 \
  --prompt ${prompt} \
  --addnoise_condition 20 \
  --offload \
  --use_ret_steps \
  --teacache \
  --teacache_thresh 0.3 \
  --video_path ${video_path}

Note:

When performing video extension, you need to pass the --video_path ${video_path} parameter to specify the video to be extended.

Start/End Frame Control

model_id=Skywork/SkyReels-V2-DF-14B-540P
# start/end frame control
python3 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 0 \
  --base_num_frames 97 \
  --num_frames 97 \
  --overlap_history 17 \
  --prompt ${prompt} \
  --addnoise_condition 20 \
  --offload \
  --use_ret_steps \
  --teacache \
  --teacache_thresh 0.3 \
  --image ${image} \
  --end_image ${end_image}

Note:

When controlling the start and end frames, you need to pass the --image ${image} parameter to control the generation of the start frame and the --end_image ${end_image} parameter to control the generation of the end frame.

Text To Video & Image To Video

# run Text-to-Video Generation
model_id=Skywork/SkyReels-V2-T2V-14B-540P
python3 generate_video.py \
  --model_id ${model_id} \
  --resolution 540P \
  --num_frames 97 \
  --guidance_scale 6.0 \
  --shift 8.0 \
  --fps 24 \
  --prompt "A serene lake surrounded by towering mountains, with a few swans gracefully gliding across the water and sunlight dancing on the surface." \
  --offload \
  --teacache \
  --use_ret_steps \
  --teacache_thresh 0.3

Note:

When using an image-to-video (I2V) model, you must provide an input image using the --image ${image_path} parameter. The --guidance_scale 5.0 and --shift 3.0 is recommended for I2V model.

Generating a 540P video using the 1.3B model requires approximately 14.7GB peak VRAM, while the same resolution video using the 14B model demands around 43.4GB peak VRAM.

Prompt Enhancer

The prompt enhancer is implemented based on Qwen2.5-32B-Instruct and is utilized via the --prompt_enhancer parameter. It works ideally for short prompts, while for long prompts, it might generate an excessively lengthy prompt that could lead to over-saturation in the generative video. Note the peak memory of GPU is 64G+ if you use --prompt_enhancer. If you want to obtain the enhanced prompt separately, you can also run the prompt_enhancer script separately for testing. The steps are as follows:

1
2

cd skyreels_v2_infer/pipelines
python3 prompt_enhancer.py --prompt "A serene lake surrounded by towering mountains, with a few swans gracefully gliding across the water and sunlight dancing on the surface."

Note:

--prompt_enhancer is not allowed if using --use_usp. We recommend running the skyreels_v2_infer/pipelines/prompt_enhancer.py script first to generate enhanced prompt before enabling the --use_usp parameter.

Advanced Configuration Options

Below are the key parameters you can customize for video generation:

Parameter	Recommended Value	Description
–prompt		Text description for generating your video
–image		Path to input image for image-to-video generation
–resolution	540P or 720P	Output video resolution (select based on model type)
–num_frames	97 or 121	Total frames to generate (97 for 540P models, 121 for 720P models)
–inference_steps	50	Number of denoising steps
–fps	24	Frames per second in the output video
–shift	8.0 or 5.0	Flow matching scheduler parameter (8.0 for T2V, 5.0 for I2V)
–guidance_scale	6.0 or 5.0	Controls text adherence strength (6.0 for T2V, 5.0 for I2V)
–seed		Fixed seed for reproducible results (omit for random generation)
–offload	True	Offloads model components to CPU to reduce VRAM usage (recommended)
–use_usp	True	Enables multi-GPU acceleration with xDiT USP
–outdir	./video_out	Directory where generated videos will be saved
–prompt_enhancer	True	Expand the prompt into a more detailed description
–teacache	False	Enables teacache for faster inference
–teacache_thresh	0.2	Higher speedup will cause to worse quality
–use_ret_steps	False	Retention Steps for teacache

Diffusion Forcing Additional Parameters

Parameter	Recommended Value	Description
–ar_step	0	Controls asynchronous inference (0 for synchronous mode)
–base_num_frames	97 or 121	Base frame count (97 for 540P, 121 for 720P)
–overlap_history	17	Number of frames to overlap for smooth transitions in long videos
–addnoise_condition	20	Improves consistency in long video generation
–causal_block_size	5	Recommended when using asynchronous inference (–ar_step > 0)
–video_path		Path to input video for video extension
–end_image		Path to input image for end frame control

Multi-GPU inference using xDiT USP

We use xDiT USP to accelerate inference. For example, to generate a video with 2 GPUs, you can use the following command:

Diffusion Forcing

model_id=Skywork/SkyReels-V2-DF-14B-540P
# diffusion forcing synchronous inference
torchrun --nproc_per_node=2 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 0 \
  --base_num_frames 97 \
  --num_frames 257 \
  --overlap_history 17 \
  --prompt "A graceful white swan with a curved neck and delicate feathers swimming in a serene lake at dawn, its reflection perfectly mirrored in the still water as mist rises from the surface, with the swan occasionally dipping its head into the water to feed." \
  --addnoise_condition 20 \
  --use_usp \
  --offload \
  --seed 42

Text To Video & Image To Video

# run Text-to-Video Generation
model_id=Skywork/SkyReels-V2-T2V-14B-540P
torchrun --nproc_per_node=2 generate_video.py \
  --model_id ${model_id} \
  --resolution 540P \
  --num_frames 97 \
  --guidance_scale 6.0 \
  --shift 8.0 \
  --fps 24 \
  --offload \
  --prompt "A serene lake surrounded by towering mountains, with a few swans gracefully gliding across the water and sunlight dancing on the surface." \
  --use_usp \
  --seed 42

Note:

When using an image-to-video (I2V) model, you must provide an input image using the --image ${image_path} parameter. The --guidance_scale 5.0 and --shift 3.0 is recommended for I2V model.

Abstract

Recent advances in video generation have been driven by diffusion models and autoregressive frameworks, yet critical challenges persist in harmonizing prompt adherence, visual quality, motion dynamics, and duration: compromises in motion dynamics to enhance temporal visual quality, constrained video duration (5-10 seconds) to prioritize resolution, and inadequate shot-aware generation stemming from general-purpose MLLMs’ inability to interpret cinematic grammar, such as shot composition, actor expressions, and camera motions. These intertwined limitations hinder realistic long-form synthesis and professional film-style generation.

To address these limitations, we introduce SkyReels-V2, the world’s first infinite-length film generative model using a Diffusion Forcing framework. Our approach synergizes Multi-modal Large Language Models (MLLM), Multi-stage Pretraining, Reinforcement Learning, and Diffusion Forcing techniques to achieve comprehensive optimization. Beyond its technical innovations, SkyReels-V2 enables multiple practical applications, including Story Generation, Image-to-Video Synthesis, Camera Director functionality, and multi-subject consistent video generation through our Skyreels-A2 system.

Methodology of SkyReels-V2

The SkyReels-V2 methodology consists of several interconnected components. It starts with a comprehensive data processing pipeline that prepares various quality training data. At its core is the Video Captioner architecture, which provides detailed annotations for video content. The system employs a multi-task pretraining strategy to build fundamental video generation capabilities. Post-training optimization includes Reinforcement Learning to enhance motion quality, Diffusion Forcing Training for generating extended videos, and High-quality Supervised Fine-Tuning (SFT) stages for visual refinement. The model runs on optimized computational infrastructure for efficient training and inference. SkyReels-V2 supports multiple applications, including Story Generation, Image-to-Video Synthesis, Camera Director functionality, and Elements-to-Video Generation.

Key Contributions of SkyReels-V2

Video Captioner

SkyCaptioner-V1 serves as our video captioning model for data annotation. This model is trained on the captioning result from the base model Qwen2.5-VL-72B-Instruct and the sub-expert captioners on a balanced video data. The balanced video data is a carefully curated dataset of approximately 2 million videos to ensure conceptual balance and annotation quality. Built upon the Qwen2.5-VL-7B-Instruct foundation model, SkyCaptioner-V1 is fine-tuned to enhance performance in domain-specific video captioning tasks. To compare the performance with the SOTA models, we conducted a manual assessment of accuracy across different captioning fields using a test set of 1,000 samples. The proposed SkyCaptioner-V1 achieves the highest average accuracy among the baseline models, and show a dramatic result in the shot related fields

model	Qwen2.5-VL-7B-Ins.	Qwen2.5-VL-72B-Ins.	Tarsier2-Recap-7b	SkyCaptioner-V1
Avg accuracy	51.4%	58.7%	49.4%	76.3%
shot type	76.8%	82.5%	60.2%	93.7%
shot angle	60.0%	73.7%	52.4%	89.8%
shot position	28.4%	32.7%	23.6%	83.1%
camera motion	62.0%	61.2%	45.3%	85.3%
expression	43.6%	51.5%	54.3%	68.8%

TYPES_type	43.5%	49.7%	47.6%	82.5%
TYPES_sub_type	38.9%	44.9%	45.9%	75.4%
appearance	40.9%	52.0%	45.6%	59.3%
action	32.4%	52.0%	69.8%	68.8%
position	35.4%	48.6%	45.5%	57.5%
is_main_subject	58.5%	68.7%	69.7%	80.9%
environment	70.4%	72.7%	61.4%	70.5%
lighting	77.1%	80.0%	21.2%	76.5%

Reinforcement Learning

Inspired by the previous success in LLM, we propose to enhance the performance of the generative model by Reinforcement Learning. Specifically, we focus on the motion quality because we find that the main drawback of our generative model is:

the generative model does not handle well with large, deformable motions.
the generated videos may violate the physical law.

To avoid the degradation in other metrics, such as text alignment and video quality, we ensure the preference data pairs have comparable text alignment and video quality, while only the motion quality varies. This requirement poses greater challenges in obtaining preference annotations due to the inherently higher costs of human annotation. To address this challenge, we propose a semi-automatic pipeline that strategically combines automatically generated motion pairs and human annotation results. This hybrid approach not only enhances the data scale but also improves alignment with human preferences through curated quality control. Leveraging this enhanced dataset, we first train a specialized reward model to capture the generic motion quality differences between paired samples. This learned reward function subsequently guides the sample selection process for Direct Preference Optimization (DPO), enhancing the motion quality of the generative model.

Diffusion Forcing

We introduce the Diffusion Forcing Transformer to unlock our model’s ability to generate long videos. Diffusion Forcing is a training and sampling strategy where each token is assigned an independent noise level. This allows tokens to be denoised according to arbitrary, per-token schedules. Conceptually, this approach functions as a form of partial masking: a token with zero noise is fully unmasked, while complete noise fully masks it. Diffusion Forcing trains the model to “unmask” any combination of variably noised tokens, using the cleaner tokens as conditional information to guide the recovery of noisy ones. Building on this, our Diffusion Forcing Transformer can extend video generation indefinitely based on the last frames of the previous segment. Note that the synchronous full sequence diffusion is a special case of Diffusion Forcing, where all tokens share the same noise level. This relationship allows us to fine-tune the Diffusion Forcing Transformer from a full-sequence diffusion model.

High-Quality Supervised Fine-Tuning (SFT)

We implement two sequential high-quality supervised fine-tuning (SFT) stages at 540p and 720p resolutions respectively, with the initial SFT phase conducted immediately after pretraining but prior to reinforcement learning (RL) stage.This first-stage SFT serves as a conceptual equilibrium trainer, building upon the foundation model’s pretraining outcomes that utilized only fps24 video data, while strategically removing FPS embedding components to streamline thearchitecture. Trained with the high-quality concept-balanced samples, this phase establishes optimized initialization parameters for subsequent training processes. Following this, we execute a secondary high-resolution SFT at 720p after completing the diffusion forcing stage, incorporating identical loss formulations and the higher-quality concept-balanced datasets by the manually filter. This final refinement phase focuses on resolution increase such that the overall video quality will be further enhanced.

Performance

To comprehensively evaluate our proposed method, we construct the SkyReels-Bench for human assessment and leveraged the open-source V-Bench for automated evaluation. This allows us to compare our model with the state-of-the-art (SOTA) baselines, including both open-source and proprietary models.

Human Evaluation

For human evaluation, we design SkyReels-Bench with 1,020 text prompts, systematically assessing three dimensions: Instruction Adherence, Motion Quality, Consistency and Visual Quality. This benchmark is designed to evaluate both text-to-video (T2V) and image-to-video (I2V) generation models, providing comprehensive assessment across different generation paradigms. To ensure fairness, all models were evaluated under default settings with consistent resolutions, and no post-generation filtering was applied.

Text To Video Models

Model Name	Average	Instruction Adherence	Consistency	Visual Quality	Motion Quality
Runway-Gen3 Alpha	2.53	2.19	2.57	3.23	2.11
HunyuanVideo-13B	2.82	2.64	2.81	3.20	2.61
Kling-1.6 STD Mode	2.99	2.77	3.05	3.39	2.76
Hailuo-01	3.0	2.8	3.08	3.29	2.74
Wan2.1-14B	3.12	2.91	3.31	3.54	2.71
SkyReels-V2	3.14	3.15	3.35	3.34	2.74

The evaluation demonstrates that our model achieves significant advancements in instruction adherence (3.15) compared to baseline methods, while maintaining competitive performance in motion quality (2.74) without sacrificing the consistency (3.35).

Image To Video Models

Model	Average	Instruction Adherence	Consistency	Visual Quality	Motion Quality
HunyuanVideo-13B	2.84	2.97	2.95	2.87	2.56
Wan2.1-14B	2.85	3.10	2.81	3.00	2.48
Hailuo-01	3.05	3.31	2.58	3.55	2.74
Kling-1.6 Pro Mode	3.4	3.56	3.03	3.58	3.41
Runway-Gen4	3.39	3.75	3.2	3.4	3.37
SkyReels-V2-DF	3.24	3.64	3.21	3.18	2.93
SkyReels-V2-I2V	3.29	3.42	3.18	3.56	3.01

Our results demonstrate that both SkyReels-V2-I2V (3.29) and SkyReels-V2-DF (3.24) achieve state-of-the-art performance among open-source models, significantly outperforming HunyuanVideo-13B (2.84) and Wan2.1-14B (2.85) across all quality dimensions. With an average score of 3.29, SkyReels-V2-I2V demonstrates comparable performance to proprietary models Kling-1.6 (3.4) and Runway-Gen4 (3.39).

VBench

To objectively compare SkyReels-V2 Model against other leading open-source Text-To-Video models, we conduct comprehensive evaluations using the public benchmark V-Bench. Our evaluation specifically leverages the benchmark’s longer version prompt. For fair comparison with baseline models, we strictly follow their recommended setting for inference.

Model	Total Score	Quality Score	Semantic Score
OpenSora 2.0	81.5 %	82.1 %	78.2 %
CogVideoX1.5-5B	80.3 %	80.9 %	77.9 %
HunyuanVideo-13B	82.7 %	84.4 %	76.2 %
Wan2.1-14B	83.7 %	84.2 %	81.4 %
SkyReels-V2	83.9 %	84.7 %	80.8 %

The VBench results demonstrate that SkyReels-V2 outperforms all compared models including HunyuanVideo-13B and Wan2.1-14B, With the highest total score (83.9%) and quality score (84.7%). In this evaluation, the semantic score is slightly lower than Wan2.1-14B, while we outperform Wan2.1-14B in human evaluations, with the primary gap attributed to V-Bench’s insufficient evaluation of shot-scenario semantic adherence.

Acknowledgements

We would like to thank the contributors of Wan 2.1, XDit and Qwen 2.5 repositories, for their open research and contributions.

Citation

@misc{chen2025skyreelsv2infinitelengthfilmgenerative,
      title={SkyReels-V2: Infinite-length Film Generative Model}, 
      author={Guibin Chen and Dixuan Lin and Jiangping Yang and Chunze Lin and Junchen Zhu and Mingyuan Fan and Hao Zhang and Sheng Chen and Zheng Chen and Chengcheng Ma and Weiming Xiong and Wei Wang and Nuo Pang and Kang Kang and Zhiheng Xu and Yuzhe Jin and Yupeng Liang and Yubing Song and Peng Zhao and Boyuan Xu and Di Qiu and Debang Li and Zhengcong Fei and Yang Li and Yahui Zhou},
      year={2025},
      eprint={2504.13074},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.13074}, 
}

Qwen3-Coder

Sun, 27 Jul 2025 15:28:39 +0800

QwenLM/Qwen3-Coder

Visit our Hugging Face or ModelScope organization (click links above), search checkpoints with names starting with Qwen3-Coder-, and you will find all you need! Enjoy!

Qwen3-Coder: Agentic Coding in the World.

Introduction

Today, we’re announcing Qwen3-Coder, our most agentic code model to date. Qwen3-Coder is available in multiple sizes, but we’re excited to introduce its most powerful variant first: Qwen3-Coder-480B-A35B-Instruct — a 480B-parameter Mixture-of-Experts model with 35B active parameters, offering exceptional performance in both coding and agentic tasks. Qwen3-Coder-480B-A35B-Instruct sets new state-of-the-art results among open models on Agentic Coding, Agentic Browser-Use, and Agentic Tool-Use, comparable to Claude Sonnet.

💻 Significant Performance: among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks, achieving results comparable to Claude Sonnet;

📚 Long-context Capabilities: with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding;

🛠 Agentic Coding: supporting for most platform such as Qwen Code, CLINE, featuring a specially designed function call format;

Basic information

✨ Supporting long context understanding and generation with the context length of 256K tokens;
✨ Supporting 358 coding languages;

['ABAP', 'ActionScript', 'Ada', 'Agda', 'Alloy', 'ApacheConf', 'AppleScript', 'Arc', 'Arduino', 'AsciiDoc', 'AspectJ', 'Assembly', 'Augeas', 'AutoHotkey', 'AutoIt', 'Awk', 'Batchfile', 'Befunge', 'Bison', 'BitBake', 'BlitzBasic', 'BlitzMax', 'Bluespec', 'Boo', 'Brainfuck', 'Brightscript', 'Bro', 'C', 'C#', 'C++', 'C2hs Haskell', 'CLIPS', 'CMake', 'COBOL', 'CSS', 'CSV', "Cap'n Proto", 'CartoCSS', 'Ceylon', 'Chapel', 'ChucK', 'Cirru', 'Clarion', 'Clean', 'Click', 'Clojure', 'CoffeeScript', 'ColdFusion', 'ColdFusion CFC', 'Common Lisp', 'Component Pascal', 'Coq', 'Creole', 'Crystal', 'Csound', 'Cucumber', 'Cuda', 'Cycript', 'Cython', 'D', 'DIGITAL Command Language', 'DM', 'DNS Zone', 'Darcs Patch', 'Dart', 'Diff', 'Dockerfile', 'Dogescript', 'Dylan', 'E', 'ECL', 'Eagle', 'Ecere Projects', 'Eiffel', 'Elixir', 'Elm', 'Emacs Lisp', 'EmberScript', 'Erlang', 'F#', 'FLUX', 'FORTRAN', 'Factor', 'Fancy', 'Fantom', 'Forth', 'FreeMarker', 'G-code', 'GAMS', 'GAP', 'GAS', 'GDScript', 'GLSL', 'Genshi', 'Gentoo Ebuild', 'Gentoo Eclass', 'Gettext Catalog', 'Glyph', 'Gnuplot', 'Go', 'Golo', 'Gosu', 'Grace', 'Gradle', 'Grammatical Framework', 'GraphQL', 'Graphviz (DOT)', 'Groff', 'Groovy', 'Groovy Server Pages', 'HCL', 'HLSL', 'HTML', 'HTML+Django', 'HTML+EEX', 'HTML+ERB', 'HTML+PHP', 'HTTP', 'Haml', 'Handlebars', 'Harbour', 'Haskell', 'Haxe', 'Hy', 'IDL', 'IGOR Pro', 'INI', 'IRC log', 'Idris', 'Inform 7', 'Inno Setup', 'Io', 'Ioke', 'Isabelle', 'J', 'JFlex', 'JSON', 'JSON5', 'JSONLD', 'JSONiq', 'JSX', 'Jade', 'Jasmin', 'Java', 'Java Server Pages', 'JavaScript', 'Julia', 'Jupyter Notebook', 'KRL', 'KiCad', 'Kit', 'Kotlin', 'LFE', 'LLVM', 'LOLCODE', 'LSL', 'LabVIEW', 'Lasso', 'Latte', 'Lean', 'Less', 'Lex', 'LilyPond', 'Linker Script', 'Liquid', 'Literate Agda', 'Literate CoffeeScript', 'Literate Haskell', 'LiveScript', 'Logos', 'Logtalk', 'LookML', 'Lua', 'M', 'M4', 'MAXScript', 'MTML', 'MUF', 'Makefile', 'Mako', 'Maple', 'Markdown', 'Mask', 'Mathematica', 'Matlab', 'Max', 'MediaWiki', 'Metal', 'MiniD', 'Mirah', 'Modelica', 'Module Management System', 'Monkey', 'MoonScript', 'Myghty', 'NSIS', 'NetLinx', 'NetLogo', 'Nginx', 'Nimrod', 'Ninja', 'Nit', 'Nix', 'Nu', 'NumPy', 'OCaml', 'ObjDump', 'Objective-C++', 'Objective-J', 'Octave', 'Omgrofl', 'Opa', 'Opal', 'OpenCL', 'OpenEdge ABL', 'OpenSCAD', 'Org', 'Ox', 'Oxygene', 'Oz', 'PAWN', 'PHP', 'POV-Ray SDL', 'Pan', 'Papyrus', 'Parrot', 'Parrot Assembly', 'Parrot Internal Representation', 'Pascal', 'Perl', 'Perl6', 'Pickle', 'PigLatin', 'Pike', 'Pod', 'PogoScript', 'Pony', 'PostScript', 'PowerShell', 'Processing', 'Prolog', 'Propeller Spin', 'Protocol Buffer', 'Public Key', 'Pure Data', 'PureBasic', 'PureScript', 'Python', 'Python traceback', 'QML', 'QMake', 'R', 'RAML', 'RDoc', 'REALbasic', 'RHTML', 'RMarkdown', 'Racket', 'Ragel in Ruby Host', 'Raw token data', 'Rebol', 'Red', 'Redcode', "Ren'Py", 'RenderScript', 'RobotFramework', 'Rouge', 'Ruby', 'Rust', 'SAS', 'SCSS', 'SMT', 'SPARQL', 'SQF', 'SQL', 'STON', 'SVG', 'Sage', 'SaltStack', 'Sass', 'Scala', 'Scaml', 'Scheme', 'Scilab', 'Self', 'Shell', 'ShellSession', 'Shen', 'Slash', 'Slim', 'Smali', 'Smalltalk', 'Smarty', 'Solidity', 'SourcePawn', 'Squirrel', 'Stan', 'Standard ML', 'Stata', 'Stylus', 'SuperCollider', 'Swift', 'SystemVerilog', 'TOML', 'TXL', 'Tcl', 'Tcsh', 'TeX', 'Tea', 'Text', 'Textile', 'Thrift', 'Turing', 'Turtle', 'Twig', 'TypeScript', 'Unified Parallel C', 'Unity3D Asset', 'Uno', 'UnrealScript', 'UrWeb', 'VCL', 'VHDL', 'Vala', 'Verilog', 'VimL', 'Visual Basic', 'Volt', 'Vue', 'Web Ontology Language', 'WebAssembly', 'WebIDL', 'X10', 'XC', 'XML', 'XPages', 'XProc', 'XQuery', 'XS', 'XSLT', 'Xojo', 'Xtend', 'YAML', 'YANG', 'Yacc', 'Zephir', 'Zig', 'Zimpl', 'desktop', 'eC', 'edn', 'fish', 'mupad', 'nesC', 'ooc', 'reStructuredText', 'wisp', 'xBase']

✨ Retain strengths in math and general capabilities from base model.

[!Important]

Qwen3-coder function calling relies on our new tool parser qwen3coder_tool_parser.py here.

We updated both the special tokens and their corresponding token ids, in order to maintain consistency with Qwen3. Please make sure to use the new tokenizer.

model name	type	length	Download
Qwen3-Coder-480B-A35B-Instruct	instruct	256k	🤗 Hugging Face • 🤖 ModelScope
Qwen3-Coder-480B-A35B-Instruct-FP8	instruct	256k	🤗 Hugging Face • 🤖 ModelScope

Detailed performance and introduction are shown in this 📑 blog.

Quick Start

[!Important] Qwen3-Coder-480B-A35B-Instruct are instruction models for chatting;

This model supports only non-thinking mode and does not generate <think></think> blocks in its output. Meanwhile, specifying enable_thinking=False is no longer required.**

👉🏻 Chat with Qwen3-Coder-480B-A35B-Instruct

You can just write several lines of code with transformers to chat with Qwen3-Coder-480B-A35B-Instruct. Essentially, we build the tokenizer and the model with from_pretrained method, and we use generate method to perform chatting with the help of chat template provided by the tokenizer. Below is an example of how to chat with Qwen3-Coder-480B-A35B-Instruct:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen3-Coder-480B-A35B-Instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "write a quick sort algorithm."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=65536
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

The apply_chat_template() function is used to convert the messages into a format that the model can understand. The add_generation_prompt argument is used to add a generation prompt, which refers to <|im_start|>assistant\n to the input. Notably, we apply ChatML template for chat models following our previous practice. The max_new_tokens argument is used to set the maximum length of the response. The tokenizer.batch_decode() function is used to decode the response. In terms of the input, the above messages is an example to show how to format your dialog history and system prompt. You can use the other size of instruct model in the same way.

Fill in the middle with Qwen3-Coder-480B-A35B-Instruct

The code insertion task, also referred to as the “fill-in-the-middle” challenge, requires the insertion of code segments in a manner that bridges the gaps within a given code context. For an approach aligned with best practices, we recommend adhering to the formatting guidelines outlined in the paper “Efficient Training of Language Models to Fill in the Middle”[arxiv].

The prompt should be structured as follows:

`1`	`prompt = '<\|fim_prefix\|>' + prefix_code + '<\|fim_suffix\|>' + suffix_code + '<\|fim_middle\|>'`

Following the approach mentioned, an example would be structured in this manner:

from transformers import AutoTokenizer, AutoModelForCausalLM
# load model
device = "cuda" # the device to load the model onto

TOKENIZER = AutoTokenizer.from_pretrained("Qwen/Qwen3-Coder-480B-A35B-Instruct")
MODEL = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-Coder-480B-A35B-Instruct", device_map="auto").eval()


input_text = """<|fim_prefix|>def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[len(arr) // 2]
    <|fim_suffix|>
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]
    return quicksort(left) + middle + quicksort(right)<|fim_middle|>"""
            
messages = [
    {"role": "system", "content": "You are a code completion assistant."},
    {"role": "user", "content": input_text}
]


text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = TOKENIZER([text], return_tensors="pt").to(model.device)

# Use `max_new_tokens` to control the maximum output length.
generated_ids = MODEL.generate(model_inputs.input_ids, max_new_tokens=512, do_sample=False)[0]
# The generated_ids include prompt_ids, we only need to decode the tokens after prompt_ids.
output_text = TOKENIZER.decode(generated_ids[len(model_inputs.input_ids[0]):], skip_special_tokens=True)

print(f"Prompt: {input_text}\n\nGenerated text: {output_text}")

Use Cases

Example: Physics-Based Chimney Demolition Simulation with Controlled Explosion

Prompt with Qwen Chat Web Dev

使用 three.js, cannon-es.js 生成一个震撼的3D建筑拆除演示。

## 场景设置：
- 地面是一个深灰色混凝土平面，尺寸80*80，
- 所有物体严格遵循现实物理规则，包括重力、摩擦力、碰撞检测和动量守恒

## 建筑结构：
- 一座圆形高层建筑，周长对应20个方块
- 建筑总高度60个方块
- 每层采用砖砌结构，方块与砖结构建筑一致, 错开50%排列，增强结构稳定性
- 建筑外墙使用米色方块
- **重要：方块初始排列时必须确保紧密贴合，无间隙，可以通过轻微重叠或调整半径来实现**
- **重要：建筑初始化完成后，所有方块应该处于物理"睡眠"状态，确保建筑在爆炸前保持完美的静止状态，不会因重力而下沉或松散**
- 建筑砖块之间使用粘性材料填充（不可见），通过高摩擦力（0.8+）和低弹性（0.05以下）来模拟粘合效果
- 砖块在建筑倒塌瞬间不会散掉，而是建筑作为一个整体倒在地面的时候才因受力过大而散掉

## 定向爆破系统：
- 在建筑的第1层的最右侧方块附近安装爆炸装置（不可见）
- 提供操作按钮点击爆炸
- **爆炸时唤醒所有相关方块的物理状态**
- 爆炸点产生半径2的强力冲击波，冲击波影响到的方块, 受到2-5单位的冲击力

## 建筑稳定性要求：
- **确保建筑在未爆炸时完全静止，无任何晃动或下沉**
- **物理世界初始化后给建筑几个物理步骤来自然稳定，或使用睡眠机制**
- **方块间的接触材料应具有高摩擦力和极低弹性，模拟砖块间的砂浆粘合**

## 震撼的倒塌效果：
- 方块在爆炸冲击下不仅飞散，还会在空中翻滚和碰撞
- 烟尘会随着建筑倒塌逐渐扩散，营造真实的拆除现场氛围

## 增强的视觉效果：
- 添加环境光照变化：爆炸瞬间亮度激增，然后被烟尘遮挡变暗
- 粒子系统包括：烟雾、灰尘

## 技术要求：
- 粒子系统用于烟雾和灰尘效果
- 所有代码集成在单个HTML文件中，包含必要的CSS样式
- 添加简单的UI控制：重置按钮、相机角度切换, 爆炸按钮, 鼠标左键控制摄像机角度，右键控制摄像机位置，滚轮控制摄像机焦距

Example: Multicolor and Interactive Animation

Prompt with Cline [act mode]

Create an amazing animation multicolor and interactive using p5js

use this cdn:
https://cdn.jsdelivr.net/npm/p5@1.7.0/lib/p5.min.js

Example: 3D Google Earth

Prompt with Qwen Chat Web Dev

`1`	`To create a 3D Google Earth, you need to load the terrain map correctly. You can use any online resource. The code is written into an HTML file.`

Example: Testing Your WPM with a Famous Quote

Prompt with Qwen-Code CLI

1
2

Create an interesting typing game with a keyboard in the lower middle of the screen and some famous articles in the upper middle. When the user types a word correctly, a cool reaction should be given to encourage him. Design a modern soft color scheme inspired by macarons. Come up with a very creative solution first, and then start writing code.
The game should be able to support typing, and you need to neglect upcase and lowercase.

Example: Bouncing Ball in Rotation Hypercube

Prompt with Qwen Chat Web Dev

`1`	`Make a page in HTML that shows an animation of a ball bouncing in a rotating hypercube`

Example: Solar System Simulation

Prompt with Cline [act mode]

`1`	`write a web page to show the solar system simulation`

Example: DUET Game

Prompt with Cline [act mode]

Create a complete, single-file HTML game with CSS and JavaScript. The game is inspired by "Duet".

Gameplay:

There are two balls, one red and one blue, rotating around a central point.
The player uses the 'A' and 'D' keys to rotate them counter-clockwise and clockwise.
White rectangular obstacles move down from the top of the screen.
The player must rotate the balls to avoid hitting the obstacles.
If a ball hits an obstacle, the game is over.
Visuals:

Make the visual effects amazing.
Use a dark background with neon glowing effects for the balls and obstacles.
Animations should be very smooth.

Star History

Citation

If you find our work helpful, feel free to give us a cite.

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report}, 
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388},
}
@article{hui2024qwen2,
  title={Qwen2. 5-Coder Technical Report},
  author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
  journal={arXiv preprint arXiv:2409.12186},
  year={2024}
}

Contact Us

If you are interested to leave a message to either our research team or product team, join our Discord or WeChat groups!

↑ Back to Top ↑

MiniCPM

Thu, 12 Jun 2025 15:30:22 +0800

OpenBMB/MiniCPM

中文 | English

MiniCPM 技术博客 | MiniCPM 知识库 | MiniCPM 论文 | MiniCPM-V 仓库 | 加入我们的 discord 和微信群

https://github.com/user-attachments/assets/ab36fd7a-485b-4707-b72f-b80b5c43d024

更新日志🔥

[2025.06.06] 发布 MiniCPM4！该模型在保持同等规模最优性能的同时，实现了极致的效率提升！在典型端侧芯片上能够实现 5 倍以上生成加速！
[2024.09.28] LLMxMapReduce 开源，支持 MiniCPM3-4B，理论上支持无限长文本输入！
[2024.09.18] SGLang 已经支持 MiniCPM3-4B (推荐使用)！由于 SGLang v0.3 对 MiniCPM3 中使用的 MLA 结构进行了推理优化，吞吐量相比于 vLLM 提高 70%！[用法]
[2024.09.16] llama.cpp 已经官方支持 MiniCPM3-4B！[GGUF模型|用法]
[2024.09.05] 发布 MiniCPM3-4B！该模型的表现超越 Phi-3.5-mini-instruct 和 GPT-3.5-Turbo-0125，并且能够比肩 Llama3.1-8B-Instruct、Qwen2-7B-Instruct、GLM-4-9B-Chat 等多个 7B-9B 参数量的模型。
[2024.07.09] MiniCPM-2B 已经支持使用 SGLang 推理！
[2024.07.05] 发布 MiniCPM-S-1B！该模型在保持下游任务性能无损的前提下，FFN 层实现了 87.89% 的平均稀疏度，将 FFN FLOPs 降低了 84%。
[2024.04.11] 发布 MiniCPM-2B-128k、MiniCPM-MoE-8x2B 和 MiniCPM-1B！点击这里查看技术博客。
[2024.03.16] MiniCPM-2B 的 30 余个中间检查点开放了！HuggingFace链接
[2024.02.01] 发布 MiniCPM-2B！该模型在公开评测集上与 Mistral-7B 表现相近（中文、数学、代码能力更优），整体性能超越 Llama2-13B、MPT-30B、Falcon-40B 等模型。

模型下载

HuggingFace	ModelScope
MiniCPM4-8B	MiniCPM4-8B
MiniCPM4-0.5B	MiniCPM4-0.5B
BitCPM4-1B	BitCPM4-1B
BitCPM4-0.5B	BitCPM4-0.5B
MiniCPM4-8B-Eagle-FRSpec	MiniCPM4-8B-Eagle-FRSpec
MiniCPM4-8B-Eagle-FRSpec-QAT	MiniCPM4-8B-Eagle-FRSpec-QAT
MiniCPM4-8B-Eagle-vLLM	MiniCPM4-8B-Eagle-vLLM
MiniCPM4-8B-marlin-Eagle-vLLM	MiniCPM4-8B-marlin-Eagle-vLLM
MiniCPM4-Survey	MiniCPM4-Survey
MiniCPM4-MCP	MiniCPM4-MCP
MiniCPM4-0.5B-QAT-Int4-unquantized	MiniCPM4-0.5B-QAT-Int4-unquantized
MiniCPM4-0.5B-QAT-Int4-GPTQ-format	MiniCPM4-0.5B-QAT-Int4-GPTQ-format
MiniCPM3-4B	MiniCPM3-4B
MiniCPM-2B-sft	MiniCPM-2B-sft
MiniCPM-2B-dpo	MiniCPM-2B-dpo
MiniCPM-2B-128k	MiniCPM-2B-128k
MiniCPM-MoE-8x2B	MiniCPM-MoE-8x2B
MiniCPM-1B	MiniCPM-1B
MiniCPM-S-1B	MiniCPM-S-1B

注: 更多模型版本见这里。

MiniCPM 4.0

MiniCPM 4 是一个极致高效的端侧大模型，从模型架构、学习算法、训练数据与推理系统四个层面进行了高效优化，实现了极致的效率提升。

🏗️ 高效模型架构：
- InfLLM v2 – 可训练的稀疏注意力机制：采用可训练的稀疏注意力机制架构，在 128K 长文本处理中，每个词元仅需与不足 5% 的词元进行相关性计算，显著降低长文本的计算开销
🧠 高效学习算法：
- 模型风洞 2.0 – 高效 Predictable Scaling：引入下游任务的 Scaling 预测方法，实现更精准的模型训练配置搜索
- BitCPM – 极致的三值量化：将模型参数位宽压缩至 3 值，实现模型位宽 90% 的极致瘦身
- 高效训练工程优化：采用 FP8 低精度计算技术，结合多词元预测（Multi-token Prediction）训练策略
📚 高知识密度训练数据：
- UltraClean – 高质量预训练数据的清洗与合成：构建基于高效验证的迭代式数据清洗策略，开源高质量中英文预训练数据集 UltraFineweb
- UltraChat v2 – 高质量有监督微调数据合成：构建大规模高质量有监督微调数据集，涵盖知识密集型数据、推理密集型数据、指令遵循数据、长文本理解数据、工具调用数据等多个维度
⚡ 高效推理系统：
- CPM.cu – 轻量级的高效CUDA推理框架：融合了稀疏注意力机制、模型量化与投机采样，充分体现MiniCPM4的效率优势
- ArkInfer – 跨平台部署系统：支持多后端环境的一键部署，提供灵活的跨平台适配能力

评测结果

效率评测

在 Jetson AGX Orin 和 RTX 4090 两款典型端侧芯片上，MiniCPM4 在长文本处理任务中展现出大幅领先同尺寸模型的处理速度。随着文本长度的增加，MiniCPM4 的性能优势愈发显著。在 Jetson AGX Orin 平台上，相较于 Qwen3-8B，MiniCPM4 实现了约 7 倍的生成速度提升。

综合评测

MiniCPM4 推出端侧 8B、0.5B 两种参数规模版本，均在同级别模型中实现了最佳性能表现。

长文本评测

MiniCPM4 基于 32K 长文本进行预训练，并通过 YaRN 技术实现长度扩展。在 128K 长文本的大海捞针任务中，MiniCPM4 展现出卓越的性能表现。

BitCPM4: 模型量化

BitCPM4 是基于 MiniCPM 系列模型进行量化感知训练（QAT）后得到的三值量化模型，在训练效率和模型参数效率实现了有效的提升。

训练方法改进
- 在小规模模型上进行风洞实验，搜索训练所需的训练超参。
- 通过使用一阶段高精训练+二阶段 QAT 的方法，充分利用已经完成或部分完成训练的高精度模型，极大地压缩了 QAT 阶段所需要的算力。
高效参数效率
- 模型使用 1.58Bit 的位宽达到的性能对标与同参数量级别的全精度模型，模型参数效率高。

BitCPM4 评测

BitCPM4 在测试中的表现可以对标同级别的业界主流全精度模型。

BitCPM4 模型推理

BitCPM4 开源的模型参数为伪量化形式，可以直接使用 Huggingface 框架进行推理。

模型应用

MiniCPM4-Survey: 综述生成

MiniCPM4-Survey 是由 THUNLP、中国人民大学和 ModelBest 联合开发的开源大语言模型智能体。它基于 MiniCPM4-8B 基座模型，接受用户质量作为输入，自主生成可信的长篇综述论文。主要特性包括：

计划-检索-写作生成框架 — 我们提出了一个多智能体生成框架，包含三个核心阶段：计划（定义综述的整体结构）、检索（生成合适的检索关键词）和写作（利用检索到的信息，生成连贯的段落）。
高质量数据集构建——我们收集并处理大量人类专家写作的综述论文，构建高质量训练集。同时，我们收集大量研究论文，构建检索数据库。
多方面奖励设计 — 我们精心设计了包含结构、内容和引用的奖励，用于评估综述的质量，在强化学习训练阶段作奖励函数。
多步强化学习训练策略 — 我们提出了一个上下文管理器，以确保在促进有效推理的同时保留必要的信息，并构建了并行环境，维持强化学习训练高效。

使用与演示案例

详见此处

评估

Method	Relevance	Coverage	Depth	Novelty	Avg.	Fact Score
Naive RAG (driven by G2FT)	3.25	2.95	3.35	2.60	3.04	43.68
AutoSurvey (driven by G2FT)	3.10	3.25	3.15	3.15	3.16	46.56
Webthinker (driven by WTR1-7B)	3.30	3.00	2.75	2.50	2.89	–
Webthinker (driven by QwQ-32B)	3.40	3.30	3.30	2.50	3.13	–
OpenAI Deep Research (driven by GPT-4o)	3.50	3.95	3.55	3.00	3.50	–
MiniCPM4-Survey	3.45	3.70	3.85	3.00	3.50	68.73
w/o RL	3.55	3.35	3.30	2.25	3.11	50.24

GPT-4o 对综述生成系统的性能比较。“G2FT” 代表 Gemini-2.0-Flash-Thinking，“WTR1-7B” 代表 Webthinker-R1-7B。由于 Webthinker 不包括引用功能，OpenAI Deep Research 在导出结果时不提供引用，因此省略了对它们的 FactScore 评估。我们的技术报告中包含评测的详细信息。

MiniCPM4-MCP: MCP增强的工具调用

MiniCPM4-MCP 是由清华大学自然语言处理实验室（THUNLP）、中国人民大学与 ModelBest 联合开发的开源本地大语言模型代理，它基于 MiniCPM-4-8B，拥有 80 亿参数。它能够通过 MCP 协议与各种工具和数据资源交互，解决多种真实世界任务。截至目前，MiniCPM4-MCP 已支持：

涵盖 16 个 MCP 服务器（servers）中工具的使用：这些服务器所包含的工具横跨了办公类、生活类、通讯类、资讯类、工作管理类等.
单工具使用的能力：可使用符合 MCP 协议的工具进行单一工具的一步或多步调用。
跨工具组合使用的能力：可组合使用符合 MCP 协议的不同工具。

使用与演示案例

详见此处

评估

MCP 服务器		gpt-4o			qwen3			minicpm4
	函数名正确率	参数名正确率	数值正确率	函数名正确率	参数名正确率	数值正确率	函数名正确率	参数名正确率	数值正确率
Airbnb	89.3	67.9	53.6	92.8	60.7	50.0	96.4	67.9	50.0
Amap-Maps	79.8	77.5	50.0	74.4	72.0	41.0	89.3	85.7	39.9
Arxiv-MCP-Server	85.7	85.7	85.7	81.8	54.5	50.0	57.1	57.1	52.4
Calculator	100.0	100.0	20.0	80.0	80.0	13.3	100.0	100.0	6.67
Computor-Control-MCP	90.0	90.0	90.0	90.0	90.0	90.0	90.0	90.0	86.7
Desktop-Commander	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
Filesystem	63.5	63.5	31.3	69.7	69.7	26.0	83.3	83.3	42.7
Github	92.0	80.0	58.0	80.5	50.0	27.7	62.8	25.7	17.1
Gaode	71.1	55.6	17.8	68.8	46.6	24.4	68.9	46.7	15.6
MCP-Code-Executor	85.0	80.0	70.0	80.0	80.0	70.0	90.0	90.0	65.0
MCP-Docx	95.8	86.7	67.1	94.9	81.6	60.1	95.1	86.6	76.1
PPT	72.6	49.8	40.9	85.9	50.7	37.5	91.2	72.1	56.7
PPTx	64.2	53.7	13.4	91.0	68.6	20.9	91.0	58.2	26.9
Simple-Time-Server	90.0	70.0	70.0	90.0	90.0	90.0	90.0	60.0	60.0
Slack	100.0	90.0	70.0	100.0	100.0	65.0	100.0	100.0	100.0
Whisper	90.0	90.0	90.0	90.0	90.0	90.0	90.0	90.0	30.0
平均值	80.2	70.2	49.1	83.5	67.7	43.8	88.3	76.1	51.2

模型推理

CPM.cu

我们推荐使用 CPM.cu 对 MiniCPM4 模型进行推理。CPM.cu 是面壁开发的一个集合了高效稀疏、投机采样、量化等技术的 CUDA 推理框架，能够完全发挥 MiniCPM4 的效率优势。

你可以通过以下脚本安装 CPM.cu 并进行推理：

1
2
3

git clone https://github.com/OpenBMB/CPM.cu.git --recursive
cd CPM.cu
python3 setup.py install

你可以通过以下命令进行推理并查看模型的运行速度。

1
2

python3 tests/long_prompt_gen.py # 生成 prompt.txt
python3 tests/test_generate.py --prompt-file prompt.txt

更多关于 CPM.cu 的细节，请参考 CPM.cu 仓库。

HuggingFace

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)

path = 'openbmb/MiniCPM4-8B'
device = "cuda"
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)

# User can directly use the chat interface
# responds, history = model.chat(tokenizer, "Write an article about Artificial Intelligence.", temperature=0.7, top_p=0.7)
# print(responds)

# User can also use the generate interface
messages = [
    {"role": "user", "content": "Write an article about Artificial Intelligence."},
]
prompt_text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([prompt_text], return_tensors="pt").to(device)

model_outputs = model.generate(
    **model_inputs,
    max_new_tokens=1024,
    top_p=0.7,
    temperature=0.7
)
output_token_ids = [
    model_outputs[i][len(model_inputs[i]):] for i in range(len(model_inputs['input_ids']))
]

responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0]
print(responses)

本模型支持稀疏注意力机制 InfLLM v2，可高效处理长序列推理。如需启用该功能，请先安装依赖库 infllmv2_cuda_impl

运行以下命令即可安装：

git clone -b feature_infer https://github.com/OpenBMB/infllmv2_cuda_impl.git
cd infllmv2_cuda_impl
git submodule update --init --recursive
pip install -e . # or python setup.py install

启用 InfLLM v2 需在 config.json 配置文件中添加 sparse_config 字段：

{
    ...,
    "sparse_config": {
        "kernel_size": 32,
        "kernel_stride": 16,
        "init_blocks": 1,
        "block_size": 64,
        "window_size": 2048,
        "topk": 64,
        "use_nope": false,
        "dense_len": 8192
    }
}

这些参数控制 InfLLM v2 的行为:

kernel_size（默认值：32）：语义核的大小。
kernel_stride（默认值：16）：相邻语义核的步长。
init_blocks（默认值：1）：每个 query token 关注的初始的块数量，用于确保关注序列开头部分。
block_size（默认值：64）：key-value blocks 的块大小。
window_size（默认值：2048）：局部滑动窗口大小。
topk（默认值：64）：每个 token 仅与最相关的 top-k 个 key-value blocks 计算注意力。
use_nope（默认值：false）：是否在块选择中使用NOPE技术以提升性能。
dense_len（默认值：8192）：稀疏注意力对短序列收益有限，当 token 长度低于此阈值时自动切换为标准注意力。设为 -1 则强制始终使用稀疏注意力。

Minicpm4 原生支持 32,768 tokens 的上下文长度。若对话总长度（输入 + 输出）远超此限制，建议通过 RoPE 缩放技术扩展上下文。我们已验证通过调整 LongRoPE 因子，模型可稳定支持 131,072 tokens 的超长上下文。

修改方法：在 config.json 文件中调整 rope_scaling 字段参数即可。

{
    ...,
    "rope_scaling": {
        "rope_type": "longrope", 
        "long_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.31638758099915, 122.6714244963338, 122.9673822552567, 123.21386397019609, 123.41898278254268, 123.58957065488238, 123.73136519024158, 123.84917421274221, 123.94701903496814, 124.02825801299717, 124.09569231686116],
        "short_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.31638758099915, 122.6714244963338, 122.9673822552567, 123.21386397019609, 123.41898278254268, 123.58957065488238, 123.73136519024158, 123.84917421274221, 123.94701903496814, 124.02825801299717, 124.09569231686116],
        "original_max_position_embeddings": 32768
    }
}

vLLM

安装

参照 vLLM 官方仓库，通过源码安装最新版本。

1
2
3

pip install -U vllm \
    --pre \
    --extra-index-url https://wheels.vllm.ai/nightly

使用 vLLM 推理 MiniCPM4-8B 模型：

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

model_name = "openbmb/MiniCPM4-8B"
prompt = [{"role": "user", "content": "推荐5个北京的景点。"}]

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
input_text = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)

llm = LLM(
    model=model_name,
    trust_remote_code=True,
    max_num_batched_tokens=32768, 
    dtype="bfloat16", 
    gpu_memory_utilization=0.8, 
)
sampling_params = SamplingParams(top_p=0.7, temperature=0.7, max_tokens=1024, repetition_penalty=1.02)

outputs = llm.generate(prompts=input_text, sampling_params=sampling_params)

print(outputs[0].outputs[0].text)

在 vLLM 中使用 Eagle 投机解码：只需如下初始化推理引擎

llm = LLM(
    model=model_name,
    trust_remote_code=True,
    max_num_batched_tokens=32768, 
    dtype="bfloat16", 
    gpu_memory_utilization=0.8, 
    speculative_config={
        "method": "eagle",
        "model": "openbmb/MiniCPM4-8B-Eagle-vLLM",
        "num_speculative_tokens": 2,
        "max_model_len": 32768,
    },
)

在 vLLM 中推理量化后的 MiniCPM4-8B：只需如下初始化推理引擎

llm = LLM(
    model="openbmb/MiniCPM4-8B-marlin-vLLM",
    trust_remote_code=True,
    max_num_batched_tokens=32768, 
    dtype="bfloat16", 
    gpu_memory_utilization=0.8, 
)

在 vLLM 中使用 Eagle 投机解码推理量化后的 MiniCPM4-8B：只需如下初始化推理引擎

llm = LLM(
    model="openbmb/MiniCPM4-8B-marlin-vLLM",
    trust_remote_code=True,
    max_num_batched_tokens=32768,
    dtype="bfloat16",
    gpu_memory_utilization=0.8,
    speculative_config={
        "method": "eagle",
        "model": "openbmb/MiniCPM4-8B-marlin-Eagle-vLLM",
        "num_speculative_tokens": 2,
        "max_model_len": 32768,
    },
)

注意：如果你使用 vLLM 中的 OpenAI 兼容的服务端，chat API 默认会将 add_special_tokens 设置为 False。这会导致缺失一些特殊标记（例如，BOS），而这些标记对 MiniCPM4 模型至关重要。为确保模型行为正常，你需要在 API 调用中显式设置 extra_body={"add_special_tokens": True}，如下所示：

import openai

client = openai.Client(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="openbmb/MiniCPM4-8B",
    messages=[
        {"role": "user", "content": "Write an article about Artificial Intelligence."},
    ],
    temperature=0.7,
    max_tokens=1024,
    extra_body={"add_special_tokens": True},  # 确保添加了诸如 BOS 等特殊标记
)

print(response.choices[0].message.content)

SGLang

安装

参考 SGLang 官方仓库，通过源码安装。

git clone -b openbmb https://github.com/OpenBMB/sglang.git
cd sglang

pip install --upgrade pip
pip install -e "python[all]"

启动推理服务

`1`	`python -m sglang.launch_server --model openbmb/MiniCPM4-8B --trust-remote-code --port 30000 --chat-template chatml`

然后用户可以通过运行以下命令来使用聊天界面：

import openai

client = openai.Client(base_url=f"http://localhost:30000/v1", api_key="None")

response = client.chat.completions.create(
    model="openbmb/MiniCPM4-8B",
    messages=[
        {"role": "user", "content": "Write an article about Artificial Intelligence."},
    ],
    temperature=0.7,
    max_tokens=1024,
)
print(response.choices[0].message.content)

使用投机加速

python3 -m sglang.launch_server --model-path [model] \ 
    --speculative_draft_model_path [draft_model] \
    --host 0.0.0.0 --trust-remote-code \
    --speculative-algorithm EAGLE --speculative-num-steps 1 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 \
    --mem-fraction 0.5

模型微调

LLaMA-Factory

目前模型微调支持 LLaMA-Factory，使用方法参考 LLaMA-Factory 微调

MiniCPM 3.0

查看 MiniCPM 3.0 的详细信息

MiniCPM 3.0 是一个 4B 参数量的语言模型，相比 MiniCPM1.0/2.0，功能更加全面，综合能力大幅提升，多数评测集上的效果比肩甚至超越众多 7B-9B 模型。

支持工具调用🛠️（Function Calling）和代码解释器💻（Code Interpreter）：Berkeley Function Calling Leaderboard (BFCL) 上取得 9B 规模以下 SOTA，超越 GLM-4-9B-Chat、Qwen2-7B-Instruct。
超强的推理能力🧮：数学能力方面，MathBench 上的效果超越 GPT-3.5-Turbo 以及多个 7B-9B 模型。在非常具有挑战性的 LiveCodeBench 上，效果超越 Llama3.1-8B-Instruct。
出色的中英文指令遵循能力🤖：英文指令遵循 IFEval、中文指令遵循 FollowBench-zh 效果超越 GLM-4-9B-Chat、Qwen2-7B-Instruct。
长文本能力：原生支持 32k 上下文长度，32k 长度内大海捞针全绿。提出 LLMxMapReduce ，理论可处理的上下文长度达到 +∞，在综合性长文本评测基准 InfiniteBench 平均得分超越GPT-4、KimiChat等标杆模型。
RAG能力：我们发布了 MiniCPM RAG 套件。基于 MiniCPM 系列模型的 MiniCPM-Embedding、MiniCPM-Reranker 在中文、中英跨语言检索测试中取得 SOTA 表现；针对 RAG 场景的 MiniCPM3-RAG-LoRA 在开放域问答等多项任务上超越 Llama3-8B、Baichuan2-13B 等模型。

评测结果

综合评测

评测集	Qwen2-7B-Instruct	GLM-4-9B-Chat	Gemma2-9B-it	Llama3.1-8B-Instruct	GPT-3.5-Turbo-0125	Phi-3.5-mini-Instruct(3.8B)	MiniCPM3-4B
英文能力
MMLU	70.5	72.4	72.6	69.4	69.2	68.4	67.2
BBH	64.9	76.3	65.2	67.8	70.3	68.6	70.2
MT-Bench	8.41	8.35	7.88	8.28	8.17	8.60	8.41
IFEVAL (Prompt Strict-Acc.)	51.0	64.5	71.9	71.5	58.8	49.4	68.4
中文能力
CMMLU	80.9	71.5	59.5	55.8	54.5	46.9	73.3
CEVAL	77.2	75.6	56.7	55.2	52.8	46.1	73.6
AlignBench v1.1	7.10	6.61	7.10	5.68	5.82	5.73	6.74
FollowBench-zh (SSR)	63.0	56.4	57.0	50.6	64.6	58.1	66.8
数学能力
MATH	49.6	50.6	46.0	51.9	41.8	46.4	46.6
GSM8K	82.3	79.6	79.7	84.5	76.4	82.7	81.1
MathBench	63.4	59.4	45.8	54.3	48.9	54.9	65.6
代码能力
HumanEval+	70.1	67.1	61.6	62.8	66.5	68.9	68.3
MBPP+	57.1	62.2	64.3	55.3	71.4	55.8	63.2
LiveCodeBench v3	22.2	20.2	19.2	20.4	24.0	19.6	22.6
工具调用能力
BFCL v2	71.6	70.1	19.2	73.3	75.4	48.4	76.0
综合能力
平均分	65.3	65.0	57.9	60.8	61.0	57.2	66.3

工具调用能力

我们在 Berkeley Function Calling Leaderboard (BFCL) 上测试了模型的工具调用能力，MiniCPM3-4B 在该榜单上的表现超越了多个 7B-9B 参数量的模型，优于 GPT-3.5-Turbo-0125。

模型	总体准确率	AST Summary	Exec Summary	Irrelevance Detection	Relevance Detection
MiniCPM3-4B	76.03%	68.55%	85.54%	53.71%	90.24%
Llama3.1-8B-Instruct	73.28%	64.61%	86.48%	43.12%	85.37%
Qwen2-7B-Instruct	71.61%	65.71%	79.57%	44.70%	90.24%
GLM-4-9B-Chat	70.08%	60.69%	80.02%	55.02%	82.93%
Phi-3.5-mini-instruct	48.44%	38.89%	54.04%	46.78%	65.85%
Gemma2-9B-it	19.18%	5.41%	18.50%	88.88%	7.32%

长文本能力

在 32k 的上下文长度进行大海捞针测试，结果如下图：

同时我们提出LLMxMapReduce，利用分治的策略，理论上可以处理无限长度的文本。我们在InfiniteBench上测试了模型的长文本处理能力，在LLMxMapReduce框架的加持下，MiniCPM3-4B在这个榜单的平均得分能够超越 GPT-4、KimiChat 等标杆模型。

	Context length	Qwen2-70b	Kimi-Chat(2024.06)	GPT-4 (From InfiniteBench)	MiniCPM 3.0 x MR	Qwen2-70b x MR	Llama3-70bx MR
Math.Find	87.9k	59.71%	18.57%	60.00%	83.43%	54.29%	91.43%
Retrieve.KV	89.9k	29.00%	69.20%	89.00%	93.80%	98.80%	98.89%
En.Dia	103.6K	23.00%	23.00%	7.50%	12.50%	46.50%	17.50%
Code.Debug	114.7k	45.43%	38.32%	54.31%	25.63%	54.82%	62.94%
Retrieve.Number	122.4k	100.00%	97.45%	100.00%	99.32%	100.00%	99.79%
Retrieve.PassKey	122.4k	100.00%	99.32%	100.00%	98.81%	100.00%	100.00%
En.Sum	171.5K	31.85%	29.94%	14.73%	25.89%	32.39%	30.63%
En.MC	184.4k	81.66%	79.91%	68.12%	66.38%	83.84%	82.10%
En.QA	192.6k	21.97%	18.80%	22.44%	28.39%	23.13%	34.70%
Zh.QA	2068.6k	21.40%	19.84%	25.96%	23.66%	19.10%	N/A
avg w/o Zh.QA	/	51.92%	52.96%	55.33%	59.29%	64.98%	68.64%
avg	/	48.86%	49.65%	52.39%	55.55%	60.39%	N/A

模型推理

Huggingface

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)

path = 'openbmb/MiniCPM3-4B'
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map='cuda', trust_remote_code=True)

responds, history = model.chat(tokenizer, "请写一篇关于人工智能的文章，详细介绍人工智能的未来发展和隐患。", temperature=0.7, top_p=0.7)
print(responds)

SGLang（推荐）

安装

参考 SGLang 官方仓库，通过源码安装最新版本。

启动推理服务

`1`	`python -m sglang.launch_server --model openbmb/MiniCPM3-4B --trust-remote-code --port 30000 --chat-template chatml`

使用示例

from sglang import function, system, user, assistant, gen, set_default_backend, RuntimeEndpoint

@function
def multi_turn_question(s, question_1, question_2):
    s += user(question_1)
    s += assistant(gen("answer_1", max_tokens=1024))
    s += user(question_2)
    s += assistant(gen("answer_2", max_tokens=1024))

set_default_backend(RuntimeEndpoint("http://localhost:30000"))

state = multi_turn_question.run(
    question_1="介绍一下人工智能",
    question_2="写一篇关于它的文章",
)

for m in state.messages():
    print(m["role"], ":", m["content"])

vLLM

安装 vllm
1

pip install "vllm>=0.6.2"

推理

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

model_name = "openbmb/MiniCPM3-4B"
prompt = [{"role": "user", "content": "请写一篇关于人工智能的文章，详细介绍人工智能的未来发展和隐患。"}]

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
input_text = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)

llm = LLM(model=model_name,
    trust_remote_code=True,
    tensor_parallel_size=1
)
sampling_params = SamplingParams(top_p=0.7, temperature=0.7, max_tokens=1024)

outputs = llm.generate(prompts=input_text, sampling_params=sampling_params)

print(outputs[0].outputs[0].text)

llama.cpp

我们提供了 MiniCPM3 的 GGUF 版本，可以直接使用 llama.cpp 推理。

安装 llama.cpp

1
2
3

  git clone https://github.com/ggerganov/llama.cpp
  cd llama.cpp
  make

推理

./llama-cli -c 1024 -m minicpm3-4b-fp16.gguf -n 1024 --top-p 0.7 --temp 0.7 --prompt "<|im_start|>user\n请写一篇关于人工智能的文章，详细介绍人工智能的未来发展和隐患。<|im_end|>\n<|im_start|>assistant\n"

模型微调

LLaMA-Factory

目前模型微调支持 LLaMA-Factory，使用方法参考 LLaMA-Factory 微调。

进阶功能

对于以下进阶功能，我们的样例代码中使用 vLLM 进行推理。

工具调用

我们提供了使用 MiniCPM3 调用工具的示例代码：

1
2

cd demo/minicpm3/function_call
python function_call.py

如果你想启动一个能够调用工具的推理服务，使用以下代码：

cd demo/minicpm3/function_call
pip install -r requirements.txt
python openai_api_server.py \
    --model openbmb/MiniCPM3-4B \
    --served-model-name MiniCPM3-4B \
    --chat-template chatml.jinja \
    --dtype auto \
    --api-key token-abc123 \
    --tensor-parallel-size 1 \
    --trust-remote-code

下面是一个调用搜索工具回答问题的演示：

代码解释器

我们提供了一个 MiniCPM3 使用代码解释器的示例代码：

1
2
3

cd demo/minicpm3/code_interpreter
pip install -r requirements.txt
python code_interpreter.py openbmb/MiniCPM3-4B

下面是一个使用代码解释器生成二维码的演示：

MiniCPM 2.0

查看 MiniCPM 2.0 的详细信息

MiniCPM 2.0 系列模型对 MiniCPM 进行了多个维度的升级，包括以下模型版本：

MiniCPM-2B-128k：将 MiniCPM-2B 的上下文长度从 4k 扩展至 128k，在 InfiniteBench 测试集上优于 ChatGLM3-6B-128k、Yi-6B-200k 等更大参数量的模型。
MiniCPM-MoE-8x2B：基于 MiniCPM-2B 进行 MoE 扩展，综合表现相比于 MiniCPM-2B 平均提高 4.5 个百分点。
MiniCPM-1B：相比于 MiniCPM-2B 成本下降 60%，综合表现仍然优于 LLaMA2-13B。
MiniCPM-S-1B：在保持下游任务性能无损的前提下，FFN 层实现了 87.89% 的平均稀疏度，将 FFN FLOPs 降低了 84%。结合 PowerInfer 推理框架，解码速度提升约 2.8 倍。

评测结果

MiniCPM-2B-128k 模型评测

Model	avg	avg w/o code&math	passkey	number_string	kv_retrieval	longbook_choice_eng	longbook_qa_chn	longbook_qa_eng	longbook_sum_eng	longdialogue_qa_eng	math_find	code_debug	code_run
LWM-Text-128k	24.45	33.62	100	97.8	0.6	28.82	15.93	14.31	9.99	1.5	3.43	20.05	1
Yarn-Mistral-7b-128k	19.84	27.36	92.71		0	27.95	15.49	9.55	9.06	7.5	17.14	0.76	1.25
Mistral-7B-Instruct-v0.2(ABF 1000w)	27.75	36.9	100	78.98	3.6	37.12	11.74	17.37	21.12	9.5	29.43	17.51	0
Yi-6B-200k	22.15	32.54	100	94.92	0	36.68	15.07	9.2	0.92	3.5	4.29	0.51	0.75
chatglm3-6b-128k	25.58	36.57	89.93	99.66	5.2	46.29	10.7	8.38	25.91	6.5	8	5.33	1
MiniCPM-2.4B-128k	27.32	37.68	98.31	99.83	9	29.69	23.06	16.33	15.73	9.5	4.29	22.08	0

MiniCPM-MoE-8x2B 模型评测

Model	BBH	MMLU	CEval	CMMLU	HumanEval	MBPP†	GSM8K	MATH
Llama2-34B*	44.1	62.6	-	-	22.6	33.0	42.2	6.24
Mistral-7B-Instruct-v0.2	39.81	60.51	42.55	41.92	36.59	39.63	40.49	4.95
Gemma-7B*	55.1	64.3	-	-	32.3	44.4	46.4	24.3
Qwen1.5-7B*	40.2	61	74.1	73.1	36	37.4	62.5	20.3
Deepseek-MoE(16B)*	-	45.0	40.6	42.5	26.8	39.2	18.8	4.3
MiniCPM-2.4B	36.87	53.46	51.13	51.07	50.00	35.93	53.83	10.24
MiniCPM-MoE-8x2B	39.22	58.90	58.11	58.80	55.49	41.68	61.56	10.52

注：* 表示结果取自技术报告。† 表示评测集为MBPP全集。

MiniCPM-S-1B 评测结果

代码生成：在 HumanEval（0-shot）和 MBPP（3-shot）上的平均 pass@1 得分。
常识推理：在 PIQA、SIQA、HellaSwag、WinoGrande 和 COPA 上的平均 0-shot 准确率。
阅读理解：在 BoolQ、LAMBADA 和 TyDi QA 上的平均 0-shot 准确率。

其他测试集：我们报告在GSM8K（8-shot）、MMLU（5-shot）、BBH（3-shot）和 AGI-Eval（0-shot）上的平均准确率。

Setting	Average Sparsity	Average Performance	Code Generation	Commonsense Reasoning	Reading Comprehension	GSM8K	MMLU	BBH	AGI Eval
LLaMA2-7B	-	37.96	16.37	69.59	61.87	12.96	44.45	32.96	27.53
ReluLLaMA-7B	66.98	37.62	15.85	69.64	70.54	5.84	38.64	35.07	27.73
ProSparse-7B*	88.11	38.31	19.47	66.29	63.33	12.74	45.21	33.59	27.55
ProSparse-7B	89.32	38.46	19.42	66.27	63.50	12.13	45.48	34.99	27.46
LLaMA2-13B	-	44.06	20.19	72.58	71.55	22.21	54.69	37.89	29.33
ReluLLaMA-13B	71.56	42.74	20.19	70.44	73.29	18.50	50.58	37.97	28.22
ProSparse-13B*	87.97	45.07	29.03	69.75	67.54	25.40	54.78	40.20	28.76
ProSparse-13B	88.80	44.90	28.42	69.76	66.91	26.31	54.35	39.90	28.67
MiniCPM-1B	-	44.44	36.85	63.67	60.90	35.48	50.44	35.03	28.71
MiniCPM-S-1B*	86.25	44.72	41.38	64.55	60.69	34.72	49.36	34.04	28.27
MiniCPM-S-1B	87.89	44.72	42.04	64.37	60.73	34.57	49.51	34.08	27.77

注：

ReluLLaMA-7B 和 ReluLLaMA-13B 的下载链接分别是 7B and 13B。“ProSparse-7B*"、“ProSparse-13B*” 和 “MiniCPM-S-1B*” 代表没有激活阈值偏移的 ProSparse 版本。
对于 PIQA、SIQA、HellaSwag、WinoGrande、COPA、BoolQ、LAMBADA、TyDi QA 和 AGI-Eval，我们根据各个选项的 PPL 来进行答案选择。对于 GSM8K、MMLU 和 BBH，我们直接生成答案。

模型推理

HuggingFace、vLLM推理

参考 MiniCPM 1.0 中的模型推理部分。

Powerinfer 推理

针对 MiniCPM-S-1B 模型，我们可以使用 Powerinfer 进行推理加速，使用方法如下：

保证cmake版本3.17以上，如果已经安装过，则跳过此步骤

  # 下载安装包
  sudo wget https://cmake.org/files/v3.23/cmake-3.23.0.tar.gz
  # 解压安装包
  sudo tar -zxvf cmake-3.23.0.tar.gz
  # 配置安装环境
  sudo ./configure
  sudo make -j8
  # 编译安装
  sudo make install
  # 查看安装后版本
  cmake --version
  # 返回版本号则安装成功
  #cmake version 3.23.0

安装powerinfer：

1
2
3

  git clone https://github.com/SJTU-IPADS/PowerInfer
  cd PowerInfer
  pip install -r requirements.txt # install Python helpers' dependencies

cpu版本powerinfer编译,如果你的机器只有cpu，或者只想使用cpu进行推理，则运行以下命令：

1
2

  cmake -S . -B build
  cmake --build build --config Release

gpu版本powerinfer编译,如果你的机器有gpu，则可以运行以下命令：

1
2

  cmake -S . -B build -DLLAMA_CUBLAS=ON
  cmake --build build --config Release

获取稀疏模型

1
2
3

git clone https://huggingface.co/openbmb/MiniCPM-S-1B-sft-gguf/tree/main
#or
git clone https://modelscope.cn/models/OpenBMB/MiniCPM-S-1B-sft-gguf

模型推理：

cd PowerInfer
# 以下是命令模版，output_token_count为最大输出tokens，thread_num 为线程数，prompt为输入prompt字符
#./build/bin/main -m /PATH/TO/MODEL -n $output_token_count -t $thread_num -p $prompt
# 以下是示例
./build/bin/main -m /root/ld/ld_model_pretrain/1b-s-minicpm/MiniCPM-S-1B-sft.gguf -n 2048 -t 8 -p '<用户>hello,tell me a story please.<AI>'

MiniCPM 1.0

查看 MiniCPM 1.0 的详细信息

MiniCPM-2B 语言模型有 24亿（2.4B）的非词嵌入参数量, 总计 2.7B 参数量。

经过 SFT 后，MiniCPM-2B 在公开评测集上与 Mistral-7B 表现相近（中文、数学、代码能力更优），整体性能超越 Llama2-13B、MPT-30B、Falcon-40B 等模型。
经过 DPO 后，MiniCPM-2B 在 MTBench 上也超越了 Llama2-70B-Chat、Vicuna-33B、Mistral-7B-Instruct-v0.1、Zephyr-7B-alpha 等众多代表性开源大模型。

注意：为了保证在学术研究用途上模型的通用性，我们未对 MiniCPM-2B 进行任何身份认同训练。同时由于我们用 ShareGPT 开源语料作为部分训练数据，模型可能会输出类似 GPT 系列模型的身份认同信息。

评测结果

评测设置

由于大模型评测难以统一，且大量评测也没有公开的prompt和测试代码，对于具体评测方式，我们只能尽量做到适合各类模型。
整体而言，我们测试时采用统一的prompt输入，并按照各模型对应的模板进行输入调整。
评测脚本及prompt已开源在我们的Github仓库中，也欢迎更多开发者来不断改进我们的评测方式。
- 文本评测部分，采用了我们的开源大模型能力评测框架UltraEval。以下为开源模型复现流程：
  - 安装UltraEval
    1 2 3
    
    git clone https://github.com/OpenBMB/UltraEval.git cd UltraEval pip install -e .
  - 下载相关数据并解压处理
    1 2 3
    
    wget -O RawData.zip "https://cloud.tsinghua.edu.cn/f/71b5232264ae4833a4d0/?dl=1" unzip RawData.zip python data_process.py
  - 执行评测脚本(提供了模板，可自定义)
    1
    
    bash run_eval.sh

部署模式

因为MiniCPM采用Mup的结构，与现有模型在具体计算上有细微差别，我们是基于vllm=0.2.2版本进行了我们模型的实现。
对于非MiniCPM模型，我们采用了vllm=0.2.7的最新版本进行推理。

评测度量

对于QA任务（选择题任务），我们选用两种方式进行测试：
- PPL：将选项作为题目生成的延续，并根据各个选项的PPL来进行答案选择；
- 第二种是直接生成答案选项。
对于不同模型，这两种方式得到的结果差异较大。MiniCPM两种模式上的结果较为接近，而Mistral-7B-v0.1等模型在PPL上表现较好，直接生成上效果较差。
在具体评测时，我们以两种评测方式得分的最高者为最终结果，以此保证对比的公平性(以下表格中*号表示采用PPL)。

文本模型评测

越级比较:

模型	平均分	英文均分	中文均分	C-Eval	CMMLU	MMLU	HumanEval	MBPP	GSM8K	MATH	BBH	ARC-E	ARC-C	HellaSwag
Llama2-7B	35.40	36.21	31.765	32.42	31.11	44.32	12.2	27.17	13.57	1.8	33.23	75.25	42.75	75.62*
Qwen-7B	49.46	47.19	59.655	58.96	60.35	57.65	17.07	42.15	41.24	5.34	37.75	83.42	64.76	75.32*
Deepseek-7B	39.96	39.15	43.64	42.82	44.45	47.82	20.12	41.45	15.85	1.53	33.38	74.58*	42.15*	75.45*
Mistral-7B	48.97	49.96	44.54	46.12	42.96	62.69	27.44	45.2	33.13	5.0	41.06	83.92	70.73	80.43*
Llama2-13B	41.48	42.44	37.19	37.32	37.06	54.71	17.07	32.55	21.15	2.25	37.92	78.87*	58.19	79.23*
MPT-30B	38.17	39.82	30.72	29.34	32.09	46.56	21.95	35.36	10.31	1.56	38.22	78.66*	46.08*	79.72*
Falcon-40B	43.62	44.21	40.93	40.29	41.57	53.53	24.39	36.53	22.44	1.92	36.24	81.94*	57.68	83.26*
MiniCPM-2B	52.33	52.6	51.1	51.13	51.07	53.46	50.00	47.31	53.83	10.24	36.87	85.44	68.00	68.25

同级比较：

模型	平均分	英文均分	中文均分	C-Eval	CMMLU	MMLU	HumanEval	MBPP	GSM8K	MATH	BBH	ARC-E	ARC-C	HellaSwag
TinyLlama-1.1B	25.36	25.55	24.525	25.02	24.03	24.3	6.71	19.91	2.27	0.74	28.78	60.77*	28.15*	58.33*
Qwen-1.8B	34.72	31.87	47.57	49.81	45.32	43.37	7.93	17.80	19.26	2.42	29.07	63.97*	43.69	59.28*
Gemini Nano-3B	-	-	-	-	-	-	-	27.2(report)	22.8(report)	-	42.4(report)	-	-	-
StableLM-Zephyr-3B	43.46	46.31	30.62	30.34	30.89	45.9	35.37	31.85	52.54	12.49	37.68	73.78	55.38	71.87*
Phi-2-2B	48.84	54.41	23.78	23.37	24.18	52.66	47.56	55.04	57.16	3.5	43.39	86.11	71.25	73.07*
MiniCPM-2B	52.33	52.6	51.10	51.13	51.07	53.46	50.00	47.31	53.83	10.24	36.87	85.44	68.00	68.25

Chat模型比较：

模型	平均分	英文均分	中文均分	C-Eval	CMMLU	MMLU	HumanEval	MBPP	GSM8K	MATH	BBH	ARC-E	ARC-C	HellaSwag
ChatGLM2-6B	37.98	35.17	50.63	52.05	49.21	45.77	10.37	9.38	22.74	5.96	32.6	74.45	56.82	58.48*
Mistral-7B-Instruct-v0.1	44.36	45.89	37.51	38.06	36.96	53.56	29.27	39.34	28.73	3.48	39.52	81.61	63.99	73.47*
Mistral-7B-Instruct-v0.2	50.91	52.83	42.235	42.55	41.92	60.51	36.59	48.95	40.49	4.95	39.81	86.28	73.38	84.55*
Qwen-7B-Chat	44.93	42.05	57.9	58.57	57.23	56.03	15.85	40.52	42.23	8.3	37.34	64.44*	39.25*	74.52*
Yi-6B-Chat	50.46	45.89	70.995	70.88	71.11	62.95	14.02	28.34	36.54	3.88	37.43	84.89	70.39	74.6*
Baichuan2-7B-Chat	44.68	42.74	53.39	53.28	53.5	53	21.34	32.32	25.25	6.32	37.46	79.63	60.15	69.23*
Deepseek-7B-chat	49.34	49.56	48.335	46.95	49.72	51.67	40.85	48.48	48.52	4.26	35.7	76.85	63.05	76.68*
Llama2-7B-Chat	38.16	39.17	33.59	34.54	32.64	47.64	14.02	27.4	21.15	2.08	35.54	74.28	54.78	75.65*
MiniCPM-2B	52.33	52.6	51.10	51.13	51.07	53.46	50.00	47.31	53.83	10.24	36.87	85.44	68.00	68.25

DPO后模型比较：

模型	MT-bench
GPT-4-turbo	9.32
GPT-3.5-turbo	8.39
Mistral-8*7b-Instruct-v0.1	8.30
Claude-2.1	8.18
Zephyr-7B-beta	7.34
MiniCPM-2B	7.25
Vicuna-33B	7.12
Zephyr-7B-alpha	6.88
LLaMA-2-70B-chat	6.86
Mistral-7B-Instruct-v0.1	6.84
MPT-34B-instruct	6.39

快速上手

在线体验

Colab

基于Gradio的网页版Demo

使用如下命令启动基于Gradio的网页版demo：

# generation powered by vllm
python demo/minicpm/vllm_based_demo.py --model_path <vllmcpm_repo_path>
# generation powered by huggingface
python demo/minicpm/hf_based_demo.py --model_path <hf_repo_path>

HuggingFace 推理

MiniCPM-2B

安装transformers>=4.36.0以及accelerate后，运行以下代码：

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)

path = 'openbmb/MiniCPM-2B-dpo-bf16'
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map='cuda', trust_remote_code=True)

responds, history = model.chat(tokenizer, "山东省最高的山是哪座山, 它比黄山高还是矮？差距多少？", temperature=0.5, top_p=0.8, repetition_penalty=1.02)
print(responds)

MiniCPM-2B （Llama Format）

我们将MiniCPM的模型权重转化成了Llama代码可以直接调用的格式，以便大家尝试:

import torch
from transformers import LlamaTokenizerFast, LlamaForCausalLM
model_path = "openbmb/MiniCPM-2B-dpo-bf16-llama-format"
tokenizer = LlamaTokenizerFast.from_pretrained(model_path)
model = LlamaForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map='cuda', trust_remote_code=True)

prompt="Now you act like a terminal situated within a beginner's C++ practice repository folder, please provide the output for the command: `ls -l`"
input_ids = tokenizer.encode("<用户>{}<AI>".format(prompt), return_tensors='pt', add_special_tokens=True).cuda()
responds = model.generate(input_ids, temperature=0.3, top_p=0.8, repetition_penalty=1.02, max_length=1024)
responds = tokenizer.decode(responds[0], skip_special_tokens=True)
print(responds)

vLLM 推理

安装 vLLM。

`1`	`pip install "vllm>=0.4.1"`

具体推理代码见这里。

SGLang 推理

安装 SGLang。

首先需要启动一个服务:

`1`	`python -m sglang.launch_server --model-path openbmb/MiniCPM-2B-dpo-fp16 --trust-remote-code --port 30000`

下面是一个推理代码的样例:

from sglang import function, gen, set_default_backend, RuntimeEndpoint

@function
def text_qa(s, question):
    s += "<用户>" + question + "<AI>"
    s += gen("answer", max_tokens=1024, temperature=0.7, top_p=0.7)

set_default_backend(RuntimeEndpoint("http://localhost:30000"))

state = text_qa.run(
    question="What is the capital of China?",
)

print(state["answer"])

llama.cpp、Ollama、fastllm、mlx_lm推理

MiniCPM支持llama.cpp 、ollama、fastllm、mlx_lm推理。感谢@runfuture对llama.cpp和ollama的适配。

请参考 MiniCPM 知识库中的边端部署教程。

模型量化

请参考 MiniCPM 知识库中的量化指南。

模型微调

一张 1080/2080 可实现高效参数微调：代码
mlx 微调：教程
xtuner: MiniCPM高效率微调的不二选择
LLaMA-Factory：MiniCPM微调一键式解决方案

开源协议

模型协议

本仓库中代码与 MiniCPM 模型权重依照 Apache-2.0 协议开源

声明

作为一个语言模型，MiniCPM 通过学习大量的文本来生成内容，但它无法理解、表达个人观点或价值判断，它所输出的任何内容都不代表模型开发者的观点和立场。
因此用户在使用 MiniCPM 生成的内容时，应自行负责对其进行评估和验证。
如果由于使用 MiniCPM 开源模型而导致的任何问题，包括但不限于数据安全问题、公共舆论风险，或模型被误导、滥用、传播或不当利用所带来的任何风险和问题，我们将不承担任何责任。

开发机构

本项目由以下机构共同开发：

工作引用

如果觉得 MiniCPM 有助于您的工作，请引用我们的论文：MiniCPM1，MiniCPM4

@article{minicpm4,
  title={MiniCPM4: Ultra-Efficient LLMs on End Devices},
  author={MiniCPM Team},
  year={2025}
}

@inproceedings{huminicpm,
  title={MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies},
  author={Hu, Shengding and Tu, Yuge and Han, Xu and Cui, Ganqu and He, Chaoqun and Zhao, Weilin and Long, Xiang and Zheng, Zhi and Fang, Yewei and Huang, Yuxiang and others},
  booktitle={First Conference on Language Modeling},
  year={2024}
}

ModelScope on Producthunt daily

SkyReels-V2

SkyworkAI/SkyReels-V2

SkyReels V2: Infinite-Length Film Generative Model

🔥🔥🔥 News!!

🎥 Demos

📑 TODO List

🚀 Quickstart

Installation

Model Download

Single GPU Inference

Multi-GPU inference using xDiT USP

Contents

Abstract

Methodology of SkyReels-V2

Key Contributions of SkyReels-V2

Video Captioner

Reinforcement Learning

Diffusion Forcing

High-Quality Supervised Fine-Tuning (SFT)

Performance

Human Evaluation

VBench

Acknowledgements

Citation

Qwen3-Coder

QwenLM/Qwen3-Coder

Qwen3-Coder: Agentic Coding in the World.

Introduction

Basic information

Quick Start

👉🏻 Chat with Qwen3-Coder-480B-A35B-Instruct

Fill in the middle with Qwen3-Coder-480B-A35B-Instruct

Use Cases

Example: Physics-Based Chimney Demolition Simulation with Controlled Explosion

Example: Multicolor and Interactive Animation

Example: 3D Google Earth

Example: Testing Your WPM with a Famous Quote

Example: Bouncing Ball in Rotation Hypercube

Example: Solar System Simulation

Example: DUET Game

Star History

Citation

Contact Us

MiniCPM

OpenBMB/MiniCPM

中文 | English

更新日志🔥

目录

模型下载

MiniCPM 4.0

评测结果

效率评测

综合评测

长文本评测

BitCPM4: 模型量化

BitCPM4 评测

BitCPM4 模型推理

模型应用

MiniCPM4-Survey: 综述生成

使用与演示案例

评估

MiniCPM4-MCP: MCP增强的工具调用

使用与演示案例

评估

模型推理

CPM.cu

HuggingFace

vLLM

SGLang

模型微调

LLaMA-Factory

MiniCPM 3.0

评测结果

综合评测

工具调用能力

长文本能力

模型推理

Huggingface

SGLang（推荐）