Yudong Wang, Jichang Guo, Huan Gao, Huihui Yue
Underwater image enhancement has attracted much attention due to the rise of marine resource development in recent years. Benefit from the powerful representation capabilities of Convolution Neural Networks(CNNs), multiple underwater image enhancement algorithms based on CNNs have been proposed in the last few years. However, almost all of these algorithms employ RGB color space setting, which is insensitive to image properties such as luminance and saturation. To address this problem, we proposed Underwater Image Enhancement Convolution Neural Network using 2 Color Space (UICE^2-Net) that efficiently and effectively integrate both RGB Color Space and HSV Color Space in one single CNN. To our best knowledge, this method is the first to use HSV color space for underwater image enhancement based on deep learning. UIEC^2-Net is an end-to-end trainable network, consisting of three blocks as follow: a RGB pixel-level block implements fundamental operations such as denoising and removing color cast, a HSV global-adjust block for globally adjusting underwater image luminance, color and saturation by adopting a novel neural curve layer, and an attention map block for combining the advantages of RGB and HSV block output images by distributing weight to each pixel. Experimental results on synthetic and real-world underwater images show the good performance of our proposed method in both subjective comparisons and objective metrics. The code are available at https://github.com/BIGWangYuDong/UWEnhancement.
Yudong Wang, Zhe Yang, Wenhan Ma, Zhifang Sui, Liang Zhao
While reinforcement learning has unlocked unprecedented complex reasoning in large language models, it has also amplified their propensity for hallucination, creating a critical trade-off between capability and reliability. This work confronts this challenge by introducing a targeted RL framework designed to mitigate both intrinsic and extrinsic hallucinations across short and long-form question answering. We address extrinsic hallucinations (flawed internal knowledge) by creating a novel training set from open-ended conversions of TriviaQA. Concurrently, we tackle intrinsic hallucinations (unfaithfulness to context) by leveraging long-form texts from FineWeb in a fact-grounding reward scheme. To further bolster reliability, our framework explicitly rewards the model for refusing to answer unanswerable questions, thereby cultivating crucial cautiousness. Extensive experiments demonstrate that our methodology yields significant performance gains across a diverse suite of benchmarks, substantially reducing both hallucination types. Ultimately, this research contributes a practical framework for resolving the critical tension between advanced reasoning and factual trustworthiness, paving the way for more capable and reliable large language models.
Yudong Wang, Zixuan Fu, Hengyu Zhao, Chen Zhao, Chuyue Zhou, Xinle Lin, Hongya Lyu, Shuaikang Xue, Yi Yi, Yingjiao Wang, Zhi Zheng, Yuzhou Zhang, Jie Zhou, Chaojun Xiao, Xu Han, Zhiyuan Liu, Maosong Sun
The development of artificial intelligence can be viewed as an evolution of data-driven learning paradigms, with successive shifts in data organization and utilization continuously driving advances in model capability. Current LLM research is dominated by a paradigm that relies heavily on unidirectional scaling of data size, increasingly encountering bottlenecks in data availability, acquisition cost, and training efficiency. In this work, we argue that the development of AGI is entering a new phase of data-model co-evolution, in which models actively guide data management while high-quality data, in turn, amplifies model capabilities. To implement this vision, we propose a tiered data management framework, designed to support the full LLM training lifecycle across heterogeneous learning objectives and cost constraints. Specifically, we introduce an L0-L4 tiered data management framework, ranging from raw uncurated resources to organized and verifiable knowledge. Importantly, LLMs are fully used in data management processes, such as quality scoring and content editing, to refine data across tiers. Each tier is characterized by distinct data properties, management strategies, and training roles, enabling data to be strategically allocated across LLM training stages, including pre-training, mid-training, and alignment. The framework balances data quality, acquisition cost, and marginal training benefit, providing a systematic approach to scalable and sustainable data management. We validate the effectiveness of the proposed framework through empirical studies, in which tiered datasets are constructed from raw corpora and used across multiple training phases. Experimental results demonstrate that tier-aware data utilization significantly improves training efficiency and model performance. To facilitate further research, we release our tiered datasets and processing tools to the community.
Yudong Wang, Damai Dai, Zhifang Sui
Most work treats large language models as black boxes without in-depth understanding of their internal working mechanism. In order to explain the internal representations of LLMs, we propose a gradient-based metric to assess the activation level of model parameters. Based on this metric, we obtain three preliminary findings. (1) When the inputs are in the same domain, parameters in the shallow layers will be activated densely, which means a larger portion of parameters will have great impacts on the outputs. In contrast, parameters in the deep layers are activated sparsely. (2) When the inputs are across different domains, parameters in shallow layers exhibit higher similarity in the activation behavior than deep layers. (3) In deep layers, the similarity of the distributions of activated parameters is positively correlated to the empirical data relevance. Further, we develop three validation experiments to solidify these findings. (1) Firstly, starting from the first finding, we attempt to configure different prune ratios for different layers, and find this method can benefit model pruning. (2) Secondly, we find that a pruned model based on one calibration set can better handle tasks related to the calibration task than those not related, which validate the second finding. (3) Thirdly, Based on the STS-B and SICK benchmark, we find that two sentences with consistent semantics tend to share similar parameter activation patterns in deep layers, which aligns with our third finding. Our work sheds light on the behavior of parameter activation in LLMs, and we hope these findings will have the potential to inspire more practical applications.
Yudong Wang, Zixuan Fu, Jie Cai, Peijun Tang, Hongya Lyu, Yewei Fang, Zhi Zheng, Jie Zhou, Guoyang Zeng, Chaojun Xiao, Xu Han, Zhiyuan Liu
Data quality has become a key factor in enhancing model performance with the rapid development of large language models (LLMs). Model-driven data filtering has increasingly become a primary approach for acquiring high-quality data. However, it still faces two main challenges: (1) the lack of an efficient data verification strategy makes it difficult to provide timely feedback on data quality; and (2) the selection of seed data for training classifiers lacks clear criteria and relies heavily on human expertise, introducing a degree of subjectivity. To address the first challenge, we introduce an efficient verification strategy that enables rapid evaluation of the impact of data on LLM training with minimal computational cost. To tackle the second challenge, we build upon the assumption that high-quality seed data is beneficial for LLM training, and by integrating the proposed verification strategy, we optimize the selection of positive and negative samples and propose an efficient data filtering pipeline. This pipeline not only improves filtering efficiency, classifier quality, and robustness, but also significantly reduces experimental and inference costs. In addition, to efficiently filter high-quality data, we employ a lightweight classifier based on fastText, and successfully apply the filtering pipeline to two widely-used pre-training corpora, FineWeb and Chinese FineWeb datasets, resulting in the creation of the higher-quality Ultra-FineWeb dataset. Ultra-FineWeb contains approximately 1 trillion English tokens and 120 billion Chinese tokens. Empirical results demonstrate that the LLMs trained on Ultra-FineWeb exhibit significant performance improvements across multiple benchmark tasks, validating the effectiveness of our pipeline in enhancing both data quality and training efficiency.
Yudong Wang, Jichang Guo, Wanru He, Huan Gao, Huihui Yue, Zenan Zhang, Chongyi Li
Underwater object detection is a crucial and challenging problem in marine engineering and aquatic robot. The difficulty is partly because of the degradation of underwater images caused by light selective absorption and scattering. Intuitively, enhancing underwater images can benefit high-level applications like underwater object detection. However, it is still unclear whether all object detectors need underwater image enhancement as pre-processing. We therefore pose the questions "Does underwater image enhancement really improve underwater object detection?" and "How does underwater image enhancement contribute to underwater object detection?". With these two questions, we conduct extensive studies. Specifically, we use 18 state-of-the-art underwater image enhancement algorithms, covering traditional, CNN-based, and GAN-based algorithms, to pre-process underwater object detection data. Then, we retrain 7 popular deep learning-based object detectors using the corresponding results enhanced by different algorithms, obtaining 126 underwater object detection models. Coupled with 7 object detection models retrained using raw underwater images, we employ these 133 models to comprehensively analyze the effect of underwater image enhancement on underwater object detection. We expect this study can provide sufficient exploration to answer the aforementioned questions and draw more attention of the community to the joint problem of underwater image enhancement and underwater object detection. The pre-trained models and results are publicly available and will be regularly updated. Project page: https://github.com/BIGWangYuDong/lqit/tree/main/configs/detection/uw_enhancement_affect_detection.
Yudong Wang, Hongjiong Tian
In this work, we present a general technique for establishing the strong convergence of numerical methods for stochastic delay differential equations (SDDEs) in the infinite horizon. This technique can also be extended to analyze certain continuous function-valued segment processes associated with the numerical methods, facilitating the numerical approximation of invariant measures of SDDEs. To illustrate the application of these results, we specifically investigate the backward and truncated Euler-Maruyama methods. Several numerical experiments are provided to demonstrate the theoretical results.
Lingfeng Li, Manqi Ruan, Yudong Wang, Yuexin Wang
The rare $b\to sν\barν$ decays are sensitive to contributions of new physics (NP) and helpful to resolve the puzzle of multiple $B$ flavor anomalies. In this work, we propose to study the $b\to sν\barν$ transition at a future lepton collider operating at the $Z$ pole through the $B_s \to φν\barν$ decay. Using the $B_s\toφ$ decay form factors from lattice simulations, we first update the SM prediction of BR($B_s \to φν\barν)_{\mathrm{SM}}=(9.93\pm 0.72)\times 10^{-6}$ and the corresponding $φ$ longitudinal polarization fraction $F_{L,{\mathrm{SM}}}=0.53\pm 0.04$. Our analysis uses the full CEPC simulation samples with a net statistic of $\mathcal{O}(10^9)$ $Z$ decays. Precise $φ$ and $B_s$ reconstructions are used to suppress backgrounds. The results show that BR($B_s \to φν\barν)$ can be measured with a statistical uncertainty of $\mathcal{O}(\%)$ and an $S/B$ ratio of $\mathcal{O}(1)$ at the CEPC. The quality measures for the event reconstruction are also derived. By combining the measurement of BR($B_s \to φν\barν)$ and $F_L$, the constraints on the effective theory couplings at low energy are given.
Yudong Wang
The lacunarity is an interesting property of a formal series. We say a series is lacunary if "almost all" of its coefficients are zero. In this article we considered about the lacunarity of some eta-products like η(z)^2η(bz)^2, and proved that they are lacunary if and only if b is 1,2,3,4 or 16. Then We write them as linear combinations of some CM forms.
Yudong Wang, Hongjiong Tian
This paper investigates the approximation of stochastic delay differential equations (SDDEs) via the backward Euler-Maruyama (BEM) method under generalized monotonicity and Khasminskii-type conditions in the infinite horizon. First, by establishing the uniform moment boundedness and finite-time strong convergence of the BEM method, we prove that for sufficiently small step sizes, the numerical approximations strongly converge to the underlying solution in the infinite horizon with a rate of $1/2$, which coincides with the optimal finite-time strong convergence rate. Next, we establish the uniform boundedness and convergence in probability for the segment processes associated with the BEM method. This analysis further demonstrates that the probability measures of the numerical segment processes converge to the underlying invariant measure of the SDDEs. Finally, a numerical example and simulations are provided to illustrate the theoretical results.
Zhengyu Chen, Yudong Wang, Teng Xiao, Ruochen Zhou, Xuesheng Yang, Wei Wang, Zhifang Sui, Jingang Wang
Recent advancements in improving the reasoning capabilities of Large Language Models have underscored the efficacy of Process Reward Models (PRMs) in addressing intermediate errors through structured feedback mechanisms. This study analyzes PRMs from multiple perspectives, including training methodologies, scalability, and generalization capabilities. We investigate the interplay between pre-training and reward model training FLOPs to assess their influence on PRM efficiency and accuracy in complex reasoning tasks. Our analysis reveals a pattern of diminishing returns in performance with increasing PRM scale, highlighting the importance of balancing model size and computational cost. Furthermore, the diversity of training datasets significantly impacts PRM performance, emphasizing the importance of diverse data to enhance both accuracy and efficiency. We further examine test-time scaling strategies, identifying Monte Carlo Tree Search as the most effective method when computational resources are abundant, while Best-of-N Sampling serves as a practical alternative under resource-limited conditions. Notably, our findings indicate that PRMs trained on mathematical datasets exhibit performance comparable to those tailored for code generation, suggesting robust cross-domain generalization. Employing a gradient-based metric, we observe that PRMs exhibit a preference for selecting responses with similar underlying patterns, further informing their optimization.
Jingchun Zhou, Zongxin He, Kin-Man Lam, Yudong Wang, Weishi Zhang, ChunLe Guo, Chongyi Li
In this paper, we present a novel Amplitude-Modulated Stochastic Perturbation and Vortex Convolutional Network, AMSP-UOD, designed for underwater object detection. AMSP-UOD specifically addresses the impact of non-ideal imaging factors on detection accuracy in complex underwater environments. To mitigate the influence of noise on object detection performance, we propose AMSP Vortex Convolution (AMSP-VConv) to disrupt the noise distribution, enhance feature extraction capabilities, effectively reduce parameters, and improve network robustness. We design the Feature Association Decoupling Cross Stage Partial (FAD-CSP) module, which strengthens the association of long and short range features, improving the network performance in complex underwater environments. Additionally, our sophisticated post-processing method, based on Non-Maximum Suppression (NMS) with aspect-ratio similarity thresholds, optimizes detection in dense scenes, such as waterweed and schools of fish, improving object detection accuracy. Extensive experiments on the URPC and RUOD datasets demonstrate that our method outperforms existing state-of-the-art methods in terms of accuracy and noise immunity. AMSP-UOD proposes an innovative solution with the potential for real-world applications. Our code is available at https://github.com/zhoujingchun03/AMSP-UOD.
Zifan Song, Yudong Wang, Wenwei Zhang, Kuikun Liu, Chengqi Lyu, Demin Song, Qipeng Guo, Hang Yan, Dahua Lin, Kai Chen, Cairong Zhao
Open-source Large Language Models (LLMs) and their specialized variants, particularly Code LLMs, have recently delivered impressive performance. However, previous Code LLMs are typically fine-tuned on single-source data with limited quality and diversity, which may insufficiently elicit the potential of pre-trained Code LLMs. In this paper, we present AlchemistCoder, a series of Code LLMs with enhanced code generation and generalization capabilities fine-tuned on multi-source data. To achieve this, we pioneer to unveil inherent conflicts among the various styles and qualities in multi-source code corpora and introduce data-specific prompts with hindsight relabeling, termed AlchemistPrompts, to harmonize different data sources and instruction-response pairs. Additionally, we propose incorporating the data construction process into the fine-tuning data as code comprehension tasks, including instruction evolution, data filtering, and code review. Extensive experiments demonstrate that AlchemistCoder holds a clear lead among all models of the same size (6.7B/7B) and rivals or even surpasses larger models (15B/33B/70B), showcasing the efficacy of our method in refining instruction-following capabilities and advancing the boundaries of code intelligence.
Yudong Wang, Chang Ma, Qingxiu Dong, Lingpeng Kong, Jingjing Xu
With promising yet saturated results in high-resource settings, low-resource datasets have gradually become popular benchmarks for evaluating the learning ability of advanced neural networks (e.g., BigBench, superGLUE). Some models even surpass humans according to benchmark test results. However, we find that there exists a set of hard examples in low-resource settings that challenge neural networks but are not well evaluated, which causes over-estimated performance. We first give a theoretical analysis on which factors bring the difficulty of low-resource learning. It then motivate us to propose a challenging benchmark hardBench to better evaluate the learning ability, which covers 11 datasets, including 3 computer vision (CV) datasets and 8 natural language process (NLP) datasets. Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness on evaluating the weaknesses of neural networks. On NLP tasks, we surprisingly find that despite better results on traditional low-resource benchmarks, pre-trained networks, does not show performance improvements on our benchmarks. These results demonstrate that there are still a large robustness gap between existing models and human-level performance.
Zhengyu Chen, Siqi Wang, Teng Xiao, Yudong Wang, Shiqi Chen, Xunliang Cai, Junxian He, Jingang Wang
Traditional scaling laws in natural language processing suggest that increasing model size and training data enhances performance. However, recent studies reveal deviations, particularly in large language models, where performance improvements decelerate, which is a phenomenon known as sub-scaling. This paper revisits these scaling laws by examining the impact of data quality and training strategies on model performance. Through extensive empirical analysis of over 400 models, we identify high data density and non-optimal resource allocation as key factors contributing to sub-scaling. High data density leads to diminishing returns due to redundant information, while optimal resource allocation is crucial for sustained performance improvements. We propose a sub-optimal scaling law that better predicts performance in sub-scaling regimes, highlighting the importance of data quality and diversity.
Zhen Cao, Felix Aharonian, Yunxiang Bai, Yiwei Bao, Denis Bastieri, Xiaojun Bi, YuJiang Bi, Mr Bian WenYi, A. Butkevich, Chengmiao Cai, Wenyu Cao, Zhe Cao, Jin Chang, Jinfan Chang, Mr Aming Chen, Ensheng Chen, Mr Guo-Hai Chen, Mr Huaxi Chen, Liang Chen, Long Chen, Mingjun Chen, Mali Chen, Qihui Chen, Shi Chen, Suhong Chen, Songzhan Chen, Tianlu Chen, Xiaobin Chen, Xuejian Chen, Yang Chen, Ning Cheng, Yaodong Cheng, Ming Chung Chu, Mingyang Cui, Shuwang Cui, Xiaohong Cui, Yidong Cui, Benzhong Dai, Hongliang Dai, Z. G. Dai, Danzengluobu ., Yangxuan Diao, Xuqiang Dong, Kaikai Duan, Junhui Fan, Yi-Zhong Fan, Jun Fang, Jianhua Fang, Kun Fang, Cunfeng Feng, Hua Feng, Li Feng, Shaohui Feng, Xiaoting Feng, Yi Feng, Youliang Feng, Stefano Gabici, Bo Gao, Chuandong Gao, Qi Gao, Wei Gao, Weikang Gao, Maomao Ge, Ting-Ting Ge, Lisi Geng, Gwenael Giacinti, Guanghua Gong, Quanbu Gou, Minhao Gu, Fulai Guo, Jing Guo, Xiaolei Guo, Yiqing Guo, Yingying Guo, Yiang Han, Otto A. Hannuksela, Mariam Hasan, Huihai He, Hao-Ning He, Jiayin He, Xinyu He, Yu He, Sergio Hernández-Cadena, Bowen Hou, Chao Hou, Xian Hou, Hongbo Hu, Shicong Hu, Chen Huang, Daihui Huang, Jiajun Huang, Tian-Qi Huang, Wenjun Huang, Xingtao Huang, Xiaoyuan Huang, Yong Huang, Yi-Yun Huang, Xiaolu Ji, Huanyu Jia, Kang Jia, Houbing Jiang, Kun Jiang, Xiaowei Jiang, Zejun Jiang, Min Jin, Samy Kaci, Mingming Kang, Ivan Karpikov, Dmitry Khangulyan, Denis Kuleshov, Kirill Kurinov, Bingbing Li, Cheng Li, Mr Cong Li, Dan Li, Fei Li, Haibo Li, Huicai Li, Jian Li, Jie Li, Kai Li, Long Li, Rong-Lan Li, Sidai Li, Tianyang Li, Wenlian Li, Xiurong Li, Xin Li, Yizhuo Li, Yuan Li, Zhe Li, Zhuo Li, Enwei Liang, Yunfeng Liang, Su-Jie Lin, Bing Liu, Cheng Liu, Dong Liu, Dangbo Liu, Hu Liu, Haidong Liu, Jia Liu, Jiali Liu, Jiren Liu, Maoyuan Liu, Ruo-Yu Liu, Siming Liu, Wei Liu, Xi Liu, Mr Yi Liu, Yu Liu, Yinong Liu, Yu-Qing Lou, Qing Luo, Yu Luo, Hongkui Lv, Bo-Qiang Ma, Lingling Ma, Xinhua Ma, Jirong Mao, Zhen Min, Warit Mitthumsiri, Guobin Mou, Huijun Mu, Andrii Neronov, Kenny Chun Yu Ng, Mingyang Ni, Lin Nie, Mr Lejian Ou, Petchara Pattarakijw anich, Zhiyuan Pei, Jincan Qi, Mengyao Qi, Jiajun Qin, Ali Raza, Chongyang Ren, David Ruffolo, Alejandro Sáiz, Dmitri Semikoz, Lang Shao, Oleg Shchegolev, Yunzhi Shen, Xiangdong Sheng, Zhaodong Shi, Fuwen Shu, Huichao Song, Vladimir Stepanov Stepanov, Yang Su, Dongxu Sun, Hao Sun, Qinning Sun, Xiaona Sun, Zhibin Sun, Mr Nabeel Tabasam, Jumpei Takata, P. H. T. Tam, Honbin Tan, Qingwen Tang, Ruiyi Tang, Zebo Tang, Wenwu Tian, Chaonan Tong, Li-Hong Wan, Chao Wang, Guangwei Wang, Hongguang Wang, Jiancheng Wang, Ke Wang, Kai Wang, Kai Wang, Liping Wang, Lingyu Wang, Luyao Wang, Ran Wang, Wei Wang, Xianggao Wang, Xinjian Wang, Xiang-Yu Wang, Yang Wang, Yudong Wang, Zhonghai Wang, Zhongxiang Wang, Zheng Wang, Daming Wei, Jun-Jie Wei, Yongjian Wei, Tao Wen, Shan-Shan Weng, Chaoyong Wu, Hanrong Wu, Qingwen Wu, Sha Wu, Xue-Feng Wu, Yusheng Wu, Shaoqiang Xi, Jie Xia, Junji Xia, Guangman Xiang, Dixuan Xiao, Gang Xiao, Yuliang Xin, Yi Xing, Dingrong Xiong, Zheng Xiong, Donglian Xu, Renfeng Xu, Renxin Xu, Weili Xu, Liang Xue, Dahai Yan, Tian Yan, Chaowen Yang, Chuyuan Yang, Fengfan Yang, Lili Yang, Mingjie Yang, Ruizhi Yang, Ms Wen-xin Yang, Zihan Yang, Zhiguo Yao, Xuanang Ye, Liqiao Yin, Na Yin, Xiaohao You, Zhiyong You, Qiang Yuan, Hua Yue, Houdun Zeng, Tingxuan Zeng, Wei Zeng, Xiangtao Zeng, Min Zha, Binbin Zhang, Bing Zhang, Chao Zhang, Feng Zhang, Hongfei Zhang, Haiming Zhang, Hengying Zhang, Jianli Zhang, Li Zhang, Pengfei Zhang, Peipei Zhang, Rui Zhang, Shaoru Zhang, Shoushan Zhang, Weiyan Zhang, Xiao Zhang, Xiaopeng Zhang, Yi Zhang, Yong Zhang, Zhipeng Zhang, Jing Zhao, Lei Zhao, Lizhi Zhao, Shiping Zhao, Xiao-Hong Zhao, Zihao Zhao, Fu Zheng, Wenjuan Zhong, Bin Zhou, Hao Zhou, Jianeng Zhou, Meng Zhou, Ping Zhou, Rong Zhou, Xiaoxi Zhou, Xunxiu Zhou, Ben-Yang Zhu, Chengguang Zhu, Fengrong Zhu, Hui Zhu, Kejun Zhu, Yuan-chuan Zou, Xiong Zuo, Oscar Macias
Zhe Yang, Yichang Zhang, Yudong Wang, Ziyao Xu, Junyang Lin, Zhifang Sui
Large Language Models (LLMs) have demonstrated the capability to refine their generated answers through self-correction, enabling continuous performance improvement over multiple rounds. However, the mechanisms underlying how and why accuracy evolves during this iterative process remain unexplored. To fill this gap, we propose a probabilistic theory to model the dynamics of accuracy change and explain the performance improvements observed in multi-round self-correction. Through mathematical derivation, we establish that the accuracy after the $t^{th}$ round of self-correction is given by: $Acc_t = Upp - α^t(Upp - Acc_0),$ where $Acc_0$ denotes the initial accuracy, $Upp$ represents the upper bound of accuracy convergence, and $α$ determines the rate of convergence. Based on our theory, these parameters can be calculated and the predicted accuracy curve then can be obtained through only a single round of self-correction. Extensive experiments across diverse models and datasets demonstrate that our theoretical predictions align closely with empirical accuracy curves, validating the effectiveness of the theory. Our work provides a theoretical foundation for understanding LLM self-correction, thus paving the way for further explorations.
Xiaomi LLM-Core Team, :, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, Bingquan Xia
We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing models with up to 78B parameters. For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens) with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward signals. We identify the importance of incorporating high-quality reasoning data with long Chain-of-Thought into pre-training stages, and the benefits of mixed RL despite challenges in simultaneous multi-domain optimization. We also contribute a comprehensive evaluation suite covering 50+ tasks to promote reproducibility and advance the field. The model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.
Rang Li, Lei Li, Shuhuai Ren, Hao Tian, Shuhao Gu, Shicheng Li, Zihao Yue, Yudong Wang, Wenhan Ma, Zhe Yang, Jingyuan Ma, Zhifang Sui, Fuli Luo
Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing benchmarks, a fundamental question remains: can MLLMs truly visually ground with human-like sophistication, or are they merely pattern-matching on simplified datasets? Current benchmarks fail to capture real-world complexity where humans effortlessly navigate intricate references and recognize when grounding is impossible. To rigorously assess MLLMs' true capabilities, we introduce GroundingME, a benchmark that systematically challenges models across four critical dimensions: (1) Discriminative: distinguishing highly similar objects, (2) Spatial: understanding complex relational descriptions, (3) Limited: handling occlusions or tiny objects, and (4) Rejection: recognizing ungroundable queries. Through careful curation combining automated generation with human verification, we create 1,005 challenging examples mirroring real-world complexity. Evaluating 25 state-of-the-art MLLMs reveals a profound capability gap: the best model achieves only 45.1% accuracy, while most score 0% on rejection tasks. We explore two strategies for improvements: (1) test-time scaling selects optimal response by thinking trajectory to improve overall performance by up to 4.5%, and (2) data-mixture training boosts rejection accuracy from 0% to 27.9%. GroundingME thus serves as both a diagnostic tool revealing current limitations in MLLMs and a roadmap toward human-level visual grounding. Project page: https://groundingme.github.io
Di Zhang, Xun Wu, Shaohan Huang, Yudong Wang, Hanyong Shao, Yingbo Hao, Zewen Chi, Li Dong, Ting Song, Yan Xia, Zhifang Sui, Furu Wei
Semi-structured N:M sparsity and low-bit quantization (e.g., 1.58-bit BitNet) are two promising approaches for improving the efficiency of large language models (LLMs), yet they have largely been studied in isolation. In this work, we investigate their interaction and show that 1.58-bit BitNet is naturally more compatible with N:M sparsity than full-precision models. To study this effect, we propose Sparse-BitNet, a unified framework that jointly applies 1.58-bit quantization and dynamic N:M sparsification while ensuring stable training for the first time. Across multiple model scales and training regimes (sparse pretraining and dense-to-sparse schedules), 1.58-bit BitNet consistently exhibits smaller performance degradation than full-precision baselines at the same sparsity levels and can tolerate higher structured sparsity before accuracy collapse. Moreover, using our custom sparse tensor core, Sparse-BitNet achieves substantial speedups in both training and inference, reaching up to 1.30X. These results highlight that combining extremely low-bit quantization with semi-structured N:M sparsity is a promising direction for efficient LLMs. Code available at https://github.com/AAzdi/Sparse-BitNet