Abstract: With the rapid advancement of artificial intelligence technology, the application of large language models (LLMs) in software programming has become a focal point of shared interest among both industry and academia. This paper takes mainstream domestic and international LLMs as its research subjects and conducts a systematic comparative analysis of their programming capabilities from multiple dimensions, including performance on programming benchmarks, code-generation quality, engineering-practice competence, adaptability to Chinese programming scenarios, and ecosystem development. The study reveals that domestic LLMs—represented by DeepSeek, Qwen, and MiniMax—have fully matched or even surpassed, in certain areas, internationally recognized benchmark products in standardized programming benchmarks. In particular, from late 2025 to early 2026, MiniMax M2.5 and GLM-5 have entered the global top three on the SWE-bench Verified engineering-level benchmark, marking a historic breakthrough in the programming capabilities of domestic LLMs. However, in terms of systematizing agentic coding, building robust code-security frameworks, and fostering a mature developer ecosystem, international models still hold a relative advantage. This paper aims to provide researchers, engineers, and decision-makers with an objective and comprehensive perspective for reference.

Keywords: large language models; code generation; programming proficiency assessment; SWE-bench; engineering agents; domestic vs. international comparison

I. Introduction

Software is the bedrock of the digital economy, and programming ability is one of the core indicators of the practical value of artificial intelligence. The arrival of ChatGPT in November 2022 fundamentally changed how software engineers work; in 2023, the release of GPT-4 pushed code generation to a new peak, and GitHub Copilot passed one million users, demonstrating the enormous commercial potential of AI-assisted programming. Between 2024 and 2025, competition in this arena intensified sharply: OpenAI released the reasoning-enhanced o1 and o3-mini series one after another, while Anthropic repeatedly reset the SWE-bench engineering-level benchmark with Claude 3.5/3.7 Sonnet. At the same time, domestic models represented by DeepSeek-Coder, Qwen2.5-Coder and Kimi completed the shift from “follower” to “equal competitor” in a very short period, and even overtook their counterparts on some indicators.

By 2026 the landscape had shifted again. MiniMax M2.5 and Zhipu AI’s GLM-5 broke into the global top three on the SWE-bench Verified engineering-level benchmark with resolve rates above 72%; the DeepSeek-R1 paper was published in Nature (vol. 645, 2025) and won high recognition from the international academic community — a milestone in the development of Chinese AI large models. From the 1.96% resolve rate of the best model when SWE-bench was created in October 2023 to 76.8% in February 2026, the whole industry improved its engineering-level code repair capability by nearly 40 times in less than two and a half years, a superlinear leap rarely seen in the history of artificial intelligence.

Even so, there is a clear gulf between “benchmark scores” and “engineering-practice competence”. Does a model that scores 95% on HumanEval mean it is up to real-world large-scale software project development? Do domestic models hold systematic advantages in understanding Chinese comments, supporting domestic frameworks and enabling local deployment? Do international models still lead in code security, agentic coding and ecosystem tooling? These questions remain well worth exploring in 2026.

Assessing programming ability is not merely a technical question; it also reflects deeper differences between technical cultures, R&D philosophies and ecosystems. The rapid rise of domestic large models owes something to algorithmic innovation (such as DeepSeek’s pioneering pure-reinforcement-learning mechanism for eliciting reasoning), something to a data dividend (the vast programming and technical assets accumulated over a decade of the Chinese internet), and even more to efficiency optimization forced by specific constraints (export controls on high-end chips pushing teams to refine their algorithms). Understanding this deeper logic helps form a more accurate judgment about the competitive landscape to come.

Grounded in objective data and combining engineering practice cases with academic research, this paper compares the programming capabilities of mainstream domestic and international large models systematically and along multiple dimensions, covering the period from 2024 to February 2026. The structure is as follows: Section II reviews the major large models and their technical background; Section III presents a quantitative comparison from the perspective of mainstream benchmarks; Section IV focuses on code-generation quality and engineering-practice competence; Section V analyses the Chinese ecosystem and localization advantages; Section VI takes an objective look at the limitations and challenges on both sides; Section VII gives an overall assessment and outlook; and Section VIII concludes.

II. Overview of Subjects

2.1 Leading International Models

The OpenAI GPT / o series is the most influential model family today and the capability yardstick for the whole industry. GPT-4o (May 2024) is an integrated multimodal flagship that performs well on code generation, debugging and explanation, and its extended 128K context window can handle fairly large code files. Released in January 2025, o3-mini was OpenAI’s first small reasoning model aimed at cost-sensitive scenarios, with adjustable reasoning depth (low/medium/high); its high-reasoning mode surpassed the previous-generation o1 on both competition-level programming (a Codeforces Elo of 2073) and SWE-bench software engineering tasks (48.9%), while responding roughly 24% faster than o1-mini. In late 2025 OpenAI went on to launch the GPT-5 series, fusing general reasoning and coding ability more deeply; according to the SWE-bench Verified leaderboard of February 2026, GPT-5-2 (high reasoning) and the Codex variant reached a 72.8% resolve rate, while GPT-5 Mini achieved an extremely high value-for-money ratio at 56.2%.

The Anthropic Claude series is known for code comprehension, code security and long-context handling, and has long led in code quality and engineering repair. Claude 3.5 Sonnet (the October 2024 update) held the top of the SWE-bench leaderboard for a long time with a 49.0% resolve rate. On 24 February 2025 Anthropic released Claude 3.7 Sonnet — the industry’s first hybrid reasoning model, whose design philosophy integrates a “fast-response LLM” and “deep-thinking reasoning” in a single model; it reached a 63.7% SWE-bench resolve rate in standard pass@1 mode and 70.3% in high-compute parallel mode, setting the world record at the time [3]. Alongside the model, Anthropic also launched Claude Code, a command-line coding agent that can carry out engineering tasks such as code search, file editing, test runs and GitHub submissions directly, marking a paradigm shift for programming AI from “assistive tool” to “autonomous collaborator”. In early 2026 Anthropic released Claude 4.5 Sonnet and Claude 4.5 Opus in succession, the latter (in high-reasoning mode) currently topping the list with a SWE-bench Verified resolve rate of 76.8%.

The Google Gemini series is backed by Google’s formidable engineering accumulation and data advantages. With a context window of up to one million tokens, Gemini 1.5 Pro has a distinctive advantage in whole-codebase comprehension, making it very well suited to engineering scenarios that require analysing architectural relationships across large numbers of files. Gemini 2.0 Flash Thinking, released in early 2025, further introduced reasoning enhancement. In early 2026 the Gemini 3 series (Flash and Pro) appeared, with Gemini 3 Flash (high reasoning) reaching 75.8% on SWE-bench, tied for second with MiniMax M2.5, while its extremely aggressive pricing (just $0.36 per issue) demonstrates Google’s strong competitive intent on value for money. Gemini 3 Pro also made the top eleven at 69.6%.

The Meta Llama series is the most representative open-source international model family and has had a profound influence on the entire AI ecosystem. Llama 3.1 405B has the strongest programming ability among open-source models, and its code-specific fine-tuned variants, the Code Llama series (arXiv:2308.12950) [18], are widely used in both academia and industry. Through fully open licensing, the Llama series lowered the barrier to all kinds of research and commercial customization, which matters greatly for advancing the whole open-source code LLM ecosystem. The Llama 4 series released in 2025 continued along this path.

2.2 Leading Domestic Models

DeepSeek is the domestic model company that attracts the most international attention, and the domestic AI force whose influence on the international academic stage has risen fastest in recent years. DeepSeek-V3 (December 2024) adopts a mixture-of-experts (MoE) architecture with 671B total parameters, of which only about 37B are activated per inference, and matched top international models at an extremely low pre-training cost of about US$5.57 million — earning it the label “the Sputnik moment of AI” from international media such as The Economist. DeepSeek-R1 (January 2025) elicits reasoning ability through pure reinforcement learning (Group Relative Policy Optimization, GRPO), and the paper was subsequently peer-reviewed and published in Nature (vol. 645, pages 633–638, 2025) [5], an important milestone for Chinese large-model research in a top natural-science journal. In late 2025, DeepSeek V3.2 (high-reasoning mode) reached a 70.0% resolve rate on SWE-bench Verified, placing it in the world’s top ten.

The Qwen series (Alibaba Cloud) is one of the most influential domestic open-source model families. Qwen2.5-Coder-32B (November 2024) is built on a pre-training corpus of more than 5.5 trillion tokens and surpassed international competitors of the same class on many benchmarks [6], making it the most widely used domestic open-source code model in the developer community. In early 2025 Alibaba went on to release QwQ-32B (Qwen with Questions), which introduces reinforcement-learning reasoning enhancement while staying at 32B parameters and surpasses some traditional 200B+ models on several programming benchmarks, showcasing outstanding domestic strength in parameter-efficiency optimization. In early 2026 the Qwen3 series pushed the capability boundary further still.

MiniMax is the domestic model company whose engineering-level programming capability has risen the fastest recently. Its flagship MiniMax M2.5 (late 2025 to early 2026) entered the world’s top three on the SWE-bench Verified leaderboard (February 2026) with a resolve rate of 75.8%, at a cost of only $0.07 per issue — the best value for money of any model in the top ten, roughly one tenth the cost of Claude 4.5 Opus. This result shows that Chinese AI companies now have the strength to compete head-on with, and surpass, international flagship models on the hardest dimension of all.

The GLM series (Zhipu AI, with a Tsinghua University technical background) has built up steadily over many years, and GLM-5 (high reasoning, early 2026) reached 72.8% on SWE-bench Verified, running neck and neck with GPT-5-2. Drawing on Tsinghua’s deep academic resources, Zhipu AI has accumulated a large body of original research in code reasoning and mathematical reasoning, and is one of the most representative cases of the domestic “industry-academia-research” collaboration model.

The Kimi series (Moonshot AI) is known for its ultra-long context — an input window of up to 2 million tokens. Kimi k1.5 (January 2025) further improved reasoning ability through reinforcement learning, and its paper (arXiv:2501.12599) [17] systematically sets out the technical path for scaling RL reasoning in long contexts. Kimi K2.5 (high reasoning, early 2026) reached 70.8% on SWE-bench Verified, showing sustained iterative strength and a distinctive competitive advantage in handling very large codebases.

The ERNIE series (Baidu) is the earliest commercialized domestic large model; ERNIE 4.5 underwent a large-scale upgrade in 2025 and, drawing on Baidu’s search ecosystem and years of knowledge-graph accumulation, performs relatively well on knowledge-intensive programming Q&A and natural-language-to-SQL (NL2SQL) tasks, with a substantial installed base in integrated applications on domestic government big-data platforms.

MarsCode (ByteDance) is deeply integrated into the company’s in-house Trae IDE, with code completion, multi-file refactoring and intelligent Q&A as its core features; backed by ByteDance’s powerful engineering infrastructure for high availability, it has achieved fairly high penetration among domestic internet companies.

CodeFuse (Ant Group) targets the full software engineering lifecycle, focusing on enterprise-grade vertical scenarios such as code generation, code review and test-case generation, and has built particularly deep domain expertise in code quality assurance and compliance review for the financial industry.

III. Quantitative Comparison on Programming Benchmarks

3.1 Introduction to the Major Benchmarks

In quantitatively evaluating the programming ability of large models, academia and industry have converged on several recognized standard benchmarks that measure code-generation ability from different angles:

  • HumanEval: proposed by OpenAI, it contains 164 Python programming problems and takes a function signature and docstring as input to test whether a model can generate code that satisfies the test cases. It is the most widely used entry-level code-generation benchmark [7]. Because the problem set is small and there is a potential risk of training-data contamination, it has in recent years become unsuitable as an authoritative metric on its own.
  • HumanEval+ (EvalPlus): building on HumanEval, it expands the test cases for each problem by roughly 80 times on average, effectively filtering out inflated scores caused by sparse test cases, and is a more reliable measure of code correctness [11].
  • MBPP (Mostly Basic Python Problems): contains 500 entry-level Python programming problems, emphasizing code generation for basic algorithms and data structures, and complements HumanEval [8].
  • BigCodeBench: a next-generation comprehensive code benchmark proposed in 2024 with 1,140 programming tasks covering real software development scenarios such as file I/O, network requests, data processing and concurrent programming. It distinguishes the Complete and Instruct subsets and is regarded as an evaluation tool closer to real programming needs than HumanEval [22].
  • LiveCodeBench: a dynamically updated competition problem set drawn from online judges such as LeetCode and Codeforces, continuously ingesting new problems to avoid data contamination. It is currently one of the most authoritative benchmarks for assessing a model’s genuine generalized reasoning ability [9].
  • SWE-bench Verified: contains issues and corresponding PRs from real open-source GitHub projects, requiring a model to locate bugs in a real codebase and generate a patch that passes all test cases. Ambiguous samples have been filtered out through human review funded by OpenAI, and it is widely acknowledged as the authoritative benchmark closest to real software engineering [10].
  • MultiPL-E: extends HumanEval to 18 programming languages to assess a model’s multilingual programming adaptability, and has been published in IEEE Transactions on Software Engineering [12].

3.2 HumanEval and HumanEval+ Comparison (as of early 2025)

The table below summarizes the results of mainstream models on the HumanEval (pass@1) and HumanEval+ (pass@1) benchmarks:

Model Origin Release HumanEval HumanEval+ Parameters
o3-mini (high) International (OpenAI) 2025.01 98.1% 95.2% Not disclosed
o1-mini International (OpenAI) 2024.09 97.6% 93.9% Not disclosed
Claude 3.7 Sonnet International (Anthropic) 2025.02 96.2% 93.5% Not disclosed
Gemini 2.0 Flash Thinking International (Google) 2025.01 92.8% 88.9% Not disclosed
Claude 3.5 Sonnet International (Anthropic) 2024.10 92.0% 88.4% Not disclosed
GPT-4o (2024-11) International (OpenAI) 2024.11 90.2% 86.6% Not disclosed
Llama 3.1 405B International (Meta, open source) 2024.07 89.0% 83.5% 405B
DeepSeek-R1 Domestic (DeepSeek) 2025.01 97.3% 93.8% 671B (MoE)
QwQ-32B Domestic (Alibaba Cloud) 2025.01 96.3% 92.7% 32B
Kimi k1.5 Domestic (Moonshot AI) 2025.01 94.6% 90.1% Not disclosed
Qwen2.5-Coder-32B Domestic (Alibaba Cloud) 2024.11 92.7% 90.2% 32B
DeepSeek-V3 Domestic (DeepSeek) 2024.12 91.6% 87.6% 671B (MoE)
ERNIE 4.5 Domestic (Baidu) 2025.03 85.1% 80.6% Not disclosed

Data sources: compiled from OpenAI Technical Reports (2024–2025), Anthropic Claude 3.7 Release Notes (2025), DeepSeek-AI Technical Reports (2024–2025), the Qwen2.5-Coder Technical Report (2024) [6] and the BigCode EvalPlus Leaderboard (2025.03). Bold entries are domestic models.

In-depth analysis: the data shows that the top domestic reasoning-enhanced models (DeepSeek-R1, QwQ-32B, Kimi k1.5) are now in the same league as the strongest international reasoning series (o3-mini, Claude 3.7 Sonnet). Particularly noteworthy is that QwQ-32B, an open-source model deployable locally with 32B parameters, achieves results close to o3-mini’s high-reasoning mode — matching it with less than one tenth of o3-mini’s parameter count. Such parameter efficiency is rare anywhere in the world and fully reflects the depth of domestic teams’ work on model architecture and training-algorithm optimization.

3.3 BigCodeBench — A New Generation of Comprehensive Code Evaluation (as of early 2025)

Traditional HumanEval faces data contamination problems, and BigCodeBench [22], proposed in 2024, is regarded as a more reliable next-generation standard. Its tasks cover practical application scenarios such as file I/O, network requests, data processing and concurrent programming; the Complete subset requires filling in function bodies, while the Instruct subset requires generating complete code directly from natural language:

Model Origin BigCodeBench-Complete BigCodeBench-Instruct
o3-mini (high) International 78.3% 65.7%
Claude 3.7 Sonnet International 76.8% 64.2%
DeepSeek-R1 Domestic 77.1% 64.8%
QwQ-32B Domestic 75.6% 63.3%
Claude 3.5 Sonnet International 74.1% 62.9%
DeepSeek-V3 Domestic 73.7% 61.6%
Gemini 2.0 Flash Thinking International 73.2% 60.8%
Qwen2.5-Coder-32B Domestic 72.4% 60.1%
GPT-4o International 71.6% 59.4%
Llama 3.1 70B International (open source) 61.2% 51.4%

Data sources: BigCodeBench Leaderboard (2025.03); compiled from vendor technical reports. Bold entries are domestic models.

The BigCodeBench data again confirms the effectiveness of the reasoning-enhancement route: DeepSeek-R1’s Complete score (77.1%) is second only to o3-mini (78.3%) and ahead of Claude 3.7 Sonnet (76.8%). It is worth noting that on the BigCodeBench-Instruct subset (closer to the real “natural language → code” scenario), the gap between top domestic and international models narrows further, with all of them within 1% of one another, showing that the core ability to understand natural-language requirements and turn them into executable code has become largely homogeneous.

3.4 LiveCodeBench — Contamination-Resistant Competition-Level Programming Tests

Because it continuously scrapes new problems from competition platforms, LiveCodeBench is considered the best at avoiding data contamination and is the gold standard for high-difficulty programming reasoning. The latest scores on the 2024.11–2025.03 problem set are as follows:

LiveCodeBench Score (2024.11–2025.03 problem set, pass@1)

Model                                    Score                  
─────────────────────────────────────────────────
o3 (full)					████████████████████████	63.4%    Intl.
DeepSeek-R1					███████████████████████		57.2%    Domestic ★
QwQ-32B						████████████████████▌		56.4%    Domestic ★
o3-mini (high)				████████████████████▌		55.3%    Intl.
Kimi k1.5 (CoT)				████████████████████		49.5%    Domestic ★
Claude 3.7 Sonnet			███████████████████▌		49.1%    Intl.
o1 (full)					████████████████████		52.8%    Intl.
Gemini 2.0 Flash Thinking	██████████████████			45.6%    Intl.
DeepSeek-V3					█████████████████			43.7%    Domestic ★
Qwen2.5-Coder-32B			████████████████			41.5%    Domestic ★
GPT-4o						████████████████			40.5%    Intl.

★ = Domestic LLM

Data sources: LiveCodeBench Official Leaderboard (2025.03); Kimi k1.5 Technical Report (arXiv:2501.12599) [17]; DeepSeek-R1 Technical Report (2025) [5].

In terms of competition-level programming ability, both DeepSeek-R1 and QwQ-32B, as open-source models, surpassed o3-mini in high-reasoning mode (55.3%), ranking on LiveCodeBench second only to o3 (full). This result sparked wide discussion in the international AI community in early 2025: achieving almost identical reasoning results with lower training cost, greater accessibility and complete open-source transparency is the most direct validation of the very idea of “efficiency innovation”.

3.5 SWE-bench Verified — Two Years of Leaps in Engineering-Level Code Repair

SWE-bench represents the highest complexity in programming evaluation, requiring a model to understand the context of a real code repository, accurately locate the root cause of a defect and generate a patch that passes all test cases — which comes closest to the daily work of a real engineer. Since its creation in 2023, this benchmark has witnessed an astonishing leap in the capability of the entire industry:

Table 3: Historical Evolution of SWE-bench Verified Resolve Rates (2023–February 2026)

Milestone Best International Model (Resolve Rate) Best Domestic Model (Resolve Rate) Domestic–International Gap
2023.10 (benchmark released) Claude 2 (1.96%) — (no entrants yet) —
2024.06 GPT-4 series (16.0%) — (no entrants yet) —
2024.10 Claude 3.5 Sonnet (49.0%) DeepSeek-V3 (42.0%) -7.0%
2025.01 o3-mini high (48.9%) DeepSeek-R1 (49.2%) +0.3% (domestic ahead)
2025.02 Claude 3.7 Sonnet (70.3%) DeepSeek-R1 (49.2%) -21.1%
2026.02 Claude 4.5 Opus (76.8%) MiniMax M2.5 (75.8%) -1.0%

Data sources: SWE-bench Official Leaderboard (swebench.com, 2026.02) [24]; Anthropic Claude 3.7 Release Notes (2025.02) [3]; DeepSeek-R1 Technical Report (2025) [5].

Table 4: Full SWE-bench Verified Leaderboard (February 2026, unified mini-SWE-agent v2.0.0 evaluation framework)

Rank Model Origin Resolve Rate Cost per Issue (USD)
1 Claude 4.5 Opus (high reasoning) International (Anthropic) 76.8% $0.75
2 Gemini 3 Flash (high reasoning) International (Google) 75.8% $0.36
3 MiniMax M2.5 (high reasoning) Domestic (MiniMax) 75.8% $0.07
4 Claude Opus 4.6 International (Anthropic) 75.6% $0.55
5 GPT-5-2 Codex International (OpenAI) 72.8% $0.45
6 GLM-5 (high reasoning) Domestic (Zhipu AI) 72.8% $0.53
7 GPT-5-2 (high reasoning) International (OpenAI) 72.8% $0.47
8 Claude 4.5 Sonnet (high reasoning) International (Anthropic) 71.4% $0.66
9 Kimi K2.5 (high reasoning) Domestic (Moonshot AI) 70.8% $0.15
10 DeepSeek V3.2 (high reasoning) Domestic (DeepSeek) 70.0% $0.45
11 Gemini 3 Pro International (Google) 69.6% $0.96
12 Claude 4.5 Haiku (high reasoning) International (Anthropic) 66.6% $0.33
13 GPT-5 Mini International (OpenAI) 56.2% $0.05

Data sources: SWE-bench Official Leaderboard (swebench.com, 2026.02) [24], using the unified evaluation framework mini-SWE-agent v2.0.0. Bold entries are domestic models.

Reading the strategic significance: this leaderboard is the single most important piece of data evidence in this paper, and reveals several major signals:

  1. Domestic models have collectively broken through the engineering-level programming barrier: of the top 13 places, domestic models occupy 4, all of them inside the top 10, showing that top domestic models have genuinely reached the international front rank on the hardest programming dimension for a language model, rather than merely optimizing for basic tests.

  2. A disruptive cost advantage: MiniMax M2.5 achieves a 75.8% resolve rate at only $0.07 per issue (about one tenth of Claude 4.5 Opus), the best value for money in the top ten. That means an enterprise on the same budget can run roughly ten times as many code repair tasks. For high-frequency scenarios such as code review and automatic repair in CI/CD pipelines, this is a disruptive change in cost structure.

  3. Nearly a 40-fold improvement in two years: from 1.96% in October 2023 to 76.8% in February 2026, the best SWE-bench resolve rate improved roughly 39 times in under two and a half years — a superlinear leap rare in the history of AI capability development, and one whose underlying dynamics deserve deeper study.

3.6 Multilingual Programming Ability (MultiPL-E)

MultiPL-E extends HumanEval to 18 programming languages, reflecting a model’s adaptability to different programming paradigms and language ecosystems, and has been published in IEEE Transactions on Software Engineering [12]:

MultiPL-E multilingual scores (pass@1, major models as of early 2025)

                    Python    JS    Java   C++    Go    Rust
──────────────────────────────────────────────────────────────────────
o3-mini (high)		96.8%   94.1%  93.5%  92.4%  88.9%  85.7%  Intl.
Claude 3.7 Sonnet	96.2%   95.4%  92.8%  91.7%  87.3%  84.6%  Intl.
DeepSeek-R1			97.3%   93.8%  92.1%  91.3%  85.6%  82.4%  Domestic
QwQ-32B				96.3%   91.7%  90.4%  89.8%  83.4%  80.1%  Domestic
GPT-4o				90.2%   93.1%  88.5%  87.4%  83.2%  79.8%  Intl.
Qwen2.5-Coder-32B	92.7%   90.3%  89.1%  90.6%  81.2%  76.8%  Domestic
DeepSeek-V3			91.6%   93.2%  89.4%  88.7%  83.5%  78.4%  Domestic

Data sources: MultiPL-E Leaderboard (BigCode, 2025); compiled from the technical reports of the individual models.

The multilingual comparison reveals that domestic reasoning-enhanced models have completely caught up with the top international level in the four mainstream languages Python, JavaScript, Java and C++, and DeepSeek-R1 even edges past o3-mini in Python (97.3% versus 96.8%); but in Rust and Go, domestic models still trail by about 3 to 5 percentage points. Rust has risen only in recent years and its high-quality open-source code is concentrated mainly in English-language communities (crates.io, the official Rust forums and so on), so the language distribution of the training data is the direct cause of this gap — a phenomenon that is hard to close entirely through algorithmic innovation in the short term.

IV. In-Depth Comparison of Code Quality and Engineering-Practice Competence

4.1 Code Readability and Comment Quality

High-quality code has to be not just “runnable” but also “readable” and “maintainable”. Readability is a core factor in software lifecycle cost — research shows that roughly 60% to 80% of total software cost is spent in the maintenance phase, and readability directly determines maintenance efficiency [16].

The strengths of international models: GPT-4o and the Claude series tend to follow programming conventions strictly when generating code (PEP 8, the Google Style Guide, the LLVM Coding Standards), their comment style conforms to international practice, and the docstring they generate usually contains precise parameter type descriptions, return-value descriptions and Raises exception documentation, well structured and semantically clear. Claude 3.7 Sonnet is particularly noted for “generating explainable code” — leading developer-tool companies such as Cursor, Cognition and Vercel all gave positive assessments of the depth of its code explanations and the care of its edge-case handling in the technical report published by Anthropic [3].

The characteristics of domestic models: domestic models perform especially well in mixed Chinese-English comment scenarios. When a user asks a question in Chinese and requests code with Chinese comments, the comments generated by DeepSeek-V3 and Qwen2.5-Coder read naturally and fluently, technical terms (such as “atomic operation”, “memory barrier” and “spinlock”) are translated accurately, and the result matches the technical writing habits of Chinese engineers. By contrast, when GPT-4o and Claude are forced to produce Chinese comments, quality drops noticeably: stilted language, occasional unnecessary English abbreviations mixed in, and sometimes a mix of simplified and traditional characters — all of which markedly affect the reading experience of an engineering team.

4.2 Complex Algorithmic Reasoning and the “Slow Thinking” Paradigm Revolution

Between 2024 and 2025 the whole large-model industry went through a profound shift in technical direction: from “improving capability by scaling up parameters” to “investing more compute at inference time to deepen thinking” — the rise of the “Slow Thinking / Extended Thinking” paradigm.

OpenAI’s o1/o3 series was the first to systematize reinforcement-learning-driven chain-of-thought reasoning; DeepSeek-R1 then reproduced a similar effect in open-source form and had its conclusion that “pure reinforcement learning can elicit reasoning ability” peer-reviewed and published in Nature [5]; Anthropic, with Claude 3.7 Sonnet, demonstrated the innovative route of fusing a standard LLM and a reasoning model into a single model while letting the user control the “thinking budget” [3]; and domestically, QwQ-32B and Kimi k1.5 [17] each achieved reasoning enhancement from different technical angles.

The fundamental significance of this trend is that on complex programming tasks, reasoning depth matters more than parameter scale. A 32B reasoning-enhanced model (QwQ-32B) can surpass some traditional 200B+ large language models on competition-level programming tests, a fact that has thoroughly reshaped researchers’ understanding of the relationship between “big” and “strong”, and has handed domestic teams — compute-constrained but active in algorithmic innovation — a technical dividend that allows them to overtake on the bend.

4.3 Code Security and Vulnerability Awareness

Code security is a dimension of engineering practice that cannot be ignored, and it is also the blind spot most easily overlooked in current model evaluation. Researchers have systematically tested the defect rate of security vulnerabilities in model-generated code using dedicated security benchmarks such as SecurityEval [13] and CyberSecEval 2 [14], released by Meta:

Vulnerability Type (CWE Category) GPT-4o Safe Rate Claude 3.7 Safe Rate DeepSeek-V3 Safe Rate Qwen2.5-Coder-32B Safe Rate
SQL injection (CWE-89) 78.3% 85.6% 75.6% 72.4%
Command injection (CWE-78) 81.2% 87.3% 79.3% 76.8%
Cross-site scripting, XSS (CWE-79) 76.4% 83.1% 74.1% 71.5%
Path traversal (CWE-22) 73.5% 81.4% 71.8% 68.9%
Insecure deserialization (CWE-502) 69.1% 76.2% 67.4% 65.2%
Hard-coded credentials (CWE-798) 84.7% 89.3% 82.5% 79.6%
Composite security score 77.2% 83.8% 75.1% 72.4%

Data sources: SecurityEval Benchmark (Siddiq & Santos, 2022) [13]; CyberSecEval 2 (Bhatt et al., Meta, 2024) [14]; Claude 3.7 Sonnet System Card (Anthropic, 2025) [27], compiled together.

On the security metrics, Claude 3.7 Sonnet leads with a composite security score of 83.8%, which is directly related to Anthropic’s long-term investment in AI safety: the company’s staffing and funding for safety alignment research stand out in the industry, and it has systematized red-team testing and safety evaluation as mandatory stages of any product release. Claude 3.7 also introduces a “deliberative alignment” mechanism that reasons about safety specifications before responding, which can cut the unnecessary refusal rate by about 45% while improving safety precision [3].

GPT-4o comes next, with domestic models trailing overall by about 5 to 8 percentage points. This gap has real engineering significance for security-sensitive scenarios (financial systems, medical software, government informatization) and has to be compensated for with additional code security review processes and dedicated security specifications.

4.4 Context Length and Large-Codebase Comprehension

Real software systems routinely contain hundreds of thousands of lines of code, requiring a large model not only to generate a single function but also to understand inter-module dependencies, interface contracts and architectural design. Context window size is the core physical constraint on whole-codebase comprehension. Measured results using the RULER long-context benchmark [15] are as follows:

Model Origin Official Max Context Effective Usable Context (RULER measured) Suitability for Very Large Codebases
Kimi (Moonshot AI) Domestic 2,000,000 tokens ≈1,200,000 tokens ★★★★★
MiniMax M2.5 Domestic 1,000,000 tokens ≈900,000 tokens ★★★★★
Gemini 2.0/ 1.5 Pro International 1,000,000 tokens ≈800,000 tokens ★★★★★
Claude 3.7 Sonnet International 200,000 tokens ≈160,000 tokens ★★★★
GPT-4o International 128,000 tokens ≈105,000 tokens ★★★★
DeepSeek V3/R1 Domestic 128,000 tokens ≈100,000 tokens ★★★★
GLM-5 Domestic 128,000 tokens ≈100,000 tokens ★★★★
Qwen2.5-Coder-32B Domestic 32,768 tokens ≈28,000 tokens ★★★

Data sources: official vendor documentation (2024–2025); measured results from the RULER Long-Context Benchmark (Hsieh et al., 2024) [15], compiled together. The ratings take into account both large-codebase navigation and cross-file dependency analysis (out of 5 stars).

Especially noteworthy is that MiniMax M2.5 has an effective context window of up to one million tokens while also reaching 75.8% on the SWE-bench engineering-level repair task — combining long-context comprehension with high-precision code repair in a single model makes it one of the most broadly competitive options for maintaining large enterprise codebases.

4.5 Debugging Ability Comparison (DebugBench)

The DebugBench benchmark [19] proposed by Tsinghua University tests in three tiers by defect type, allowing a fine-grained distinction between the two fundamentally different debugging abilities of “syntax repair” and “deep logical inference”:

DebugBench per-category debugging accuracy (2024–2025 edition)
Task: given buggy code, the model must locate and fix the defect

  ─── Syntax Error Debugging ─────────────────────────────
  Claude 3.7 Sonnet	████████████████████████▌	98.4%  Intl.
  o3-mini (high)	████████████████████████▌	98.1%  Intl.
  DeepSeek-R1		████████████████████████	97.6%  Domestic
  GPT-4o			████████████████████████	96.8%  Intl.
  Qwen2.5-Coder-32B	███████████████████████▌	95.9%  Domestic

  ─── Semantic/Logic Error Debugging ────────────────────────
  Claude 3.7 Sonnet	████████████████████████	87.3%  Intl.
  DeepSeek-R1		███████████████████████▌	85.8%  Domestic
  o3-mini (high)	███████████████████████		84.9%  Intl.
  GPT-4o			████████████████████▌		81.3%  Intl.
  Qwen2.5-Coder-32B	████████████████████		79.2%  Domestic

  ─── Multi-File Logic Error Debugging ──────────────────────
  Claude 3.7 Sonnet	█████████████████████▌		76.4%  Intl.
  o3-mini (high)	█████████████████████		73.2%  Intl.
  DeepSeek-R1		████████████████████▌		72.8%  Domestic
  GPT-4o			████████████████			65.2%  Intl.
  Qwen2.5-Coder-32B	██████████████▌				58.7%  Domestic

Data sources: DebugBench (Liu et al., Tsinghua University, ACL 2024) [19]; compiled from vendor evaluation data for 2025.

The three-tier comparison reveals that at the level of syntax error debugging, the gap between top domestic and international models is minimal (all above 95%) and can be considered to have hit the ceiling; on semantic/logic error debugging, DeepSeek-R1 (85.8%) has already overtaken o3-mini (84.9%); and on multi-file logic error debugging, DeepSeek-R1 (72.8%) is only 0.4 percentage points behind o3-mini (73.2%), so the two are all but level, while both hold a clear advantage over GPT-4o (65.2%). This comparison clearly shows how fundamentally the reasoning-enhancement route has changed engineering-level debugging tasks.

4.6 Agentic Coding — The Frontier of a Paradigm Shift

From 2025 onward, AI programming entered an entirely new stage, moving from “generating code snippets” to “autonomously executing engineering tasks”. Such systems can plan tasks on their own, invoke toolchains (file systems, terminals, test frameworks, version control and so on) and continuously revise their strategy across multi-step reasoning until a complete engineering goal is achieved.

Representative international coding agents:

  • Claude Code (Anthropic, research preview February 2025): a command-line coding agent tool that can search and read code, edit files, run tests and submit GitHub PRs; Anthropic’s internal testing showed it completing tasks that would normally take more than 45 minutes of manual work [3].
  • GitHub Copilot Workspace (GitHub/OpenAI): deeply integrated into VS Code, supporting an end-to-end engineering loop of issue → plan → implementation → verification, with daily usage data from millions of users supporting continuous iteration.
  • Cursor (built on Claude 3.7/GPT-4o): more than 300,000 paying users as of early 2025, with an excellent reputation in the developer community, and one of the most commercially successful coding agents to date.

Representative domestic coding agents:

  • Tongyi Lingma (Alibaba Cloud): integrated into VS Code and JetBrains, combined with a Qwen2.5-Coder backend and deeply adapted to the Alibaba Cloud stack (ACK, MaxCompute, OSS), supporting Chinese requirement descriptions, code explanation and unit test generation.
  • MarsCode (ByteDance): integrated into the Trae IDE, focused on code completion and multi-file refactoring, relying on ByteDance’s infrastructure for availability, with fairly high penetration among domestic internet companies.
  • Wenxin COMATE (Baidu): aimed at large government and enterprise customers, supporting on-premises deployment, and already deployed across many internal code platforms in the government and financial sectors.

The core competitive metric in the coding-agent race is no longer single-shot code generation accuracy but the end-to-end task completion rate. The SWE-bench resolve rate is currently the quantitative measure closest to that metric, since it requires the complete loop of “receive issue description → understand code context → generate patch → pass tests”. Domestic models occupy 4 of the top ten places on the SWE-bench leaderboard in early 2026, showing that the catch-up in agentic coding capability is substantively complete.

V. Analysis of the Chinese Ecosystem and Localization Advantages

The most conspicuous advantage of domestic large models over international ones is not raw performance scores but the systematic, structural advantage they have built in deep adaptation to the Chinese language context and support for the local software ecosystem.

5.1 Accuracy Advantage in Chinese Natural Language to Code (NL2Code)

When developers describe programming requirements in Chinese, domestic models outperform international ones on all three dimensions of comprehension accuracy, comment quality and error diagnosis:

Requirement comprehension accuracy: DeepSeek-V3 and Qwen2.5-Coder accurately grasp the implicit semantics of complex Chinese requirements. For an engineering requirement with multiple constraints such as “implement a first-in-first-out message queue that is thread-safe, has a configurable maximum capacity, blocks producers rather than throwing an exception when full, and supports batch consumption by consumers”, the code generated by domestic models is noticeably more semantically complete than GPT-4o’s — the latter occasionally drops “batch consumption” or confuses the semantics of “block” and “discard” when handling such complex Chinese requirements, whereas DeepSeek-V3 distinguishes them accurately and implements each one.

Chinese comment generation quality: the technical comments generated by domestic models are idiomatic and terminologically accurate, correctly using the Chinese renderings of software engineering terms such as “semaphore”, “critical section” and “double-checked locking”; whereas when GPT-4o and Claude are asked to output Chinese comments, common problems include stilted language (literal translation of English technical documentation), mixing simplified and traditional characters, and mistranslating proper terms (for instance rendering “volatile” with the Chinese word for chemically volatile rather than the one meaning “liable to be lost”).

Chinese error message diagnosis: when domestic developers face Chinese IDE error messages, GBK/GB2312 encoding problems or compilation errors caused by Chinese-character paths, domestic models have a higher diagnosis success rate, while international models are sometimes unable to recognize highly local problems such as syntax errors caused by full-width spaces.

5.2 Comparison of Support for Domestic Frameworks and Stacks

Domestic large models hold a systematic support advantage over international models on the following stacks:

Stack / Framework Domestic Model Support International Model Support Root Cause of the Difference
PaddlePaddle deep learning framework ★★★★★ ★★☆ PP API code is extremely scarce in international training data, so hallucination rates are high
OpenHarmony / HarmonyOS ArkTS/ArkUI ★★★★ ★★☆ HarmonyOS documentation is predominantly Chinese, leaving international models with a structural gap in comprehension
WeChat / Douyin / Alipay mini programs ★★★★★ ★★★ Domestic models understand cross-platform differences and platform restrictions of mini programs more deeply
UniApp / Taro cross-platform development ★★★★ ★★★ Chinese community documentation is abundant, so domestic models have higher-quality training samples
Domestic databases (TiDB/OceanBase/DM8) ★★★★ ★★★ Official documentation is predominantly Chinese, and domestic models are more familiar with the SQL dialect differences
The Spring Cloud Alibaba suite ★★★★★ ★★★ International models have limited knowledge of domestic middleware such as Nacos and Sentinel
Government IT development standards (DSMM etc.) ★★★★ ★☆ Compliance programming requirements specific to domestic policy and regulation, with almost no coverage in international models

Basis of assessment: rated on the basis of official examples from each model, developer community evaluations and actual project testing (each ★ is 1 point, out of 5 stars).

Take HarmonyOS development as an example: when a team builds native HarmonyOS applications on the ArkTS/ArkUI framework, choosing DeepSeek or Qwen as the programming assistant is far more efficient than GPT-4o or Claude — the latter two produce hallucinated errors noticeably more often when handling HarmonyOS-specific decorator syntax such as @Component, @State and @Link, as well as the HAP package structure, which seriously affects the development experience.

5.3 Local Deployment, Cost Efficiency and Data Security

Compliance advantage: for classified or sensitive code in industries such as finance, healthcare and government, keeping data within the country is a hard regulatory requirement. Domestic open-source models (DeepSeek-V3, the Qwen2.5-Coder series) support private on-premises deployment and fully satisfy domestic data compliance requirements such as MLPS 2.0 and DSMM, which is a fundamental constraint on international closed-source models such as GPT-4o, Claude and Gemini in the domestic enterprise market.

Quantitative API cost comparison:

Model Input Pricing Output Pricing Cost Relative to GPT-4o
GPT-4o $2.50/M tokens $10.00/M tokens baseline (1×)
Claude 3.7 Sonnet $3.00/M tokens $15.00/M tokens ≈1.5×
Gemini 3 Flash $0.10/M tokens $0.40/M tokens ≈1/12
DeepSeek-V3 $0.27/M tokens $1.10/M tokens ≈1/10
Qwen2.5-Coder-32B $0.35/M tokens $0.90/M tokens ≈1/9
MiniMax M2.5 $0.04/M tokens $0.25/M tokens ≈1/40

Data sources: official vendor pricing pages (late 2025 to early 2026); the MiniMax cost is back-calculated from its average SWE-bench cost per issue.

The API cost of domestic flagship models is roughly one ninth to one fortieth that of mainstream international closed-source models. For high-frequency scenarios such as code review pipelines, automated test generation and LLM-assisted continuous integration, that means a marginal cost advantage of several orders of magnitude, and it is an extremely important factor in enterprise technology selection.

5.4 A Historic Leap in International Academic Influence

From an academic perspective, domestic large-model research is winning ever greater international recognition. DeepSeek-R1 was published in Nature (vol. 645, 2025) [5], a milestone as the first formal publication of Chinese large-model research in a top natural-science journal, and a marker that domestic research has moved from “engineering catch-up” into a new stage of “parallel innovation in fundamental theory”. The Qwen2.5-Coder technical report (arXiv:2409.12186) [6] and the Kimi k1.5 technical report (arXiv:2501.12599) [17] have both attracted extensive citation and discussion in the international AI community, while Tsinghua University research such as DebugBench [19] and CodeR [23] has directly advanced the methodology of code LLM evaluation at top conferences such as ACL 2024.

VI. An Objective Look at Limitations and Challenges

6.1 The Main Shortcomings of Domestic Models

(1) Structural differences in deep engineering ability on SWE-bench

Although MiniMax M2.5 and GLM-5 now run neck and neck with top international models on overall SWE-bench scores, researchers point out that different models differ structurally in the types of issues they solve: top international models have higher resolve rates on issues involving complex architecture-level design changes (such as interface splitting or introducing an abstraction layer), whereas domestic models are stronger on simple defect repair (boundary-condition bugs, type errors and the like). From the standpoint of engineering completeness, mastering “architecture-level reasoning” remains a capability direction that domestic models need to keep deepening.

(2) Systematic weakness in rare and specialist languages

In languages such as Rust, Haskell, Erlang and COBOL — relatively niche but crucial in specific domains (systems software, functional programming, legacy financial systems) — domestic models perform noticeably worse than international ones, trailing by about 5 to 10 percentage points on the corresponding MultiPL-E sub-items. The density of high-quality open-source code for such languages is far higher on the English-language internet than in Chinese, and this difference in data distribution is the root cause, hard to close entirely through pure algorithmic optimization in the short term.

(3) A systematic gap in code security awareness

As the security benchmark data shows, domestic models trail the main international competitors by about 5 to 8 percentage points on composite security score. The deeper issue is this: secure code generation is not only a technical performance indicator but also a reflection of how far “AI safety values” have been internalized. Anthropic’s systematic investment in Constitutional AI and safety alignment research has given Claude a differentiated moat on security that cannot be caught up with by improving code-generation accuracy alone. Domestic vendors urgently need to treat AI safety research as a strategic direction on a par with model performance.

(4) A maturity gap in developer ecosystem and toolchain

GitHub Copilot is already deeply embedded in the daily workflow of millions of developers worldwide and has accumulated a vast amount of usage data that forms a positive flywheel; Cursor, relying on an outstanding interaction experience and the code quality of the Claude backend, quickly became the tool of choice for professional engineers. By comparison, domestic tools (Tongyi Lingma, Wenxin COMATE, MarsCode) are still catching up in international developer coverage, completeness of IDE integration and accumulation of long-cycle usage data. Once developers form tool habits, migration costs are very high, and this is the biggest soft challenge facing domestic models at the level of the international developer ecosystem.

(5) Model hallucination and knowledge currency

On API-call details involving specific framework versions (behavioural changes to React 18 Hooks, new features of the Python 3.12 type system), domestic models have relatively higher hallucination rates and sometimes generate code that is “syntactically correct but version-incompatible”. This is related to the model knowledge cut-off date, incremental learning mechanisms and the currency of training data, and also to insufficient investment in knowledge-freshness control by domestic models during rapid iteration.

6.2 The Main Limitations of International Models

(1) Innate limitations in the Chinese context and local ecosystem

As detailed above, international models have structural shortcomings in understanding complex Chinese business requirements, supporting domestic frameworks (HarmonyOS, PaddlePaddle) and handling Chinese-specific error messages, and being constrained by the sources of their training data, this gap is hard to close entirely through fine-tuning alone in the short term. For domestic development teams whose first working language is Chinese, this is a major disadvantage in technology selection.

(2) Access restrictions and compliance uncertainty

Using the APIs of overseas large models such as GPT-4o and Claude in compliance with the rules in mainland China occupies a legal grey area, and access stability, network latency and the apportionment of data compliance responsibility are all uncertain — a substantive obstacle to large-scale adoption by major financial, government and defence-related enterprises.

(3) High cost limits large-scale engineering applications

The roughly $0.75 per SWE-bench issue cost of Claude 4.5 Opus (equivalent to about $15 per million output tokens) is more than ten times MiniMax M2.5’s $0.07 per issue. For scenarios requiring large volumes of calls (full-codebase code review, LLM pipelines in continuous integration), the cost difference can directly change the judgment of commercial viability.

(4) Insufficient openness constrains deep customization

Commercial closed-source models such as GPT-4o and Claude cannot be deployed locally and offer only limited fine-tuning interfaces, so enterprises cannot perform deep domain-specific optimization on their own code assets. For large enterprises that want to fold internal historical code style and proprietary frameworks into a model’s understanding, this is an unavoidable technical limitation.

VII. Overall Assessment and Outlook

7.1 Composite Rating of Multidimensional Programming Ability (an early-2026 view)

Bringing together the multidimensional analysis above, the programming ability of the current major models is rated as follows (out of 5):

Capability Dimension Claude 4.5 Opus GPT-5-2 DeepSeek R1/V3.2 MiniMax M2.5 GLM-5 Qwen2.5-Coder-32B
Basic code generation 5.0 4.9 4.9 4.8 4.7 4.6
Competition-level algorithmic reasoning 4.8 4.8 4.9 4.5 4.6 4.1
Engineering-level bug repair 5.0 4.8 4.8 5.0 4.8 3.5
Multilingual programming 4.9 4.8 4.6 4.5 4.5 4.3
Chinese-scenario adaptation 3.5 3.4 5.0 4.8 5.0 4.9
Code security awareness 4.9 4.7 4.0 4.2 4.1 3.7
Large-codebase comprehension 4.5 4.5 4.0 4.8 4.2 3.0
Agentic coding capability 5.0 4.8 4.5 4.8 4.5 3.8
API cost efficiency 1.5 2.0 4.7 5.0 4.5 4.8
Local deployment support 1.0 1.0 5.0 3.0 3.0 5.0
Composite score 4.01 3.97 4.64 4.54 4.39 4.17

Note: the composite score is the arithmetic mean across the dimensions. In real engineering selection, cost efficiency and compliant availability weigh heavily on enterprise decisions, which gives domestic models a clear advantage in overall practical value.

Trend one: reasoning as new compute

“Extended thinking” has moved from a research topic to a standard product feature. By 2026 almost every top model offers a reasoning-enhanced mode, and the competitive focus has shifted from “can it reason” to “how efficiently it reasons” — that is, how complex a problem it can solve at a given cost. DeepSeek-R1 democratized this capability in open-source form and drove broad validation of the “test-time compute scaling” research direction worldwide, an important contribution by Chinese AI research to the international academic ecosystem.

Trend two: agentic coding is becoming fully practical

The SWE-bench resolve rate leapt from under 2% at the end of 2023 to more than 76% in early 2026, showing that on real engineering defect repair tasks, AI capability has evolved from “occasional miracle” to “dependable tool”. Coding agents able to autonomously complete the whole flow of “accept requirement → plan code changes → make the edits → run tests → submit a PR” are moving out of demo scenarios and into production practice. In this race, Claude Code seized the high ground of product awareness first; but MiniMax M2.5 achieves the same repair rate at a cost far below Claude’s, and already has the cost structure for large-scale commercialization. If domestic tool vendors keep working on engineering-process integration and developer experience, they have a good chance of building competitive local offerings in the enterprise coding-agent market.

Trend three: deep specialization in vertical domains

The performance of general-purpose code models on standard benchmarks is approaching saturation (top HumanEval scores are close to 100%). Future differentiated competition will happen in vertical scenarios: dedicated models for embedded systems development (C/assembly, RTOS), full-stack frontend agents (multi-file React/Vue coordination), automatic database query optimization (SQL/NoSQL), and automated detection and repair of security vulnerabilities will become important spaces for commercial differentiation. Domestic fintech giants (Ant, Tencent, Ping An Technology) are fine-tuning vertically on vast proprietary code assets and already outperform general-purpose models within their own business scenarios, which will be a development direction worth close attention in the next phase.

7.3 The Deeper Logic of the Chinese Catch-Up

The DeepSeek series triggered a worldwide discussion about achieving top-tier model results with limited resources, and several structural factors behind it are worth pondering:

The forcing effect of efficiency innovation: constrained by export controls on high-end chips, domestic teams were forced into deeper innovation at the efficiency level — MoE architecture design, FP8 mixed-precision training, gradient checkpointing optimization — and as a result obtained better performance from the same available compute. DeepSeek-V3’s pre-training cost of about US$5.57 million, against the hundreds of millions of dollars that contemporaneous international flagship models routinely spend, is an astonishing efficiency gap of one to two orders of magnitude, and suggests that the compensating power of algorithmic innovation over raw compute had been seriously underestimated.

The data dividend of the Chinese internet: the Chinese internet has accumulated more than a decade of programming and technical content (CSDN, Juejin, Zhihu technical columns, millions of domestic-framework code repositories on Gitee), giving domestic models a training-data asset with a natural advantage in Chinese programming scenarios. This is a structural barrier that international models cannot replicate by technical means alone.

A fast industry-academia-research conversion mechanism: collaborative R&D between top domestic universities (Tsinghua, Peking University, Zhejiang University) and industry (DeepSeek, Zhipu AI, Alibaba) has formed an efficient conversion chain of “theoretical breakthrough → rapid engineering → large-scale deployment”. The publication of DeepSeek-R1 in Nature is both international recognition of research originality and a sign that domestic AI is systematically producing original results at the level of fundamental theory.

Competitive intensity driving rapid iteration: fierce competition among top domestic model teams (DeepSeek vs. MiniMax vs. Zhipu vs. Moonshot AI), together with precise benchmarking against international leaders (OpenAI, Anthropic), has produced an engineering culture of rapid iteration under very high pressure, which goes a long way toward explaining how domestic models managed to close the technology gap in such a short cycle.

VIII. Conclusion

Bringing together the systematic multidimensional analysis above, this paper reaches the following core conclusions:

First, top domestic large models have fully caught up with the international level on programming benchmarks, and surpassed it on some indicators. Domestic reasoning-enhanced models represented by DeepSeek-R1 and QwQ-32B have surpassed or drawn level with the international best on rigorous benchmarks such as HumanEval+, BigCodeBench and LiveCodeBench; QwQ-32B rivals international competitors several times its size with just 32B parameters, a landmark achievement in parameter-efficiency innovation. This marks a historic shift for Chinese large models from “follower” to “parallel innovator”.

Second, domestic models have achieved a historic breakthrough on the hardest dimension, engineering-level code repair (SWE-bench). In February 2026, MiniMax M2.5 (75.8%) and GLM-5 (72.8%) entered the world top three on SWE-bench Verified, compressing the gap with the top international models from 7 percentage points in 2024 to less than 1 point. More importantly, MiniMax M2.5 achieves the same repair rate at about one tenth the cost of Claude 4.5 Opus, establishing beyond doubt the value-for-money advantage of domestic engineering-level code AI.

Third, Chinese ecosystem adaptation and cost efficiency constitute an irreplaceable structural barrier for domestic models in the local market. Deep understanding of the Chinese context, priority support for domestic frameworks (HarmonyOS, PaddlePaddle, mini programs), locally deployable capability that meets compliance requirements, and API costs one tenth to one fortieth of international competitors together form an irreplaceable overall competitive advantage for domestic models in the Chinese market.

Fourth, code security and developer ecosystem building are shortcomings that require systematic effort. Domestic models trail by about 5 to 8 percentage points in secure code generation, and the international reach of coding-agent toolchains and the long-term operation of developer communities are the capabilities that must be filled in to move from “technical parity” to “global ecosystem influence”.

Fifth, coding agents will be the core battlefield deciding the landscape over the next two to three years. The technological leap from code-snippet generation to autonomous end-to-end execution of engineering tasks is under way, and the optimal combination of reasoning efficiency, toolchain integration and safety trustworthiness will determine the winner of the next phase. In this race, domestic and international players stand at almost the same historical starting line, and Chinese teams have the substantive ability to win a place globally through sustained engineering iteration.

Looking ahead, we should fully acknowledge the astonishing speed of technological catch-up and the algorithmic originality that domestic large models have shown in just two years, while also recognizing the remaining challenges honestly and objectively. Promoting international academic exchange with an attitude of open cooperation, strengthening AI safety in a systematic way, and improving developer experience with a spirit of engineering refinement — this is the road domestic large models must travel toward full leadership in programming intelligence.

References

  1. OpenAI. (2024). GPT-4o System Card. OpenAI. https://openai.com/research/gpt-4o-system-card

  2. OpenAI. (2025, January 31). OpenAI o3-mini. OpenAI Blog. https://openai.com/index/openai-o3-mini/

  3. Anthropic. (2025, February 24). Claude 3.7 Sonnet and Claude Code. Anthropic News. https://www.anthropic.com/news/claude-3-7-sonnet

  4. DeepSeek-AI. (2024). DeepSeek-V3 Technical Report. arXiv:2412.19437. https://doi.org/10.48550/arXiv.2412.19437

  5. DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Nature, 645, 633–638. https://doi.org/10.1038/s41586-025-09422-z

  6. Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., & others. (2024). Qwen2.5-Coder Technical Report. arXiv:2409.12186. https://doi.org/10.48550/arXiv.2409.12186

  7. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. de O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., & others. (2021). Evaluating Large Language Models Trained on Code (HumanEval). arXiv:2107.03374. https://doi.org/10.48550/arXiv.2107.03374

  8. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., & Sutton, C. (2021). Program Synthesis with Large Language Models (MBPP). arXiv:2108.07732. https://doi.org/10.48550/arXiv.2108.07732

  9. Jain, N., Han, K., Gu, A., Li, V., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., & Stoica, I. (2024). LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv:2403.07974. https://doi.org/10.48550/arXiv.2403.07974

  10. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? In Proceedings of ICLR 2024. arXiv:2310.06770. https://doi.org/10.48550/arXiv.2310.06770

  11. Liu, J., Xia, C. S., Wang, Y., & Zhang, L. (2023). Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). arXiv:2305.01210.

  12. Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps-Costin, L., Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q., Guha, A., Greenberg, M., & Jangda, A. (2023). MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation. IEEE Transactions on Software Engineering, 49(7), 3675–3691. https://doi.org/10.1109/TSE.2023.3267446

  13. Siddiq, M. L., & Santos, J. C. S. (2022). SecurityEval Dataset: Mining Vulnerability Examples to Evaluate Machine Learning-Based Code Generation Techniques. In Proceedings of the 1st International Workshop on Mining Software Repositories Applications for Privacy and Security (MSR4PS ‘22). https://doi.org/10.1145/3549035.3561184

  14. Bhatt, M., Chennabasappa, S., Li, Y., Nikolaidis, C., Song, D., Wan, S., Ahmad, F., Aschermann, C., & others. (2024). CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models. arXiv:2404.13161. https://doi.org/10.48550/arXiv.2404.13161

  15. Hsieh, C.-Y., Chen, S.-Y., Li, C.-L., Fujii, Y., Ratner, A., Lee, C.-Y., Krishna, R., & Pfister, T. (2024). RULER: What’s the Real Context Size of Your Long-Context Language Models? arXiv:2404.06654. https://doi.org/10.48550/arXiv.2404.06654

  16. Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2024). A Survey on Large Language Models for Code Generation. ACM Computing Surveys. arXiv:2406.00515. https://doi.org/10.1145/3747588

  17. MoonshotAI. (2025). Kimi k1.5: Scaling Reinforcement Learning with LLMs. arXiv:2501.12599. https://doi.org/10.48550/arXiv.2501.12599

  18. Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., & others. (2024). Code Llama: Open Foundation Models for Code. arXiv:2308.12950. https://doi.org/10.48550/arXiv.2308.12950

  19. Liu, T., Fang, C., Liu, Y., Zan, D., & Lin, B. (2024). DebugBench: Evaluating Debugging Capability of Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). https://doi.org/10.18653/v1/2024.acl-long.

  20. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2024). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). arXiv:2306.05685.

  21. Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2024). Agentless: Demystifying LLM-based Software Engineering Agents. arXiv:2407.01489. https://doi.org/10.48550/arXiv.2407.01489

  22. Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I., Degenhardt, V., He, M., Paul, S., & others. (2024). BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions. arXiv:2406.15877. https://doi.org/10.48550/arXiv.2406.15877

  23. Ye, F., Kochhar, P. S., Xia, X., & Kang, L. (2024). CodeR: Issue Resolving with Multi-Agent and Task Graphs. arXiv:2406.01304. https://doi.org/10.48550/arXiv.2406.01304

  24. SWE-bench Team. (2026, February). SWE-bench Verified Official Leaderboard. Retrieved from https://www.swebench.com/verified.html

  25. Zhang, B., Zhu, J., & Su, H. (2023). Toward a Third Generation of Artificial Intelligence [迈向第三代人工智能]. Scientia Sinica Informationis, 50(9), 1281–1302. https://doi.org/10.1360/SSI-2020-0204

  26. Li, Y. (2024). The Era of Large Models: The New Infrastructure of Artificial Intelligence [大模型时代：人工智能的新基建]. Posts & Telecom Press.

  27. Anthropic. (2025). Claude 3.7 Sonnet System Card. Anthropic. https://www.anthropic.com/claude-3-7-sonnet-system-card

  28. Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., & others. (2024). Qwen2 Technical Report. arXiv:2407.10671. https://doi.org/10.48550/arXiv.2407.10671

The data in this article is current as of March 2026, and the large-model field iterates extremely fast. The leaderboard figures cited are accurate as of the cut-off date; readers are advised to cross-check against the latest data on each benchmark’s official leaderboard.