Open Source AI: SOTA Models 2025

By Ivan Traus

August 6, 2025

About this collection

## Open Source AI Models: A New Competitive Landscape This collection showcases the remarkable advancement of open source AI models across multiple domains, demonstrating how they now compete directly with proprietary systems from major tech companies. The landscape includes **general-purpose LLMs** like GLM-4.5 (355B parameters, 3rd globally), DeepSeek-V3-0324 (685B parameters with significant reasoning improvements), and Kimi K2 (1T total parameters with 32B active); **specialized coding models** such as Qwen3-Coder-480B-A35B-Instruct and DeepSeek-Coder-V2 that rival GPT-4 Turbo in coding tasks; and **multimodal embedding models** including jina-embeddings-v4 and Nomic Embed Multimodal that achieve state-of-the-art performance in visual document retrieval. Key themes emerging from this collection include the widespread adoption of **Mixture-of-Experts (MoE) architectures** for computational efficiency, **agentic capabilities** becoming a primary focus with models designed for tool use and autonomous problem-solving, **multimodal integration** enabling unified text-image processing, and **long context support** extending to millions of tokens. Performance benchmarks consistently show these open models matching or exceeding proprietary alternatives on specialized tasks, while leaderboard data confirms their competitive positioning across reasoning, coding, and retrieval benchmarks. This represents a fundamental shift in AI development, where open source models are no longer playing catch-up but are setting new standards and driving innovation in the field.

Curated Sources

Qwen-Image-Edit: Image Editing with Higher Quality and Efficiency | Qwen

Qwen-Image-Edit is an advanced image editing model built upon the 20B Qwen-Image model, extending its text rendering capabilities to image editing tasks. It supports both semantic and appearance editing, allowing for precise text editing in bilingual (Chinese and English) contexts. The model achieves state-of-the-art performance on multiple public benchmarks, demonstrating exceptional capabilities in tasks such as IP creation, object rotation, style transfer, and detailed element modification. Qwen-Image-Edit's applications include virtual avatar creation, novel view synthesis, and adjusting backgrounds or clothing in images. A key feature is its chained editing approach, enabling progressive correction of errors in generated images. The model's capabilities are showcased through various examples, including editing the Qwen mascot Capybara, creating MBTI-themed emoji packs, and correcting errors in calligraphy artworks.

Key Takeaways

Qwen-Image-Edit sets a new standard in image editing with its state-of-the-art performance on multiple benchmarks, offering both semantic and appearance editing capabilities.
The model's precise text editing feature, supported by its bilingual capabilities, opens up new possibilities for editing text within images while preserving original font, size, and style.
The chained editing approach demonstrated in correcting calligraphy artworks showcases the model's potential for iterative refinement and high-precision editing tasks.
Qwen-Image-Edit's applications in virtual avatar creation, style transfer, and object rotation highlight its versatility and potential for creative industries.
The model's ability to maintain semantic consistency while allowing for significant pixel changes enables diverse and original content creation, such as generating MBTI-themed emoji packs based on the Qwen mascot.

Qwen-Image Technical Report

Qwen-Image is a novel image generation foundation model that achieves significant advances in complex text rendering and precise image editing. It employs a comprehensive data pipeline, progressive training strategy, and enhanced multi-task training paradigm to improve text rendering capabilities and image editing consistency. The model demonstrates state-of-the-art performance across multiple public benchmarks, including GenEval, DPG, and OneIG-Bench for general image generation, as well as GEdit, ImgEdit, and GSO for image editing.

Key Takeaways

Qwen-Image redefines priorities in generative modeling by emphasizing precise alignment between text and image, particularly in text rendering, enabling future interfaces to evolve into vision-language VLUIs.
The model demonstrates strong generalization beyond 2D image synthesis, outperforming dedicated 3D models in novel view synthesis and maintaining coherence in pose editing tasks, essential for video generation.
Qwen-Image advances the vision of seamless integration between perception and creation, forming a balanced foundation for the next generation of multimodal AI when combined with Qwen2.5-VL, which excels in visual understanding.

Qwen/Qwen-Image-Edit · Hugging Face

Qwen-Image-Edit is an advanced image editing model built upon the 20B Qwen-Image model, extending its text rendering capabilities to image editing tasks. It supports both semantic and appearance editing, enabling precise text editing in multiple languages. The model achieves state-of-the-art performance in image editing tasks and has various applications, including IP creation, object rotation, style transfer, and virtual avatar creation. Qwen-Image-Edit can be used for tasks such as adding, removing, or modifying elements in images while preserving the original content. It also supports chained editing approaches for progressive error correction. The model is licensed under Apache 2.0 and can be accessed through various platforms, including Qwen Chat, Hugging Face, and ModelScope.

Key Takeaways

Qwen-Image-Edit's unique architecture enables both semantic and appearance editing, making it a powerful tool for various image editing tasks.
The model's precise text editing capabilities stem from Qwen-Image's expertise in text rendering, allowing for accurate editing of text in images.
Qwen-Image-Edit's applications extend beyond simple image editing to IP creation, virtual avatar creation, and other innovative uses, lowering technical barriers to visual content creation.

Qwen3 Technical Report

The Qwen3 series is a collection of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities. It includes models of both dense and Mixture-of-Expert (MoE) architectures, with parameter scales ranging from 0.6 to 235 billion. Qwen3 integrates thinking mode and non-thinking mode into a unified framework, allowing dynamic mode switching based on user queries or chat templates. The models are pre-trained on 36 trillion tokens covering up to 119 languages and dialects. Empirical evaluations demonstrate that Qwen3 achieves state-of-the-art results across diverse benchmarks, including tasks in code generation, mathematical reasoning, and agent tasks. The post-training pipeline involves a four-stage process: Long-CoT Cold Start, Reasoning RL, Thinking Mode Fusion, and General RL. Strong-to-Weak Distillation is used to optimize lightweight models.

Key Takeaways

Qwen3 models achieve state-of-the-art performance across various tasks and domains, outperforming larger MoE models and proprietary models.
The integration of thinking and non-thinking modes into a single model allows for dynamic mode switching based on user queries.
The thinking budget mechanism enables users to allocate computational resources adaptively during inference, balancing latency and performance.
Qwen3 expands multilingual support from 29 to 119 languages and dialects, enhancing global accessibility through improved cross-lingual understanding and generation capabilities.

zai-org/GLM-4.5 · Hugging Face

The GLM-4.5 series models are foundation models for intelligent agents, unifying reasoning, coding, and intelligent agent capabilities. GLM-4.5 has 355 billion total parameters with 32 billion active parameters, while GLM-4.5-Air has 106 billion total parameters and 12 billion active parameters. Both models provide two modes: thinking mode for complex reasoning and tool usage, and non-thinking mode for immediate responses. The models have been open-sourced under the MIT license and achieve exceptional performance in 12 industry-standard benchmarks, with GLM-4.5 scoring 63.2 and GLM-4.5-Air scoring 59.8. The base models, hybrid reasoning models, and FP8 versions have been released for commercial use and secondary development.

Key Takeaways

The GLM-4.5 models demonstrate exceptional performance in industry-standard benchmarks while maintaining efficiency, making them suitable for complex intelligent agent applications.
The open-sourcing of GLM-4.5 models under the MIT license enables commercial use and secondary development, potentially driving innovation in AI research and applications.
The hybrid reasoning capability of GLM-4.5 models allows for both complex reasoning and immediate responses, enhancing their versatility in various AI-driven tasks.

GLM-4.5: Reasoning, Coding, and Agentic Abililties

The document introduces GLM-4.5, a frontier AI model excelling in reasoning, coding, and agentic tasks. It features a hybrid reasoning model with thinking and non-thinking modes, achieving top performance in various benchmarks. GLM-4.5 is built with 355 billion total parameters and 32 billion active parameters, utilizing a MoE architecture. The model demonstrates superior capabilities in agentic tasks, complex reasoning, and coding, outperforming several competing models. It is available on Z.ai, Z.ai API, HuggingFace, and ModelScope. The document details the model's architecture, training methods, and performance comparisons, highlighting its advancements in AI capabilities.

Key Takeaways

GLM-4.5 achieves top-tier performance in agentic tasks, complex reasoning, and coding benchmarks, rivaling models like Claude 4 Sonnet and GPT-4.
The model's MoE architecture and reinforcement learning techniques enable efficient and scalable training, particularly for agentic capabilities.
GLM-4.5 demonstrates significant advancements in full-stack development, coding projects, and presentation material generation, showcasing its practical applications.

Open LLM Leaderboard 2025

The Open LLM Leaderboard displays the latest public benchmark performance for state-of-the-art (SOTA) open-sourced model versions released after April 2024. The data comes from model providers and independently run evaluations by Vellum or the AI community, featuring results from non-saturated benchmarks. The leaderboard compares various open-source models across different tasks such as reasoning, high school math, agentic coding, tool use, and adaptive reasoning. Models like Nemotron Ultra 253B, Llama 4 Behemoth, DeepSeek-R1, and Llama 4 Maverick are evaluated based on their performance in benchmarks like GPQA Diamond, AIME 2024, SWE Bench, BFCL, and GRIND. The comparison also includes metrics such as context size, cutoff date, I/O cost, max output, latency, and speed.

Key Takeaways

The Nemotron Ultra 253B model outperforms others in reasoning and adaptive reasoning tasks, scoring 76% in GPQA Diamond and 57.1% in GRIND.
DeepSeek-R1 excels in high school math and agentic coding, achieving 79.8% in AIME 2024 and 49.2% in SWE Bench.
Llama 3.1 405b shows strong performance in tool use with 81.1% in BFCL, while Llama 4 Scout is the fastest model with 2600 tokens per second.
The comparison highlights a trade-off between model performance and cost, with models like Gemma 3 27b being the cheapest and Llama 3.1 405b being the most expensive.

LLM Leaderboard - Comparison of over 100 AI models from OpenAI, Google, DeepSeek & others | Artificial Analysis

This document presents a comprehensive comparison and ranking of over 100 AI models (LLMs) from various creators including OpenAI, Google, DeepSeek, and others. The comparison is based on key metrics such as intelligence, price, performance, speed (output speed and latency), and context window. The document highlights frontier models, open weights, and size classes, and provides detailed rankings and analysis of the models' performance across these metrics. It includes a large table comparing features such as creator, context window, artificial analysis intelligence index, blended USD/1M tokens, median tokens per second, and median first chunk time for various models. The document also provides links to further analysis and model providers.

Key Takeaways

The comparison reveals that models like Grok 4, o3-pro, and Gemini 2.5 Pro are among the top performers in terms of intelligence and speed.
The pricing of AI models varies significantly, with some models like Grok 3 mini Reasoning (high) being very cost-effective at $0.35 per 1M tokens, while others like o1-pro are much more expensive at $262.50 per 1M tokens.
The context window of the models ranges from 4k to 2m, with larger context windows generally associated with more advanced models.
The document highlights the trade-offs between different metrics, such as intelligence, price, and speed, allowing users to choose models that best fit their specific needs.
The detailed comparison and analysis provide valuable insights for developers and researchers looking to select the most appropriate AI models for their applications.

Aider LLM Leaderboards | aider

The document presents performance metrics for various Large Language Models (LLMs) on the Aider polyglot coding leaderboard. Aider evaluates LLMs based on their ability to follow instructions and edit code successfully without human intervention using 225 challenging Exercism coding exercises across multiple programming languages. The results show that the o3-pro model achieved an 84.9% correct edit rate, while the gemini-2.5-pro-preview-06-05 model scored 83.1%. The document provides detailed statistics, including pass rates, error outputs, and cost metrics for the evaluated models.

Key Takeaways

The o3-pro model outperformed other evaluated LLMs with an 84.9% correct edit rate on the Aider polyglot benchmark.
The Aider polyglot benchmark provides a comprehensive evaluation of LLMs across multiple programming languages and coding exercises.
Performance metrics such as pass rates, error outputs, and cost provide valuable insights into the capabilities and limitations of different LLMs for code editing tasks.

Nomic Blog: Nomic Embed Multimodal: State-of-the-Art Multimodal Retrieval

Nomic Team released Nomic Embed Multimodal, a suite of models achieving state-of-the-art performance in embedding PDFs, images, papers, and charts. The models support interleaved text and image inputs, making them ideal for visually rich content. ColNomic Embed Multimodal 7B achieved 62.7 NDCG@5 on Vidore-v2, a visual document retrieval benchmark. The models simplify retrieval pipelines by embedding visual and textual content together, improving accuracy and reducing complexity.

Key Takeaways

Nomic Embed Multimodal models outperform previous state-of-the-art models by up to 2.8 points on visual document retrieval benchmarks.
The models simplify RAG workflows by embedding visual and textual content together, reducing preprocessing steps and complexity.
Techniques like sampling from the same source and hard negative mining improved multimodal embedding performance, with gains of up to 5.2 points on Vidore-v2 NDCG@5.

nomic-ai/colnomic-embed-multimodal-7b · Hugging Face

The ColNomic Embed Multimodal 7B model is a state-of-the-art multimodal embedding model that excels at visual document retrieval tasks. It achieves high performance with 62.7 NDCG@5 on Vidore-v2, outperforming other models. The model features unified text-image encoding, directly processing interleaved text and images without complex preprocessing. With 7 billion parameters, it is fine-tuned from Qwen2.5-VL 7B Instruct and offers a multi-vector output option for enhanced performance. The model seamlessly integrates with Retrieval Augmented Generation (RAG) workflows, allowing for direct document embedding and eliminating the need for OCR and complex processing. Recommended use cases include research papers, technical documentation, product catalogs, financial reports, and visually rich content. The model's performance may vary with unconventional layouts, non-English content, or complex documents.

Key Takeaways

The ColNomic Embed Multimodal 7B model's unified text-image encoding enables efficient processing of interleaved text and images, making it ideal for visually rich documents.
Its integration with RAG workflows simplifies document embedding and retrieval, capturing both textual and visual cues in a single embedding.
While the model excels at various document types, its performance may be affected by factors such as unconventional layouts, non-English content, and complex documents, highlighting areas for further exploration and potential improvement.

Jina Embeddings v4: Universal Embeddings for Multimodal Multilingual Retrieval

Jina Embeddings v4 is a 3.8 billion parameter universal embedding model that supports both text and image inputs, achieving state-of-the-art performance in multimodal and multilingual retrieval tasks. It features a unified architecture that processes both modalities through a shared pathway, eliminating the modality gap. The model includes task-specific LoRA adapters for retrieval, semantic similarity, and code tasks, and supports both single-vector and multi-vector embeddings. Jina Embeddings v4 outperforms leading closed-source models, including OpenAI's text-embedding-3-large and Google's gemini-embedding-001, particularly in visually rich document retrieval. The model is available through various channels, including the Jina API, CSP marketplaces, and Hugging Face.

Key Takeaways

The unified architecture of Jina Embeddings v4 eliminates the modality gap between text and images, enabling true multimodal processing and achieving a 0.71 cross-modal alignment score.
The model's performance is particularly strong in visually rich document retrieval, achieving 90.2 on ViDoRe and 80.2 on Jina-VDR benchmarks.
Jina Embeddings v4 outperforms leading closed-source models, delivering 12% better performance than OpenAI's text-embedding-3-large on multilingual retrieval and 28% improvement on long document tasks.

jinaai/jina-embeddings-v4 · Hugging Face

Jina Embeddings v4 is a universal embedding model for multimodal and multilingual retrieval, supporting text, images, and visual documents. It features unified embeddings, multilingual support for 30+ languages, and task-specific adapters for retrieval, text matching, and code-related tasks. The model is built on Qwen/Qwen2.5-VL-3B-Instruct and offers flexible embedding sizes. It can be used via the Jina AI Embeddings API, transformers, or sentence-transformers. The model is licensed under CC BY-NC 4.0 and available for commercial use through the Jina Embeddings API or by contacting the developers.

Key Takeaways

The model's multimodal capabilities enable retrieval across text, images, and visual documents, making it suitable for complex document retrieval tasks.
Task-specific adapters allow for flexible application in various tasks such as retrieval, text matching, and code understanding.
The release of Jina VDR, a multilingual, multi-domain benchmark for visual document retrieval, complements the model and provides a standardized evaluation framework.

jina-embeddings-v4 : Universal Embeddings for Multimodal Multilingual Retrieval

The document introduces jina-embeddings-v4, a 3.8 billion parameter multimodal embedding model that unifies text and image representations. It incorporates task-specific Low-Rank Adaptation (LoRA) adapters to optimize performance across diverse retrieval scenarios. The model achieves state-of-the-art performance on both single-modal and cross-modal retrieval tasks, particularly in processing visually rich content. A novel benchmark, Jina-VDR, is introduced to evaluate the model's capability in visually rich image retrieval.

Key Takeaways

jina-embeddings-v4 reduces the modality gap between text and image embeddings by using a unified encoder, resulting in improved cross-modal alignment.
The model's performance on visually rich document retrieval is state-of-the-art, thanks to its ability to process mixed-media formats and its training on a diverse dataset.
The introduction of Jina-VDR provides a comprehensive benchmark for evaluating embedding models on visually rich document retrieval tasks, extending beyond conventional question-answering tasks.

The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation

Meta introduces Llama 4, a new generation of multimodal AI models that include Llama 4 Scout and Llama 4 Maverick, both featuring a mixture-of-experts architecture and native multimodality. Llama 4 Scout is a 17 billion active parameter model with 16 experts, while Llama 4 Maverick is a 17 billion active parameter model with 128 experts. Both models outperform previous Llama models and other industry benchmarks in various tasks, including image and text understanding, coding, and reasoning. The models are available for download on llama.com and Hugging Face. A larger teacher model, Llama 4 Behemoth, with 288 billion active parameters, was used to distill the smaller models and demonstrates state-of-the-art performance on STEM benchmarks. The development of Llama 4 involved several innovations, including a new training technique called MetaP, improved vision encoder, and a revamped post-training pipeline. Meta also emphasizes the importance of safeguards and protections in AI development, including data filtering, system-level mitigations, and red-teaming exercises to address potential risks and biases.

Key Takeaways

The Llama 4 models represent a significant advancement in multimodal AI, offering improved performance and efficiency through their mixture-of-experts architecture and native multimodality.
The use of a larger teacher model, Llama 4 Behemoth, to distill the smaller Llama 4 models resulted in substantial quality improvements across various tasks.
Meta's emphasis on safeguards and protections in AI development, including data filtering and red-teaming exercises, highlights the importance of addressing potential risks and biases in AI models.

meta-llama/Llama-4-Maverick-17B-128E-Instruct · Hugging Face

The document outlines the Llama 4 Community License Agreement and provides detailed information about the Llama 4 AI models developed by Meta. The agreement specifies the terms and conditions for using, reproducing, distributing, and modifying the Llama Materials. The Llama 4 models are natively multimodal AI models that enable text and multimodal experiences, leveraging a mixture-of-experts architecture. The document highlights the intended use cases, model architecture, training data, benchmarks, quantization, safeguards, and critical risk areas associated with the Llama 4 models. The Llama 4 models are designed to be used for commercial and research purposes in multiple languages and are optimized for various tasks such as text and image understanding, captioning, and visual reasoning. The document also discusses the safety features and protections implemented in the Llama 4 models, including model-level fine-tuning, system prompts, and system-level protections. Additionally, the document provides information on the training data, energy use, and greenhouse gas emissions associated with the training of the Llama 4 models.

Key Takeaways

The Llama 4 models are natively multimodal AI models that enable text and multimodal experiences.
The Llama 4 Community License Agreement outlines the terms and conditions for using the Llama Materials.
The models are designed to be used for commercial and research purposes in multiple languages.
The Llama 4 models have been tested for various tasks such as text and image understanding, captioning, and visual reasoning.
The document highlights the safety features and protections implemented in the Llama 4 models, including model-level fine-tuning and system-level protections.

DeepSeek-V3-0324 Release | DeepSeek API Docs

The DeepSeek-V3-0324 release brings significant improvements in reasoning performance, front-end development skills, and tool-use capabilities. For non-complex reasoning tasks, it is recommended to use V3 with 'DeepThink' turned off. The API usage remains unchanged, and the models are released under the MIT License. The open-source weights are available on Hugging Face. This release is part of the DeepSeek series, following previous releases like DeepSeek-R1-0528.

Key Takeaways

The DeepSeek-V3-0324 model offers enhanced reasoning performance and development skills, making it suitable for complex tasks.
The release under the MIT License and availability of open-source weights on Hugging Face facilitate community engagement and development.
The recommendation to turn off 'DeepThink' for non-complex tasks suggests a strategic approach to optimizing model usage based on task complexity.

deepseek-ai/DeepSeek-V3-0324 · Hugging Face

The DeepSeek-V3-0324 model demonstrates significant improvements over its predecessor in reasoning capabilities, front-end web development, Chinese writing proficiency, Chinese search capabilities, and function calling. The model has 685B parameters and is licensed under the MIT License. Usage recommendations include specific system prompts, temperature settings, and prompt templates for file uploading and web search. The model supports features such as function calling, JSON output, and FIM completion.

Key Takeaways

The DeepSeek-V3-0324 model shows substantial improvements in benchmark performance across various tasks, indicating enhanced reasoning capabilities.
The model's temperature parameter is adjusted using a mapping mechanism to optimize performance for API calls.
Specific prompt templates are recommended for file uploading and web search tasks to maximize the model's effectiveness.

Kimi K2: Open Agentic Intelligence

Kimi K2 is a state-of-the-art Mixture-of-Experts model with 32 billion activated parameters and 1 trillion total parameters, achieving outstanding performance in knowledge, math, coding, and agentic tasks. It is meticulously optimized for agentic tasks and is open-sourced as Kimi-K2-Base and Kimi-K2-Instruct models. Kimi K2 demonstrates exceptional capabilities in tool use, math, and coding benchmarks, outperforming many proprietary models. The model is supported by innovative techniques such as the MuonClip optimizer, which stabilizes training by controlling attention logits, and a comprehensive pipeline for large-scale agentic data synthesis. Kimi K2 can be accessed through various interfaces, including the Kimi Platform API and self-deployment on inference engines like vLLM and TensorRT-LLM.

Key Takeaways

Kimi K2 achieves state-of-the-art performance in various benchmarks, including coding, math, and tool use tasks, often surpassing proprietary models like Claude Sonnet 4 and GPT-4.1.
The MuonClip optimizer addresses training instability in large-scale LLM training by introducing a qk-clip technique that rescale query and key projections, effectively preventing logit explosions.
Kimi K2's agentic capabilities are enhanced through large-scale agentic data synthesis and general reinforcement learning, enabling sophisticated tool-use and self-judging mechanisms.
The model is available through multiple access points, including the Kimi Platform API and self-deployment options, making it versatile for various applications.
Future developments for Kimi K2 include adding advanced capabilities such as thinking and visual understanding to further enhance its agentic intelligence.

MoonshotAI/Kimi-K2: Kimi K2 is the large language model series developed by Moonshot AI team

Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model developed by Moonshot AI team with 32 billion activated parameters and 1 trillion total parameters. It achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being optimized for agentic capabilities. The model is available in two variants: Kimi-K2-Base for fine-tuning and custom solutions, and Kimi-K2-Instruct for general-purpose chat and agentic experiences. Kimi K2 demonstrates superior performance in various benchmarks, including coding tasks, tool use tasks, math & STEM tasks, and general tasks. The model is released under the Modified MIT License, and its API is available on the Moonshot AI platform.

Key Takeaways

Kimi K2's exceptional performance is attributed to its large-scale training on 15.5T tokens and the use of the Muon optimizer.
The model's agentic intelligence capabilities make it suitable for tool use, reasoning, and autonomous problem-solving.
Kimi K2 outperforms other models in various benchmarks, including coding tasks and math & STEM tasks.
The model's API is available on the Moonshot AI platform, and it supports OpenAI/Anthropic-compatible API for easy integration.

moonshotai/Kimi-K2-Instruct · Hugging Face

Kimi K2 is a state-of-the-art Mixture-of-Experts (MoE) language model with 32 billion activated parameters and 1 trillion total parameters. It achieves exceptional performance across frontier knowledge, reasoning, and coding tasks. The model is trained with the Muon optimizer and is specifically designed for agentic capabilities, including tool use and autonomous problem-solving. Kimi K2 has two variants: Kimi-K2-Base for fine-tuning and custom solutions, and Kimi-K2-Instruct for general-purpose chat and agentic experiences. The model demonstrates superior performance in various benchmarks, including coding tasks, tool use tasks, math and STEM tasks, and general tasks. Kimi K2's API is available on Moonshot AI's platform, compatible with OpenAI and Anthropic APIs.

Key Takeaways

Kimi K2 outperforms other models in coding tasks, achieving a Pass@1 score of 53.7 on LiveCodeBench v6.
The model's agentic capabilities enable it to autonomously decide when and how to invoke tools, demonstrated through the tool_call_with_client function.
Kimi K2's performance in math and STEM tasks is exceptional, with a score of 97.4 on MATH-500 and 89.5 on AutoLogi.
The model's large-scale training and novel optimization techniques contribute to its state-of-the-art performance.

QwenLM/Qwen3-Coder: Qwen3-Coder is the code version of Qwen3, the large language model series developed by Qwen team, Alibaba Cloud.

Qwen3-Coder is a large language model developed by the Qwen team at Alibaba Cloud, designed for agentic coding tasks. It features a 480B-parameter Mixture-of-Experts model with 35B active parameters, supporting long-context understanding up to 256K tokens and 358 coding languages. The model is available in multiple sizes and offers exceptional performance in coding and agentic tasks, comparable to Claude Sonnet. It includes features like function calling, agentic browser-use, and tool-use capabilities. The model is designed to support various coding tasks, including code completion, generation, and understanding.

Key Takeaways

Qwen3-Coder sets new state-of-the-art results in agentic coding and related tasks among open models.
The model's long-context capabilities and support for multiple coding languages make it versatile for various coding applications.
The model's performance is comparable to Claude Sonnet, indicating its high capability in coding tasks.

Qwen3-Coder: Agentic Coding in the World | Qwen

The Qwen Team has announced Qwen3-Coder, a powerful agentic code model with a 480B-parameter Mixture-of-Experts variant, Qwen3-Coder-480B-A35B-Instruct. This model supports a context length of 256K tokens natively and 1M tokens with extrapolation methods. It achieves state-of-the-art results in Agentic Coding, Agentic Browser-Use, and Agentic Tool-Use tasks. Qwen3-Coder is accompanied by Qwen Code, a command-line tool for agentic coding, and can be integrated with various developer tools. The model has been trained on 7.5T tokens with a 70% code ratio and utilizes reinforcement learning to improve performance. Qwen3-Coder can be accessed through Alibaba Cloud Model Studio and used with OpenAI SDK or Claude Code.

Key Takeaways

Qwen3-Coder sets new benchmarks in agentic coding tasks with its advanced Mixture-of-Experts architecture and extensive training data.
The model's ability to handle long context lengths and its reinforcement learning-based training significantly enhance its coding capabilities.
Qwen3-Coder's integration with developer tools like Qwen Code and Claude Code enables seamless agentic coding experiences.
The model's performance on SWE-Bench Verified highlights its potential for real-world software engineering applications.
Future developments include releasing more model sizes and exploring self-improvement capabilities for the Coding Agent.

Qwen/Qwen3-Coder-480B-A35B-Instruct · Hugging Face

The Qwen3-Coder-480B-A35B-Instruct is a powerful AI model designed for agentic coding tasks, featuring long-context capabilities with native support for 256K tokens and extendable up to 1M tokens. It has 480B total parameters and 35B activated parameters, making it comparable to Claude Sonnet in performance. The model is optimized for repository-scale understanding and supports tool calling capabilities. It can be used with various applications such as Ollama, LMStudio, and MLX-LM. The model requires the latest version of transformers and specific sampling parameters for optimal performance.

Key Takeaways

The Qwen3-Coder-480B-A35B-Instruct model offers significant performance enhancements in agentic coding and long-context capabilities, making it suitable for complex coding tasks and repository-scale understanding.
The model's tool calling capabilities allow for flexible integration with various tools and applications, enhancing its utility in real-world coding scenarios.
To achieve optimal performance, specific sampling parameters such as temperature=0.7, top_p=0.8, and repetition_penalty=1.05 are recommended, along with an adequate output length of 65,536 tokens.

deepseek-ai/DeepSeek-Coder-V2: DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

DeepSeek-Coder-V2 is an open-source Mixture-of-Experts code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. It is further pre-trained from DeepSeek-V2 with 6 trillion additional tokens, enhancing coding and mathematical reasoning capabilities while maintaining general language performance. The model is available in 16B and 236B parameter versions, with active parameters of 2.4B and 21B respectively. DeepSeek-Coder-V2 demonstrates significant advancements in code generation, code completion, code fixing, and mathematical reasoning tasks, outperforming closed-source models like GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in various benchmarks. The model supports 338 programming languages and has a context length of 128K. It is available under the MIT License and supports commercial use.

Key Takeaways

DeepSeek-Coder-V2 achieves state-of-the-art performance in code intelligence tasks, rivaling closed-source models.
The model's Mixture-of-Experts architecture allows for efficient use of parameters, with 2.4B and 21B active parameters in the 16B and 236B versions respectively.
DeepSeek-Coder-V2 significantly expands the supported programming languages to 338 and increases the context length to 128K, making it a versatile tool for various coding tasks.

deepseek-ai/DeepSeek-Coder-V2-Instruct · Hugging Face

DeepSeek-Coder-V2 is an open-source Mixture-of-Experts code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. It is further pre-trained from DeepSeek-V2 with 6 trillion additional tokens, enhancing coding and mathematical reasoning capabilities while maintaining general language task performance. The model is available in two sizes: 16B and 236B parameters, with active parameters of 2.4B and 21B, respectively. DeepSeek-Coder-V2 supports 338 programming languages and has a context length of 128K. It outperforms closed-source models like GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks. The model can be used for code completion, insertion, and chat completion tasks, and is available on HuggingFace for download. Inference can be performed using Huggingface's Transformers or vLLM.

Key Takeaways

DeepSeek-Coder-V2 achieves state-of-the-art performance in code-specific tasks, rivaling closed-source models like GPT4-Turbo.
The model's Mixture-of-Experts architecture allows for efficient inference with active parameters of 2.4B and 21B.
DeepSeek-Coder-V2 supports a wide range of programming languages (338) and has a large context length (128K), making it a versatile tool for various coding tasks.

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

DeepSeek-Coder-V2 is an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. It is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens, enhancing coding and mathematical reasoning capabilities while maintaining general language performance. The model supports 338 programming languages and extends the context length from 16K to 128K tokens. Experimental results demonstrate superior performance compared to closed-source models like GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.

Key Takeaways

DeepSeek-Coder-V2 significantly enhances coding and mathematical reasoning capabilities while maintaining general language performance comparable to DeepSeek-V2.
The model achieves state-of-the-art performance in code generation and mathematical reasoning tasks, rivaling top closed-source models.
DeepSeek-Coder-V2 supports a significantly larger number of programming languages (338) and extends the maximum context length to 128K tokens.
Despite impressive performance, there is still a gap in instruction-following capabilities compared to state-of-the-art models like GPT-4 Turbo, highlighting the need for further improvement.