Neura Market
MNN-LLM logo

MNN-LLM

Free

- A Device-Inference framework, including LLM Inference on device(Mobile Phone/PC/IOT)

FreeFree tier
Inputs: text, image, audioOutputs: text, image
Type
Open Source
Company
Alibaba
Visit Website GitHub (38 stars)

About MNN-LLM

MNN is a highly efficient and lightweight deep learning framework developed by Alibaba. It supports both inference and training of deep learning models, with industry-leading performance on-device (mobile phones, PCs, IoT devices). MNN has been battle-tested in over 30 Alibaba apps (e.g., Taobao, Tmall, Youku, DingTalk, Xianyu) covering more than 70 scenarios including live broadcast, short video capture, search recommendation, product searching by image, interactive marketing, and security risk control. The project includes MNN-LLM, a large language model runtime supporting models like Qwen, Baichuan, Zhipu, LLAMA, DeepSeek, and others, and MNN-Diffusion, a stable diffusion runtime for on-device image generation. It also offers multimodal capabilities including text-to-text, image-to-text, audio-to-text, and text-to-image generation. MNN is open source and widely used in production edge AI deployments.

Key Features

Highly efficient and lightweight deep learning framework
Supports both inference and training of deep learning models
On-device LLM runtime (MNN-LLM) supporting Qwen, Baichuan, Zhipu, LLAMA, DeepSeek, and more
Stable diffusion runtime (MNN-Diffusion) for on-device image generation
Multimodal support: text-to-text, image-to-text, audio-to-text, text-to-image
Integrated into over 30 Alibaba apps covering 70+ scenarios
Open source (Apache 2.0 license)
Active development with frequent model and feature updates
Supports multiple platforms: Android, iOS, PC, IoT

Pros & Cons

Pros
  • Blazing-fast inference performance on resource-constrained devices
  • Lightweight footprint suitable for mobile and edge deployment
  • Broad support for popular LLMs and diffusion models
  • Multimodal capabilities (text, image, audio) in a single framework
  • Production-proven at massive scale within Alibaba's ecosystem
  • Open source with permissive license and active community
  • Regular updates adding new models and features
Cons
  • Primarily designed for on-device deployment, less suitable for large-scale server clusters
  • Documentation and community resources may be more focused on Chinese-speaking developers
  • Some advanced features require understanding of MNN's internal architecture and custom operators
  • Model conversion and optimization can be complex for non-standard architectures

Best For

On-device LLM inference (chatbots, assistants)Real-time image and video analysis on mobile devicesProduct search by image in e-commerce appsShort video capture and content recommendationInteractive marketing and equity distributionSecurity risk control and fraud detectionOffline text-to-image generation (Stable Diffusion)IoT and embedded device AI inference

FAQ

What models does MNN-LLM support?
MNN-LLM supports a wide range of popular large language models including Qwen (Qwen3, Qwen2.5 Omni), Baichuan, Zhipu, LLAMA, DeepSeek (R1 1.5b), and others.
Is MNN free to use?
Yes, MNN is open source under the Apache 2.0 license and is completely free to use, modify, and distribute.
Can MNN run on iOS devices?
Yes, MNN supports iOS. The project includes an iOS multimodal LLM app released in 2025 that supports text, image, and audio inputs.
What is MNN-Diffusion?
MNN-Diffusion is a stable diffusion model runtime based on the MNN engine, designed to deploy stable diffusion models locally on mobile phones, PCs, and IoT devices for text-to-image generation.
Does MNN support multimodal inputs?
Yes, MNN supports multimodal inputs including text, image, and audio. The MNN-LLM Android app offers text-to-text, image-to-text, audio-to-text, and text-to-image generation.