VisCPM-10B

VisCPM is an open-source family of large multimodal models developed by OpenBMB, supporting bilingual (Chinese and English) multimodal conversation via the VisCPM-Chat model and text-to-image generation via the VisCPM-Paint model. The VisCPM-Chat model is built on the CPM-Bee large language model with 10 billion parameters, integrating a Q-Former visual encoder to process image inputs and generate conversational responses. It achieves state-of-the-art performance among Chinese open-source multimodal models, leveraging a two-stage training process: pre-training on approximately 100 million high-quality English image-text pairs to align visual and language representations, followed by instruction fine-tuning on LLaVA-150K data (with translated Chinese data) to align with user intent. The model demonstrates strong bilingual capabilities, generalizing from English-only pre-training to effectively handle Chinese inputs and outputs. The VisCPM family also includes the VisCPM-Paint model for text-to-image generation, using a Diffusion-UNet decoder trained on LAION 2B data. The models require a GPU with at least 40GB of memory and are available for use through Hugging Face Transformers.