SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning)

ModelScope Community Website
中文   ｜   English  

📖 Table of Contents

• Introduction
• Groups
• News
• Installation
• Getting Started
• Classroom
• License
• Citation

📝 Introduction

SWIFT supports training(PreTraining/Fine-tuning/RLHF), inference, evaluation and deployment of 300+ LLMs and 50+ MLLMs (multimodal large models). Developers can directly apply our framework to their own research and production environments to realize the complete workflow from model training and evaluation to application. In addition to supporting the lightweight training solutions provided by PEFT, we also provide a complete Adapters library to support the latest training techniques such as NEFTune, LoRA+, LLaMA-PRO, etc. This adapter library can be used directly in your own custom workflow without our training scripts.

To facilitate use by users unfamiliar with deep learning, we provide a Gradio web-ui for controlling training and inference, as well as accompanying deep learning courses and best practices for beginners. SWIFT web-ui is available both on Huggingface space and ModelScope studio, please feel free to try!

SWIFT has rich documentations for users, please feel free to check our documentation website:

English Documentation   ｜   中文文档  

☎ Groups

You can contact us and communicate with us by adding our group:

Discord Group	微信群

🎉 News

• 2024.08.06: Support for minicpm-v-v2_6-chat is available. You can use swift infer --model_type minicpm-v-v2_6-chat for inference experience. Best practices can be found here.
• 2024.08.06: Supports internlm2.5 series of 1.8b and 20b. Experience it using swift infer --model_type internlm2_5-1_8b-chat.
• 🔥2024.08.05: Support evaluation for multi-modal models! Same command with new datasets.
• 🔥2024.08.02: Support Fourier Ft. Use --sft_type fourierft to begin, Check parameter documentation here.
• 🔥2024.07.29: Support the use of lmdeploy for inference acceleration of LLM and VLM models. Documentation can be found here.
• 🔥2024.07.24: Support DPO/ORPO/SimPO/CPO alignment algorithm for vision MLLM, training scripts can be find in Document. support RLAIF-V dataset.
• 🔥2024.07.24: Support using Megatron for CPT and SFT on the Qwen2 series. You can refer to the Megatron training documentation.
• 🔥2024.07.24: Support for the llama3.1 series models, including 8b, 70b, and 405b. Support for openbuddy-llama3_1-8b-chat.
• 2024.07.20: Support mistral-nemo series models. Use --model_type mistral-nemo-base-2407 and --model_type mistral-nemo-instruct-2407 to begin.
• 2024.07.19: Support Q-Galore, this algorithm can reduce the training memory cost by 60% (qwen-7b-chat, full, 80G -> 35G), use swift sft --model_type xxx --use_galore true --galore_quantization true to begin!
• 2024.07.17: Support newly released InternVL2 models: model_type are internvl2-1b, internvl2-40b, internvl2-llama3-76b. For best practices, refer to here.
• 2024.07.17: Support the training and inference of NuminaMath-7B-TIR. Use with model_type numina-math-7b.
• 🔥2024.07.16: Support exporting for ollama and bitsandbytes. Use swift export --model_type xxx --to_ollama true or swift export --model_type xxx --quant_method bnb --quant_bits 4
• 2024.07.08: Support cogvlm2-video-13b-chat. You can check the best practice here.
• 2024.07.08: Support internlm-xcomposer2_5-7b-chat. You can check the best practice here.
• 🔥2024.07.06: Support for the llava-next-video series models: llava-next-video-7b-instruct, llava-next-video-7b-32k-instruct, llava-next-video-7b-dpo-instruct, llava-next-video-34b-instruct. You can refer to llava-video best practice for more information.
• 🔥2024.07.06: Support InternVL2 series: internvl2-2b, internvl2-4b, internvl2-8b, internvl2-26b.
• 2024.07.06: Support codegeex4-9b-chat.
• 2024.07.04: Support internlm2_5-7b series: internlm2_5-7b, internlm2_5-7b-chat, internlm2_5-7b-chat-1m.
• 2024.07.02: Support for using vLLM for accelerating inference and deployment of multimodal large models such as the llava series and phi3-vision models. You can refer to the Multimodal & vLLM Inference Acceleration Documentation for more information.
• 2024.07.02: Support for llava1_6-vicuna-7b-instruct, llava1_6-vicuna-13b-instruct and other llava-hf models. For best practices, refer to here.
• 🔥2024.06.29: Support eval-scope&open-compass for evaluation! Now we have supported over 50 eval datasets like BoolQ, ocnli, humaneval, math, ceval, mmlu, gsk8k, ARC_e, please check our Eval Doc to begin! Next sprint we will support Multi-modal and Agent evaluation, remember to follow us : )