llm-toys

小型（7B及以下）、适用于生产环境的LLM微调模型，用于一系列有用的任务。

支持的任务：改写、改变文章的语气、从对话生成摘要和主题， 检索增强的问答（WIP）。

我们在量化的3B和7B模型上进行LoRAs微调。3B模型针对特定任务进行微调，而7B模型则针对所有任务进行微调。

目标是能够在非常普通的消费级硬件上微调和使用所有这些模型。

安装

pip install llm-toys

可能需要CUDA支持的GPU才能工作

如果遇到bitsandbytes出现"安装的bitandbytes版本没有GPU支持"的问题，那么请查看 https://github.com/TimDettmers/bitsandbytes/issues/112

或尝试

cp <path_to_your_venv>/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so <path_to_your_venv>/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so

请注意，我们使用的是源目录中的transformers和peft包，而不是已安装的包。4bit的bitsandbytes量化仅在transformers和peft的主分支上工作。一旦transformers版本4.31.0和peft版本0.4.0发布到pypi，我们将使用发布的版本。

可用模型

使用方法

任务专用的3B模型

改写

from llm_toys.tasks import Paraphraser

paraphraser = Paraphraser()
paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?")
# "Could you kindly assist me in canceling my previous order?"

改变语气

paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?", tone="casual")
# "Hey, could you help me cancel my order?"

paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?", tone="professional")
# "I would appreciate guidance on canceling my previous order."

paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?", tone="witty")
# "Hey, I need your help with my last order. Can you wave your magic wand and make it disappear?"

对话摘要和主题生成

from llm_toys.tasks import SummaryAndTopicGenerator

summary_topic_generator = SummaryAndTopicGenerator()
summary_topic_generator.generate_summary_and_topic(
    """
    #Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!
    #Person2#: What's got you so hyped?
    #Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.
    #Person2#: Which movie is it?
    #Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.
    #Person2#: Sounds amazing! I'm in for the premiere.
    #Person1#: Great! We're in for a visual masterpiece and a heartfelt story.
    #Person2#: Can't wait to be transported to their world.
    #Person1#: It'll be an unforgettable experience, for sure!
    """.strip()
)
# {"summary": "#Person1# 对最新的吉卜力工作室电影首映感到兴奋。
#              #Person1# 认为动画、故事和感人故事将是难忘的。
#              #Person2# 也对首映感到兴奋。",
#  "topic": "吉卜力工作室电影"}

通用的7B模型

from llm_toys.tasks import GeneralTaskAssitant
from llm_toys.config import TaskType

gta = GeneralTaskAssitant()
gta.complete(TaskType.PARAPHRASE_TONE, "Hey, can yuo hepl me cancel my last order?")
# "Could you assist me in canceling my previous order?"

gta.complete(TaskType.PARAPHRASE_TONE, "Hey, can yuo hepl me cancel my last order?", tone="casual")
# "Hey, can you help me cancel my last order?"

gta.complete(TaskType.PARAPHRASE_TONE, "Hey, can yuo hepl me cancel my last order?", tone="professional")
# "I would appreciate if you could assist me in canceling my previous order."

gta.complete(TaskType.PARAPHRASE_TONE, "Hey, can yuo hepl me cancel my last order?", tone="witty")
# "Oops! Looks like I got a little carried away with my shopping spree. Can you help me cancel my last order?"

chat = """
#Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!
#Person2#: What's got you so hyped?
#Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.
#Person2#: Which movie is it?
#Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.
#Person2#: Sounds amazing! I'm in for the premiere.
#Person1#: Great! We're in for a visual masterpiece and a heartfelt story.
#Person2#: Can't wait to be transported to their world.
#Person1#: It'll be an unforgettable experience, for sure!
""".strip()
gta.complete(TaskType.DIALOGUE_SUMMARY_TOPIC, chat)
# {"summary": "#Person1# 告诉 #Person2# 即将上映的吉卜力工作室电影。
#              #Person1# 认为它很神奇，#Person2# 也很期待观看。",
#  "topic": "电影首映"}

训练

数据

• 改写和改变语气的数据: 包含段落及其改写版本，以及不同语气的段落，如随意、专业和幽默。用于模型改写和改变段落语气。数据是使用gpt-35-turbo生成的。一小部分训练段落还来自quora问题和squad_2数据集。

• 对话摘要和主题生成的数据: 包含对话及其摘要和主题。训练数据来自Dialogsum数据集训练拆分的约1k条记录。还包含约20个开发拆分样本。在抽样时，优先考虑较长的摘要和主题的数据点。请注意，最终训练数据中有一些（约30个）主题经过手动编辑，因为原始标注的主题只是一个词，不够描述。

示例训练脚本

查看所有选项

python llm_toys/train.py --help

训练一个改写和改变语气的模型

python llm_toys/train.py \
    --task_type paraphrase_tone \
    --model_name meta-llama/Llama-2-7b \
    --max_length 128 \
    --batch_size 8 \
    --gradient_accumulation_steps 1 \
    --learning_rate 1e-4 \
    --num_train_epochs 3 \
    --eval_ratio 0.05

评估

改写和改变语气

WIP

对话摘要和主题生成

在来自Dialogsum test拆分的500条记录上进行评估。

# llm-toys/RedPajama-INCITE-Base-3B-v1-dialogue-summary-topic
{"rouge1": 0.453, "rouge2": 0.197, "rougeL": 0.365, "topic_similarity": 0.888}

# llm-toys/falcon-7b-paraphrase-tone-dialogue-summary-topic
{'rouge1': 0.448, 'rouge2': 0.195, 'rougeL': 0.359, 'topic_similarity': 0.886}

路线图

• 添加测试。
• 能够在不重新初始化主干模型和分词器的情况下切换LoRAs（用于任务模型）。
• 检索增强的问答。
• 探索3B模型在更多任务中的泛化性。
• 探索更小的模型。
• 针对没有测试/评估数据集随手可得的任务的评估策略。
• 数据收集策略和微调一个用于OpenAI类似"函数调用"的模型