地址

GitHub地址： https://github.com/LianjiaTech/BELLE

论文地址：

《Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation》

《A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model》

时间

时间：2023年4月

作者

地址

论文地址： https://arxiv.org/abs/2304.07854

时间

时间：2023年4月16日

Recently, significant public efforts have been directed towards developing low-cost models with capabilities akin to ChatGPT, thereby fostering the growth of open-source conversational models. However, there remains a scarcity of comprehensive and in-depth evaluations of these models' performance. In this study, we examine the influence of training data factors, including quantity, quality, and linguistic distribution, on model performance. Our analysis is grounded in several publicly accessible, high-quality instruction datasets, as well as our own Chinese multi-turn conversations. We assess various models using a evaluation set of 1,000 samples, encompassing nine real-world scenarios. Our goal is to supplement manual evaluations with quantitative analyses, offering valuable insights for the continued advancement of open-source chat models. Furthermore, to enhance the performance and training and inference efficiency of models in the Chinese domain, we extend the vocabulary of LLaMA - the model with the closest open-source performance to proprietary language models like GPT-3 - and conduct secondary pre-training on 3.4B Chinese words. We make our model, data, as well as code publicly available.

最近，公众已经付出了大量努力， 开发 具有ChatGPT类似能力的 低成本模型 ，从而促进了开源对话模型的发展。然而，这些模型性能的 全面深入评估 仍然 很少 。在本研究中，我们考察了 训练数据因素 （包括数量、质量和语言分布） 对模型性能的影响 。我们的分析基于几个公开可获取的高质量指令数据集，以及我们自己的中文多轮对话数据。我们使用一个包含9个真实场景的评估集对各种模型进行评估，该集合包含1000个样本。我们的目标是 通过定量分析来补充手动评估 ，为开源对话模型的持续进展 提供有价值的见解 。此外，为了增强 模型在中文领域的性能、训练和推理效率 ，我们 扩展了LLaMA的词汇量 （LLaMA是与GPT-3等专有语言模型最接近的开源模型），并在 34亿中文词汇上 进行了辅助预训练。我们将我们的模型、数据和代码公开提供。

In conclusion, this study addresses the grow-ing need for comprehensive evaluations of open-source conversational models by investigating the influence of various training data factors, such as quantity, quality, and linguistic distribution. By utilizing publicly accessible high-quality in-struction datasets and Chinese multi-turn conver-sations, we assess different models on a evaluation set of 1,000 samples across nine real-world scenar-ios. We also conclude several challenges of build-ing a comprehensive evaluation dataset and argue the necessity of prioritizing the development of such evaluation set. Moreover, this study extends the vocabulary of LLaMA and conducts secondary pre-training with 3.4B Chinese words to enhance its performance and efficiency in the Chinese do-main. This results in a 60% reduction in train-ing and inference time without sacrificing perfor-mance. By making the model, data, and code pub-licly available, this research contributes to the on-going efforts of the open-source community to de-velop more accessible and efficient conversational models especially for Chinese.

总之，本研究 通过研究训练数据的数量、质量和语言分布等各种因素对开源对话模型进行全面评估 ，以满足日益增长的这方面需求。我们利用公开 可获取的高质量指令数据集 和 中文多轮对话数据 ，在包含9个真实场景的1000个样本的评估集上评估了不同的模型。我们还总结了构建全面评估数据集的几个挑战，并提出了优先发展这种评估集的必要性。此外，本研究扩展了LLaMA的词汇量，并 在34亿中文词汇上进行了辅助预训练 ，以提高其在中文领域的性能和效率。这使得 训练和推理时间减少了60% ，而不会牺牲性能。通过公开提供模型、数据和代码，这项研究为开源社区不断努力开发更易于访问和高效的对话模型，特别是针对中文，做出了贡献。

地址

论文地址： https://arxiv.org/abs/2304.08109

时间

时间：2023年4月17日

Recently, the instruction-tuning of large language models is a crucial area of research in the field of natural language processing. Due to resource and cost limitations, several researchers have employed parameter-efficient tuning techniques, such as LoRA, for instruction tuning, and have obtained encouraging results In comparison to full-parameter fine-tuning, LoRA-based tuning demonstrates salient benefits in terms of training costs. In this study, we undertook experimental comparisons between full-parameter fine-tuning and LoRA-based tuning methods, utilizing LLaMA as the base model. The experimental results show that the selection of the foundational model, training dataset scale, learnable parameter quantity, and model training cost are all important factors. We hope that the experimental conclusions of this paper can provide inspiration for training large language models, especially in the field of Chinese, and help researchers find a better trade-off strategy between training cost and model performance. To facilitate the reproduction of the paper's results, the dataset, model and code will be released.

最近，在自然语言处理领域，对大型语言模型进行 指令微调是一项重要的研究 领域。由于资源和成本的限制，一些研究人员采用了参数高效的微调技术，如 LoRA ，进行指令微调，并取得了令人鼓舞的结果。与全参数微调相比，LoRA-based微调在训练成本方面表现出明显的优势。在本研究中，我们 使用LLaMA作为基础模型 ，对 全参数微调和LoRA-based微调 方法进行了 实验比较 。实验结果显示， 基础模型的选择 、训练 数据集规模 、 可学习参数数量 和模型 训练成本 都是重要因素。我们希望本文的实验结论能够为训练大型语言模型，特别是 在中文领域 ，提供启示，并帮助研究人员找到更好的 训练成本 和 模型性能之间的权衡 策略。为了便于复现本文的结果，数据集、模型和代码将被发布。

In this article, we conducted an experimental com-parison between full-parameter fine-tuning and LoRA-based tuning methods using LLaMA as the base model. We also explored the impact of differ-ent amounts of training data and model parameters on the effectiveness of LoRA-based tuning. From the experimental results comparison, some inter-esting ideas can observed:

(1)、The choice of the base model has a signif-icant impact on the effectiveness of LoRA-based tuning. Comparing LLaMA-7B+LoRA(0.6M) and LLaMA-7B+FT(0.6M), as well as LLaMA- 7B+LoRA(2M) and LLaMA-7B+FT(2M), it is evident that LoRA-based tuning on a base model that has not undergone instruction tun-ing has limited effectiveness and is far less ef-fective than full-parameter fine-tuning (averag-ing 10 points lower). However, by compar-ing LLaMA-7B+FT(2M)+FT(math_0.25M) and LLaMA-7B+FT(2M)+LoRA(math_0.25M), it can be seen that LoRA-based tuning on a model that has undergone instruction tuning can achieve com-parable results to fine-tuning. This indicates that the choice of the base model is crucial to the ef-fectiveness of the LoRA-based tuning method.

(2)、Increasing the amount of training data can continuously improve the model’s effectiveness. Comparing LLaMA-7B+LoRA(0.6M), LLaMA- 7B+LoRA(2M), and LLaMA-7B+LoRA(4M) shows that as the amount of training data in-creases, the model’s effectiveness improves (an average of approximately 2 points improvement for every doubling of data).

(3)、LoRA-based tuning benefits from the number of model parameters. Comparing LLaMA- 7B+LoRA(4M) and LLaMA-13B+LoRA(2M) shows that the number of model parameters has a greater impact on the effectiveness of LoRA-based tuning than the amount of training data.

在本文中，我们使用LLaMA作为基础模型，进行了 全参数微调 和 LoRA-based微调方 法的实验比较。我们还探讨了不同数量的训练数据和模型参数对LoRA-based微调效果的影响。通过实验结果比较，我们得出了一些有趣的观点：

(1)、基础模型的选择 对LoRA-based微调的效果有显著影响。比较LLaMA-7B+LoRA(0.6M)和LLaMA-7B+FT(0.6M)，以及LLaMA-7B+LoRA(2M)和LLaMA-7B+FT(2M)，可以明显看出，在 未经过指令微调的基础模型上进行LoRA-based微调的效果有限 ， 远不如全参数微调 （平均低10个点）。然而，通过比较LLaMA-7B+FT(2M)+FT(math_0.25M)和LLaMA-7B+FT(2M)+LoRA(math_0.25M)，可以看出，在 经过 指令微调 的模型上进行 LoRA-based微调 可以达到与 全参数微调相当 的结果。这表明基础模型的选择对LoRA-based微调方法的有效性至关重要。

(2)、增加训练数据量可以不断提高模型的效果 。比较LLaMA-7B+LoRA(0.6M)，LLaMA-7B+LoRA(2M)和LLaMA-7B+LoRA(4M)可以发现，随着训练数据量的增加，模型的效果也在改善（ 每倍数据增加平均约2个点 ）。

(3)、LoRA-based微调受益于模型参数的数量 。比较LLaMA-7B+LoRA(4M)和LLaMA-13B+LoRA(2M)可以看出， 模型参数的数量 对LoRA-based微调的 有效性影响更大 ，而不是训练数据量。