民间版中文羊驼模型 v4.2

[ymcui]

本版本以功能性更新为主，包括新增8K上下文支持、支持Gradio Demo流式输出、支持仿OpenAI API形式调用等。

🔬 新增8K+上下文支持

新增8K+上下文支持方法，无需对模型权重本身做出修改。

• transformers：提出自适应RoPE，动态适配4K~8K+上下文，已集成在 gradio_demo.py, inference_hf.py等（Extend context size without fine-tuning #705）
• llama.cpp：可支持8K+上下文，相关修改步骤详见讨论区（llama.cpp下8K context支持（实验性） #696）

🚀 支持Gradio Demo流式输出（#630）

• Gradio Demo现已支持流式输出形式，参考gradio_demo.py. Contribued by @sunyuhan19981208
• 修复流式输出时速度过慢的问题（Fix unexpected slow down in gradio web demo #707）. Contributed by @GoGoJoestar

🤖 支持仿OpenAI API形式调用（#530）

• 使用fastapi实现的仿OpenAI API风格的服务器Demo，使用方法参考Wiki. Contribued by @sunyuhan19981208
• 修复一处system message相关错误（fix system message attribute error #684）. Contribued by @bigbaldy1128
• 增加do_sample参数（Add do_sample argument for API demo #699）Contribued by @sunyuhan19981208

其他更新、修复、新闻

• 修复了训练脚本中training steps的问题（fix training steps #669）
• 修复了build_dataset.py和run_clm_sft_with_peft.py中的一处命名错误，优化log输出（Fix a typo in the function name #691、fix log message to reduce confusion #692）
• 修复了低资源模型合并脚本中vocab_size更新问题（fix vocab_size after low-mem-merge #703）
• llama.cpp已支持仿OpenAI API部署，还可以部署一个简单的web demo，感兴趣的可访问llama.cpp#server

For English release note, please refer to Discussion.

---

This discussion was created from the release 民间版中文羊驼模型 v4.2.

[ymcui]

English Release Note v4.2

This version primarily features functional updates, including the addition of 8K context support, Gradio Demo stream output, and support for imitation of the OpenAI API call, etc.

🔬 Added support for 8K+ context

New method to support 8K+ context has been added, without the need to modify the model weights themselves.

• transformers: Proposes adaptive RoPE, dynamically adapting 4K~8K+ context, now integrated in gradio_demo.py, inference_hf.py and others (Extend context size without fine-tuning #705)
• llama.cpp: Can support 8K+ context, for detailed modification steps, please see the discussion area (llama.cpp下8K context支持（实验性） #696)

🚀 Support for Gradio Demo stream output (#630)

• Gradio Demo now supports stream output, refer to gradio_demo.py. Contributed by @sunyuhan19981208
• Fixed the problem of slow speed during stream output (Fix unexpected slow down in gradio web demo #707). Contributed by @GoGoJoestar

🤖 Support for imitation of OpenAI API calls (#530)

• A server Demo in the style of the OpenAI API, implemented using fastapi, for usage refer to the Wiki. Contributed by @sunyuhan19981208
• Fixed an error related to a system message (fix system message attribute error #684). Contributed by @bigbaldy1128
• Add do_sample argument (Add do_sample argument for API demo #699). Contributed by @sunyuhan19981208

Other updates, fixes, news

• Fixed the problem with training steps in the training script (fix training steps #669)
• Fixed a naming error in build_dataset.py and run_clm_sft_with_peft.py, optimized log output (Fix a typo in the function name #691, fix log message to reduce confusion #692)
• Fixed the issue with vocab_size update in the low-resource model merging script (fix vocab_size after low-mem-merge #703)
• llama.cpp now supports deployment with imitation OpenAI API, and can also deploy a simple web demo, interested parties can visit llama.cpp#server

[AceyKubbo]

大佬厉害,现在正苦于context的长度问题呢,你们就出了对应的了,llama.cpp最近好像也支持RoPE了,我这就把模型都重新合并一次试试看😊.