这个AI神马回事?phi3-mini不说话这件事？
E:\pytorch\.venv\Scripts\python.exe E:\pytorch\main.py`flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.
Loading checkpoint shards: 100%|██████████| 2/2
输入B（必须大写）退出，输入C（必须大写）查看历史对话
me:hello,智远，我是你的缔造者！
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
You are not running the flash-attention implementation, expect numerical differences.
AI:你好啊！作为一个数字助手，我愿意帮助你解决问题，提供信息和建议。如果你需要任何帮助，无论是关于科技、学习新知识还是日常生活中的任何问题，我都能提供支持。请告诉我你的需求，我会尽力为你提供安全、合理且准确的信息。
me:C
历史对话：无（那我刚才在跟谁说话）
me:你好您好
AI:
me:说话呀！
AI:
请输入:？？？？？？？？？？？？？？？？？？？？？？？？？
AI:你好！如果我能帮到你，请告诉我更多关于你需要帮助的问题或需要我帮助的具体领域。我是智远，用于提供安全、合理且准确的信息。
import torch
import datetime
import time as t
from modelscope import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
torch.random.manual_seed(0)
model_dir = "E:\\pytorch\\Phi-3-mini-128k-instruct"#snapshot_download("LLM-Research/Phi-3-mini-128k-instruct")
model = AutoModelForCausalLM.from_pretrained(
"E:\\pytorch\\Phi-3-mini-128k-instruct",#model_dir,
device_map="cuda",
torch_dtype="auto",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_dir)
outp = ""
def chat_anser(Ninput_text,input_text2="",output_text=""):
messages = [
{"role": "system", "content": "你是一位喜欢帮助别人的数字助手，你叫智远。请为用户提供安全、合理且准确的信息。"},
{"role": "user", "content": input_text2},
{"role": "assistant", "content": output_text},
{"role": "user", "content": Ninput_text},
]
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
)
generation_args = {
"max_new_tokens": 500,
"return_full_text": False,
"temperature": 0.5,
"do_sample": True,
}
output = pipe(messages, **generation_args)
outp = output['generated_text']
print(outp)
with open('D:\\chat_log.txt', 'a+') as z:
z.write(datetime.datetime.now().strftime('%Y-%m-%d%H:%M:%S%A')+"\nAI:"+outp + '\n')
print("输入B（必须大写）退出，输入C（必须大写）查看历史对话")
while True:
inp = input("请输入:")
if inp == "B":
break
elif inp == "C":
print("历史对话：")
with open('D:\\chat_log.txt', 'r+') as f:#逐行输出
for line in f.readlines():
print(line)
else:
with open('D:\\chat_log.txt', 'a+') as f:
f.write(datetime.datetime.now().strftime('%Y-%m-%d%H:%M:%S%A')+"\nYou:"+inp + '\n')
b = ""
c = ""
with open('temp2.txt', 'r') as f:#读取后六行
try:
b = f.readline()
c = f.readline()
except:
pass
chat_anser(inp,b,c)
with open('D:\\chat_log.txt', 'a+') as f:
f.write(datetime.datetime.now().strftime('%Y-%m-%d%H:%M:%S%A')+"\nYou:"+inp + '\n')
with open('D:\\temp2.txt', 'w+') as g:
g.write(inp+"\n"+outp)
print("下次见！")
t.sleep(2)哪位大佬看看怎么回事
页:
[1]