Spaces:

open-nlp
/

Chris-lab

Sleeping

kz209 commited on 11 days ago

Commit

331a464

1 Parent(s): 65ef8da

update dtyoe

Files changed (1) hide show

utils/model.py CHANGED Viewed

@@ -31,7 +31,7 @@ class Model(torch.nn.Module):
             # 使用vLLM加载模型
             self.llm = LLM(
                 model=model_name,
-                dtype="bfloat16",
                 tokenizer=model_name,
                 trust_remote_code=True
             )

             # 使用vLLM加载模型
             self.llm = LLM(
                 model=model_name,
+                dtype="half",
                 tokenizer=model_name,
                 trust_remote_code=True
             )