bytedance-research/ChatTS-14B · Sample code fails

9 days ago

•

Sample code generate fails:

File ~/.cache/huggingface/modules/transformers_modules/bytedance-research/ChatTS-14B/b54f0d8e86d15bc7168c602c24203c480e8aee02/modeling_qwen2.py:1509, in Qwen2TSForCausalLM.prepare_inputs_for_generation(self, input_ids, past_key_values, attention_mask, inputs_embeds, timeseries, **kwargs)
   1507     cache_length = past_key_values.get_seq_length()
   1508     past_length = past_key_values.seen_tokens
-> 1509     max_cache_length = past_key_values.get_max_length()
   1510 else:
   1511     cache_length = past_length = past_key_values[0][0].shape[2]

AttributeError: 'DynamicCache' object has no attribute 'get_max_length'

I'm on latest : transformers 4.51.3

csabakecskemeti

9 days ago

•

edited 9 days ago

In fact if I've downgraded transformers to the version available when the modeling_qwen2.py file has been uploaded
transformers==4.47.1 the code has worked!

Also suggesting to update the sample code to use the hf_model_id instead of the checkpoint folder, and use device_map = "auto"

hf_model = "bytedance-research/ChatTS-14B"
# Load the model, tokenizer and processor
model = AutoModelForCausalLM.from_pretrained(hf_model, trust_remote_code=True, device_map="auto", torch_dtype='float16')
tokenizer = AutoTokenizer.from_pretrained(hf_model, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(hf_model, trust_remote_code=True, tokenizer=tokenizer)

xiezhe24

bytedance-research org 6 days ago

Thank you for pointing out this issue! We've already updated modeling_qwen2.py. You can test it in the latest version of transformers.

xiezhe24 changed discussion status to closed 6 days ago