Text Generation
Transformers
PyTorch
chatts
feature-extraction
conversational
custom_code

Fix dimension calculation issue in mask list during forward due to precision error

#12
by Wannabtl - opened
Files changed (1) hide show
  1. modeling_qwen2.py +1 -1
modeling_qwen2.py CHANGED
@@ -130,7 +130,7 @@ class TimeSeriesEmbedding(nn.Module):
130
  batch_size = x.size(0)
131
  x = x.reshape(batch_size, -1, self.num_features)
132
 
133
- mask = x[:, :, -1]
134
  valid_lengths = mask.sum(dim=1).long() # Shape: (batch_size)
135
 
136
  patch_cnt = (valid_lengths + self.patch_size - 1) // self.patch_size # 向上取整
 
130
  batch_size = x.size(0)
131
  x = x.reshape(batch_size, -1, self.num_features)
132
 
133
+ mask = x[:, :, -1].long()
134
  valid_lengths = mask.sum(dim=1).long() # Shape: (batch_size)
135
 
136
  patch_cnt = (valid_lengths + self.patch_size - 1) // self.patch_size # 向上取整