add eos token and the end of assistant content

#8
by rbao2018 - opened
inclusionAI org
β€’
edited 5 days ago

Repository Enhancement and Fixes

  1. tokenizer_config.json: Added the eos_token to improve tokenization by clearly defining the end-of-sequence marker, which is beneficial for tasks such as text generation.
  2. config.json: Added support for the TokenClassificationModel to be compatible with the PPO algorithm in repositories like verl, providing configuration support for token classification tasks.
  3. modeling_bailing_moe.py
    • lm_head Normalization: Fixed the logic so that the normalization operation is carried out regardless of whether it is in training or inference mode, ensuring stable and consistent output.
    • load_banlancing_loss: Added support and implementation to optimize the distribution of computational load in multi-expert models, improving training efficiency and performance.
inclusionAI org

I appreciate your contribution; it looks good to me.

m1ngcheng changed pull request status to merged
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment