Update README.md

README.md

@@ -79,15 +79,14 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+The base model was trained on the Glaive code assistant dataset from Glaive AI. This dataset contains ~140,000 code problems and solutions. The problems are formatted in a natural language process, such as you would expect a human user to query. Approximately 60% of the data uses the Python language. The dataset only contains a test split, and this was split into a training and validation set. For reproducibility, the data was shuffled after loading in using data.sample() with only the hyperparameters frac=1 and random_state=42 used. I split the data approximately 90-10, with 122,000 rows in the train set and the remainder in the validation set.
 
 ### Training Procedure
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+A Program of Thought (PoT) prompting approach was used to modify the training data. PoT specifically focuses on generating structured, step-by-step reasoning in a more formalized and executable format than chain of thought. With PoT, the model generates a sequence of logical steps that resemble pseudo-code, which can then be translated into actual code. PoT should allow structured reasoning for code generation, should be able to debug broken code, and should provide a step-by-step breakdown of the plan ([Chen et al., 2023](https://arxiv.org/abs/2211.12588)).
 
-#### Preprocessing [optional]
-
-[More Information Needed]
+PoT was implemented by creating a new column in the dataset that initiated a “plan” for the model to follow. The plan was a 4-step process: analyze the question from the train column, think step by step and plan an efficient solution before writing the code, consider any necessary programming constructs or tools, and explain the approach with well-organized and documented code. The answer to the question from the dataset was appended after the plan so there was a single text block for training. Lastly, before training started, the pandas dataframes (train and validation) were converted into Hugging Face Dataset objects, and the tokenizer was applied to the newly created column for each row.
 
 
 #### Training Hyperparameters