The model mistralai/Mistral-7B-Instruct-v0.2 is too large to be loaded automatically (14GB > 10GB).
Hi Team,
I am using this model for long, but suddenly I got below error message, could anyone please confirm how can we fix this issue ?
Error Message:
💾 Model Size Error (Environment Limitation)
The model mistralai/Mistral-7B-Instruct-v0.2 is too large to be loaded automatically (14GB > 10GB).
This tells us your machine or environment can't handle the model size. Specifically:
Hugging Face's default pipeline auto-loading can't handle models >10GB.
The model Mistral-7B-Instruct v0.2 is about 14GB in memory.
This part is machine/environment-specific — you might be using a local machine or hosted runtime that doesn't have enough memory.
I have been getting the same errors... This was working fine until last week, but I think there is a quota now. I used model v0.3 instead, which works fine, but it is so easy to exceed the free monthly allowance.