README.md

metadata

title: Visual Question Answering (VQA) System
emoji: 🏞️
colorFrom: blue
colorTo: purple
sdk: streamlit
sdk_version: 1.43.1
app_file: app.py
pinned: false

Visual Question Answering (VQA) System

A multi-modal AI application that allows users to upload images and ask questions about them. This project uses pre-trained models from Hugging Face to analyze images and answer natural language questions.

Features

• Upload images in common formats (jpg, png, etc.)
• Ask questions about image content in natural language
• Get AI-generated answers based on image content
• User-friendly Streamlit interface
• Support for various types of questions (objects, attributes, counting, etc.)

Technical Stack

• Python: Main programming language
• PyTorch & Transformers: Deep learning frameworks for running the models
• Streamlit: Interactive web application framework
• HuggingFace Models: Pre-trained visual question answering models
• PIL: Image processing

Setup Instructions

1. Clone this repository:

   git clone 
   cd visual-question-answering
   

2. Create a virtual environment (recommended):

   python -m venv venv
   # On Windows
   venv\Scripts\activate
   # On macOS/Linux
   source venv/bin/activate
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Run the application:

   python app.py
   

   Or directly with Streamlit:

   streamlit run app.py
   

5. Open a web browser and go to http://localhost:8501

Usage

1. Upload an image using the file upload area
2. Type your question about the image in the text field
3. Select a model from the sidebar (BLIP or ViLT)
4. Click "Get Answer" to get an AI-generated response
5. View the answer displayed on the right side of the screen

Models Used

This application uses the following pre-trained models from Hugging Face:

• BLIP: For general visual question answering with free-form answers
• ViLT: For detailed understanding of image content and yes/no questions

Project Structure

• models/: Contains model handling code
• utils/: Utility functions for image processing and more
• static/: Static files including uploaded images
• app.py: Script to run the application

Acknowledgments

• Hugging Face for their excellent pre-trained models
• The open-source community for various libraries used in this project