|
--- |
|
title: Video Human Fall Detector |
|
emoji: 🐠 |
|
colorFrom: purple |
|
colorTo: red |
|
sdk: gradio |
|
sdk_version: 5.25.0 |
|
app_file: app.py |
|
pinned: false |
|
license: apache-2.0 |
|
short_description: Fall Detection Demo using LightCLIP |
|
--- |
|
|
|
# Fall Detection Demo using LightCLIP on Hugging Face Spaces |
|
|
|
This project demonstrates a lightweight, transformer-based approach to detect human falls in video clips using a vision–language model (VLM). The demo is designed for complex scenes including multiple persons, obstacles, and varying lighting conditions. It employs a sliding-window technique to check multiple frames for robust detection and aggregates predictions over time to reduce false alarms. |
|
|
|
## Overview |
|
|
|
The demo uses a pre-trained LightCLIP (or CLIP) model to compute image–text similarity scores between video frames and natural language prompts. Two prompts are used: |
|
- **Fall Prompt:** "A person falling on the ground." |
|
- **Non-Fall Prompt:** "A person standing or walking." |
|
|
|
For each window of frames extracted from the video, the model computes similarity scores for each frame. The scores are aggregated over a sliding window, and if the average score for the "fall" prompt exceeds a defined threshold, a fall event is registered along with an approximate timestamp. |
|
|
|
## Project Files |
|
|
|
- **app.py:** The main application file containing the Gradio demo. |
|
- **requirements.txt:** Lists all the required Python libraries. |
|
- **README.md:** This file. |
|
|
|
## How to Run |
|
|
|
1. **Clone or download the repository** into your Hugging Face Spaces. |
|
2. Ensure the project is set to use the **GPU plan** in Spaces. |
|
3. Spaces will automatically install the required libraries from `requirements.txt`. |
|
4. Launch the demo by running `app.py` (Gradio will start the web interface). |
|
|
|
## Code Overview |
|
|
|
- **Frame Extraction:** The video is processed using OpenCV to extract frames (resized to 224×224). |
|
- **LightCLIP Inference:** The demo uses the Hugging Face Transformers library to load a CLIP model (acting as LightCLIP). It computes image embeddings for each frame and compares them to text embeddings of the fall and non-fall descriptions. |
|
- **Temporal Aggregation:** A sliding window (e.g. 16 frames with a stride of 8) is used to calculate average "fall" scores. Windows exceeding a threshold (e.g. 0.8) are flagged as fall events. |
|
- **User Interface:** A simple Gradio UI allows users to upload a video clip and displays the detection result along with a representative frame and list of detected fall times. |
|
|
|
## Customization |
|
|
|
- **Model:** Replace `"openai/clip-vit-base-patch32"` in `app.py` with your own LightCLIP model checkpoint if available. |
|
- **Threshold & Window Size:** Adjust parameters such as the detection threshold, window size, and stride for better results on your dataset. |
|
- **Deployment:** This demo is configured to run on a GPU-backed Hugging Face Space for real-time inference. |
|
|
|
Enjoy experimenting with fall detection! |
|
|
