README.md

---
title: MusicGenClassify
emoji: 🎼
colorFrom: red
colorTo: yellow
sdk: gradio
sdk_version: 5.25.2
app_file: app.py
pinned: false
short_description: Classify a song for use in MusicGen model training
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/6346595c9e5f0fe83fc60444/1O6woCP6rRcUAE6dZqbuB.png
---

# Audio Classifier for MusicGen Fine-Tuning

## Overview
The **Audio Classifier for MusicGen Fine-Tuning** is a web-based application designed to generate metadata for `.wav` audio files, facilitating dataset preparation for fine-tuning music generation models like MusicGen from Audiocraft. Built using **Gradio**, the app allows users to upload a single `.wav` file, input a custom label, and receive a JSON output containing detailed metadata. This tool is optimized for deployment as a **Hugging Face Space**, enabling community use for music classification tasks.

## Functionality
The application processes an uploaded `.wav` file to extract key audio features and predict its music genre, producing a JSON metadata file suitable for MusicGen training. The main features include:

- **User Inputs**:
  - **Audio File**: Upload a `.wav` file via a Gradio `Audio` component.
  - **Custom Label**: Enter a descriptive label (e.g., "A calm melody") in a textbox.
- **Audio Processing**:
  - **Duration**: Extracts the audio file's length in seconds using `librosa`.
  - **Tempo**: Estimates beats per minute (BPM) with `librosa.beat.tempo`.
  - **Sample Rate**: Retrieves the native sample rate (e.g., 44.1kHz) of the `.wav` file.
  - **Genre Prediction**: Uses a pre-trained audio classification model (`sanchit-gandhi/distilhubert-finetuned-gtzan`) to predict one of 10 genres: blues, classical, country, disco, hiphop, jazz, metal, pop, reggae, or rock.
- **Output**: A JSON object displayed in a Gradio `JSON` component, containing:
  - `filename`: Name of the uploaded file.
  - `duration`: Length in seconds (rounded to 3 decimal places).
  - `description`: The user-provided label.
  - `genre`: Predicted music genre.
  - `tempo`: Tempo in BPM (rounded to 2 decimal places).
  - `sample_rate`: Native sample rate in Hz.

## Example Output
For a file `song.wav` with the user-provided label "A calm melody," the JSON output might be:
```json
{
    "filename": "song.wav",
    "duration": 5.123,
    "description": "A calm melody",
    "genre": "jazz",
    "tempo": 90.5,
    "sample_rate": 44100
}
```

## Technical Details
- **Framework**: Built with Gradio 5.24.0 for a user-friendly web interface.
- **Libraries**:
  - `librosa`: Audio feature extraction (duration, tempo, sample rate).
  - `transformers` and `torch`: Genre prediction using a pre-trained model.
  - `numpy`: Numerical operations for probability calculations.
- **Model**: Utilizes `sanchit-gandhi/distilhubert-finetuned-gtzan`, a DistilHuBERT model fine-tuned on the GTZAN dataset for music genre classification.
- **Audio Processing**: Resamples audio to 16kHz for the model if the native sample rate differs, ensuring compatibility.
- **Error Handling**: Returns a JSON error message if processing fails (e.g., invalid file).

## Deployment
The application is designed for deployment as a **Hugging Face Space**, making it accessible to the community for MusicGen fine-tuning. Deployment steps include:
- Creating a Space with Gradio as the framework.
- Uploading the script as `app.py`.
- Including a `requirements.txt` with dependencies:

    transformers
    torch
    torchaudio
    librosa
    numpy

- Committing files to build and host the app.

## Use Case
This tool is ideal for users preparing datasets for MusicGen fine-tuning, particularly those with collections of `.wav` files from games, music, or movies. It automates the generation of metadata, reducing manual effort in labeling audio files. The JSON output is compatible with Audiocraft’s training requirements, enabling users to fine-tune music generation models efficiently.

## Benefits
- **User-Friendly**: Simple interface for uploading files and entering labels.
- **Automated Metadata**: Generates detailed metadata (genre, tempo, sample rate) to enhance descriptions.
- **Community Access**: Deployment on Hugging Face Spaces allows others to use it for similar tasks.
- **Audiocraft Compatibility**: Produces JSON files with required fields (`duration`, `description`) for MusicGen training.

## Limitations
- **Single File Processing**: Processes one file at a time, suitable for individual classification but not batch processing.
- **Genre Scope**: Limited to 10 genres from the GTZAN dataset, which may not cover all music types.
- **Label Dependency**: Relies on user input for the description, requiring manual effort for custom labels.

This application streamlines the preparation of audio datasets for MusicGen fine-tuning, offering a practical tool for researchers and enthusiasts in music generation.