metadata

title: MusicGenClassify
emoji: 🎼
colorFrom: red
colorTo: yellow
sdk: gradio
sdk_version: 5.25.2
app_file: app.py
pinned: false
short_description: Classify a song for use in MusicGen model training
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/6346595c9e5f0fe83fc60444/1O6woCP6rRcUAE6dZqbuB.png

Audio Classifier for MusicGen Fine-Tuning

Overview

The Audio Classifier for MusicGen Fine-Tuning is a web-based application designed to generate metadata for .wav audio files, facilitating dataset preparation for fine-tuning music generation models like MusicGen from Audiocraft. Built using Gradio, the app allows users to upload a single .wav file, input a custom label, and receive a JSON output containing detailed metadata. This tool is optimized for deployment as a Hugging Face Space, enabling community use for music classification tasks.

Functionality

The application processes an uploaded .wav file to extract key audio features and predict its music genre, producing a JSON metadata file suitable for MusicGen training. The main features include:

User Inputs:
- Audio File: Upload a .wav file via a Gradio Audio component.
- Custom Label: Enter a descriptive label (e.g., "A calm melody") in a textbox.
Audio Processing:
- Duration: Extracts the audio file's length in seconds using librosa.
- Tempo: Estimates beats per minute (BPM) with librosa.beat.tempo.
- Sample Rate: Retrieves the native sample rate (e.g., 44.1kHz) of the .wav file.
- Genre Prediction: Uses a pre-trained audio classification model (sanchit-gandhi/distilhubert-finetuned-gtzan) to predict one of 10 genres: blues, classical, country, disco, hiphop, jazz, metal, pop, reggae, or rock.
Output: A JSON object displayed in a Gradio JSON component, containing:
- filename: Name of the uploaded file.
- duration: Length in seconds (rounded to 3 decimal places).
- description: The user-provided label.
- genre: Predicted music genre.
- tempo: Tempo in BPM (rounded to 2 decimal places).
- sample_rate: Native sample rate in Hz.

Example Output

For a file song.wav with the user-provided label "A calm melody," the JSON output might be:

{
    "filename": "song.wav",
    "duration": 5.123,
    "description": "A calm melody",
    "genre": "jazz",
    "tempo": 90.5,
    "sample_rate": 44100
}

Technical Details

Framework: Built with Gradio 5.24.0 for a user-friendly web interface.
Libraries:
- librosa: Audio feature extraction (duration, tempo, sample rate).
- transformers and torch: Genre prediction using a pre-trained model.
- numpy: Numerical operations for probability calculations.
Model: Utilizes sanchit-gandhi/distilhubert-finetuned-gtzan, a DistilHuBERT model fine-tuned on the GTZAN dataset for music genre classification.
Audio Processing: Resamples audio to 16kHz for the model if the native sample rate differs, ensuring compatibility.
Error Handling: Returns a JSON error message if processing fails (e.g., invalid file).

Deployment

The application is designed for deployment as a Hugging Face Space, making it accessible to the community for MusicGen fine-tuning. Deployment steps include:

Creating a Space with Gradio as the framework.
Uploading the script as app.py.
Including a requirements.txt with dependencies:

transformers torch torchaudio librosa numpy
Committing files to build and host the app.

Use Case

This tool is ideal for users preparing datasets for MusicGen fine-tuning, particularly those with collections of .wav files from games, music, or movies. It automates the generation of metadata, reducing manual effort in labeling audio files. The JSON output is compatible with Audiocraft’s training requirements, enabling users to fine-tune music generation models efficiently.

Benefits

User-Friendly: Simple interface for uploading files and entering labels.
Automated Metadata: Generates detailed metadata (genre, tempo, sample rate) to enhance descriptions.
Community Access: Deployment on Hugging Face Spaces allows others to use it for similar tasks.
Audiocraft Compatibility: Produces JSON files with required fields (duration, description) for MusicGen training.

Limitations

Single File Processing: Processes one file at a time, suitable for individual classification but not batch processing.
Genre Scope: Limited to 10 genres from the GTZAN dataset, which may not cover all music types.
Label Dependency: Relies on user input for the description, requiring manual effort for custom labels.

This application streamlines the preparation of audio datasets for MusicGen fine-tuning, offering a practical tool for researchers and enthusiasts in music generation.