136 words
1 minute
Trying out the facebook/musicgen-small sound generation model
Anubhav Gain
2024-10-26
machinelearning
/
/
machinelearning
/
python
/
web

Trying out the facebook/musicgen-small sound generation model#

Facebook’s musicgen is a model that generates snippets of audio from a text description - it’s effectively a Stable Diffusion for music.

It turns out it’s pretty easy to run it using Python, thanks to the Hugging Face transformers library.

Here’s the code that worked for me. First, install the dependencies:

pip install scipy transformers

The following will download the small model - around 2GB - and store it in ~/.cache/huggingface/hub/models--facebook--musicgen-small the first time you run it.

from transformers import AutoProcessor, MusicgenForConditionalGeneration
import scipy


processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")


def save(prompt, filename, num_tokens=1503):
    inputs = processor(
        text=[prompt],
        padding=True,
        return_tensors="pt",
    )
    audio_values = model.generate(**inputs, max_new_tokens=num_tokens)
    sampling_rate = model.config.audio_encoder.sampling_rate
    scipy.io.wavfile.write(filename, rate=sampling_rate, data=audio_values[0, 0].numpy())

Then you can use that save() function like this to generate and save an audio sample:

save("trumpet mariachi frenetic excitement", "trumpet_mariachi.wav")

Here’s the audio that generated:

https://static.simonwillison.net/static/2023/trumpet_mariachi.wav

Trying out the facebook/musicgen-small sound generation model
https://mranv.pages.dev/posts/trying-out-the-facebookmusicgen-small-sound-generation-model/
Author
Anubhav Gain
Published at
2024-10-26
License
CC BY-NC-SA 4.0
© 2026 Anubhav Gain. All Rights Reserved. RSS Sitemap Built with Astro View Source