LocalLLaMA

2258 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

Are there any good open source text-to-music models, preferably with lyrical abilities? (lemmy.dbzer0.com)

submitted 3 months ago by [email protected] to c/localllama

4 comments fedilink hide all child comments

Only recently did I discover the text-to-music AI companies (udio.com, suno.com) and I was surprised about how good the results are. Both are under lawsuit from RIAA.

I am curious if there are any local ones I can experiment with or train myself. I know there is facebook/musicgen-large on HuggingFace. That model is over 1 year old and there might be others by now. Also, based on the card I get the feeling that model is not going to be good at doing specific song lyrics (maybe the lyrics just were absent from the training data?). I am most interested in trying my hand at writing songs and fine-tuning a model on specific types of music to get the sounds I am looking for.

top 4 comments

sorted by: hot top controversial new old

[–] [email protected] 5 points 3 months ago (1 children)

The only text-to-audio model I can think of at the moment is Stable Audio Open, which AFAIK is rather underwhelming for your use-case, if it can even handle stuff more complex than basic sounds - and no lyrics.
It is even under the "new" membership licensing of SAI.

I remember reading about a more recent one, but I currently can't find it, and I don't think that that one too could handle lyrics.

I suppose the Music industry is a lot harder to fight, so not a lot of people want to entangle themself with it.

[–] [email protected] 2 points 3 months ago

Interestingly, Jukebox from OpenAI was trained on what appears to be copyrighted music and involved styles and renditions that explicitly referenced specific artists. It's now four years old though. The demo songs don't seem to be available anymore on Soundcloud.

There is MusicLM from Google (2023) - no lyrics. Also, AudioCraft from Meta (2023) - also no lyrics as far as I can tell.

[–] [email protected] 2 points 3 months ago

ChatMusician isn't exactly new and the underlying dataset isn't particularly diverse, but it's one of the few models made specifically for classical music.

Are there any others, by the way?

[–] [email protected] 0 points 3 months ago

Maybe it would be possible to use a regular text-to-voice model and then use something similar to autotune