hok

joined 1 year ago
sorted by: new top controversial old
Fish Speech 1.5, an open source voice cloning TTS that's actually good in c/[email protected]
[–] [email protected] 3 points 1 week ago

You are right. Their description of "SOTA Open Source TTS" caused me to assume it was open source, but it's clear that

This codebase and all models are released under CC-BY-NC-SA-4.0 License.

So, it's "source available" and not released under a permissive licence.

Llama 3.3 70b - End of open-weight pretrained models from Meta or just a better Llama 3.1 405b finetune? in c/localllama
[–] [email protected] 2 points 1 week ago

Thank you so much, that exactly answers my question with the official response (that guy works at Meta) that confirms it's the same base model!

I was concerned primarily because in the release notes it strangely didn't mention it anywhere, and I thought it would have been important enough to mention.

17
Llama 3.3 70b - End of open-weight pretrained models from Meta or just a better Llama 3.1 405b finetune? (lemmy.dbzer0.com)
 

People are talking about the new Llama 3.3 70b release, which has generally better performance than Llama 3.1 (approaching 3.1's 405b performance): https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_3

However, something to note:

Llama 3.3 70B is provided only as an instruction-tuned model; a pretrained version is not available.

Is this the end of open-weight pretrained models from Meta, or is Llama 3.3 70b instruct just a better-instruction-tuned version of a 3.1 pretrained model?

Comparing the model cards: 3.1: https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md 3.3: https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md

The same knowledge cutoff, same amount of training data, and same training time give me hope that it's just a better finetune of maybe Llama 3.1 405b.

Fish Speech 1.5, an open source voice cloning TTS that's actually good in c/[email protected]
[–] [email protected] 4 points 1 week ago* (last edited 1 week ago)

I followed their instructions here: https://speech.fish.audio/

I am using the locally-run API server to do inference: https://speech.fish.audio/inference/#http-api-inference

I don't know about other ways. To be clear, this is not (necessarily) an LLM, it's just for speech synthesis, so you don't run it on ollama. That said I think it does technically use Llama under the hood since there are two models, one for encoding text and the other for decoding to audio. Honestly the paper is terrible but it explains the architecture somewhat: https://arxiv.org/pdf/2411.01156

34
Fish Speech 1.5, an open source voice cloning TTS that's actually good (github.com)
 

I've been waiting for an open source TTS model that was actually good enough to capture some of the subtleties of language and synthesize them in a natural-sounding way that makes sense. I think I finally found one that fits the requirements.

Model: https://huggingface.co/fishaudio/fish-speech-1.5

It uses an encoder rather than relying on phonemes, and generations sometimes vary because of that, but the amount of errors I've gotten are minimal, and the variations in the generation are all surprisingly natural in slightly different ways, which is very exciting.

Give it a spin if you are also looking for a TTS model that sounds good. It uses voice cloning, so find a good 10-20 second reference clip to have the generations use the same voice.

Discussing jury nullification is against the ToS of lemmy.world and will get you banned, according to worldnews mod in c/[email protected]
[–] [email protected] 15 points 1 week ago

On Lemmy, everything is a bit leftist at the moment.

8
What models can we use for img2img today? (lemmy.dbzer0.com)
 

I'd like to fine tune a model that does img2img with a text prompt to guide the output. I think img2img-turbo might be the closest to what I'm after, though by default it uses a fixed prompt which can be made variable with some tweaking of the training code.

At the moment I only have access to 24GB VRAM which limits my options. What I'm after is training a model to make specific text-based modifications to images, and I have plenty of before to after images plus the modification text prompts to train on. Worst case, I can try to see if reducing the image size during training makes it possible with my setup.

Are there any other options available today?

Is there anything that makes training a translation task easy? in c/[email protected]
[–] [email protected] 1 points 1 year ago

Thanks for the tips. After doing a bunch of searching, I found that what I needed was BPE, or byte-pair encoding. This allows the token set to contain sub-word sequences, which lets the tokenizer represent a unique constant like 0x0373 as ['__sow', '0x', '03', '73', '__eow'].

Is there anything that makes training a translation task easy? in c/[email protected]
[–] [email protected] 0 points 1 year ago (2 children)

Thanks, the quickstart guide was straightforward to follow. Do you have any suggestions on how to do word splitting with code, if any? For example, on a test run, I found that the model was not able to synthesize unique constants correctly even though this test run consisted only of obvious "a to b" relationships.

0
Is there anything that makes training a translation task easy? (lemmy.dbzer0.com)
 

I have thousands of side-by-side translations for two computer languages (lower level to higher level), and I would like to train a model that is able to do translations on new data with higher accuracy.

Got any suggestions on what to do? I don't think I want to fine tune a ChatGPT-style model since I think the task is more structured than that. Also, I consider myself technically competent but probably would fail at designing my own model and pipeline.