Home Creators Posts Import Register Favorites Logout

Content

I already created some tutorials for how to run these two models locally with ollama and LM Studio. I'll add tutorial for oobabooga soon and

I already created some tutorials for how to run these two models locally with ollama and LM Studio. I'll add tutorial for oobabooga soon and also how to link these setups with Sugoi Toolkit

14B gguf q4_k_m:

https://sugoi-file.sfo3.cdn.digitaloceanspaces.com/sugoi14b.gguf

32B gguf q4_k_m:

https://sugoi-file.sfo3.cdn.digitaloceanspaces.com/sugoi32b.gguf

Connect with LM Studio

https://blog.sugoitoolkit.com/tutorial-sugoi-llm-14b-and-32b/

Connect with Ollama

https://blog.sugoitoolkit.com/sugoi-llm-14b-32b-with-ollama/

Connect with Oobabooga

https://blog.sugoitoolkit.com/sugoi-llm-14b-32b-with-lm-studio/

Connect with Sugoi Toolkit (after you use either of the programs above)

https://blog.sugoitoolkit.com/connecting-sugoi-14b-32b-to-sugoi-toolkit/

If you found anything interesting, feel free to share on our Discord server so everyone can learn from it

Comments

Amazing Flapples

Been waiting for this for a while, really surprised to see they are already in gguf format so i don't have to convert them for llama.cpp. Since I only have 12GB of vram I'll have to figure out how to quantize a bit... Are the models already uncensored? Or should I do an abliteration pass? In any case, thanks a lot!

Rkraiem100 (edited)

Comment edits

2026-11-01 14:00 I'm doing some tests. I have a 3090 and 128gb of RAM so I'm able to run 32b easily. The results are.... descent. I'm underwhelmed sadly. Granted the fact you made this yourself is quite impressive. But frankly the translation quality just doesnt compare to public LLM models
2025-06-16 02:41

MingShiba

When this model is public, maybe more people can share optimal settings to boost accuracy. Anyway, appreciate your feedback. Training this model was very difficult and expensive, I can't help but think maybe the age of garage model achieving SOTA result is long gone

Amazing Flapples

alright, thanks again mate, I've already implemented one of them in a private manga translation server, and written up some stuff to work it with LunaTranslator

Amazing Flapples

I've personally found at least 14b in my case needs some fiddling with repeat_penalty top_p and top_k have you tried that out yet?

Chakiik

Trying them out now, thanks!