SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 25 days agoDo you host your own AI?message-squaremessage-square204linkfedilinkarrow-up1185arrow-down142file-text
arrow-up1143arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 25 days agomessage-square204linkfedilinkfile-text
minus-squaretristynalxander@mander.xyzdeleted by creatorlinkfedilinkEnglisharrow-up4·edit-223 hours agodeleted by creator
minus-squareBlackLaZoR@lemmy.worldlinkfedilinkEnglisharrow-up2·24 days agoI use LMStudio, because it has quality of life improvements like nice GUI and huggingface search engine. Also they have Vulkan backend that at least on 7900XTX is ~10% faster than rocm (on LLama 3 8b Q4_0 it gets 115Tokens/s vs 105 on rocm
deleted by creator
I use LMStudio, because it has quality of life improvements like nice GUI and huggingface search engine. Also they have Vulkan backend that at least on 7900XTX is ~10% faster than rocm (on LLama 3 8b Q4_0 it gets 115Tokens/s vs 105 on rocm