Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm
Make sure to use quantized models for the best performace, q4k_M being the standard.
QKSMS has less active development but I don’t think that’s an issue as it works well as is. I haven’t dug too deeply into the more advanced stuff but I’ve yet to have any issues with it.
I’ve done it before using their update tool on freedos. Not sure if all versions support this but it was pretty quick and painless.
Smaller communities aren’t necessarily a bad thing. Compared to reddit I rarely feel like I’m commenting into the void.
That isn’t neccesarily true, though for now there’s no way to tell since they’ve yet to release their code. If the timeline is anything like their last paper it will be out around a month after publication, which will be Nov 20th.
There have been similar papers for confusing image classification models, not sure how successful they’ve been IRL.
I’ve used the tplink ones that they’re using and they’ve been pretty solid. I can’t say how they’d fare in a 24/7 setup though since they’re not really intended for that.
I sort of understand rounding outside edges for aesthetics since there’s nothing lost and it might be easier as a target for resizing, but inside corners are just stupid. You’re arbitrarily cutting corners out of content for no good reason.
There should be no performance difference. The only difference should be in loading screens and possibly pop-in from streamed assets.
The issue is the marketing. If they only marketed language models for the things they are able to be trusted with, summarization, cleaning text, writing assistance, entertainment, etc. there wouldn’t be nearly as much debate.
The creators of the image generation models have done a much better job of this, partially because the limitations can be seen visually, rather than requiring a fact check on every generation. They also aren’t claiming that they’re going to revolutionize all of scociety, which helps.
LLMs only predict the next token. Sometimes those predictions are correct, sometimes they’re incorrect. Larger models trained on a greater number of examples make better predictions, but they are always just predictions. This is why incorrect responses often sound plausable even if logically they don’t make sense.
Fixing hallucinations is more about decreasing inaccuracies rather than fixing an actual problem with the model itself.
You can install any extension you want on the Dev version and some forks like mull by setting a custom extension collection. It’s a bit of a pain but it works.
Duckduckgo doesn’t have anywhere near the capacity to collect data that google does, and their ads are keyword based, rather than being influenced by other data. Their search engine is really the only thing I’d recommend using however since their add-on and browser don’t offer anything that others don’t.
Infinity for Lemmy has the option to open links with an in-app browser or with an external one.
Cohere’s command-r models are trained for exactly this type of task. The real struggle is finding a way to feed relevant sources into the model. There are plenty of projects that have attempted it but few can do more than pulling the first few search results.