this post was submitted on 20 Nov 2023
148 points (100.0% liked)
Technology
37799 readers
76 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Considering that every new model seems to be getting worse for anything but highly sanitized corporate usage, I’m not sure that I want more AI safety …
For my usage, I use Chat GPT 3.5 turbo with the march checkpoint because I can’t get the current one to stop moralizing about bullshit instead of doing what it’s supposed to (I run two twitch bots with it). GPT4 used to be okay there, but the new preview is now starting to have the same issue with more frequent "I can’t do that Dave"-style answers, though it’s still mostly circumventable with enough prompt massaging, but it is getting harder.
In a year, I don’t see anything but self-hosted models usable for anything not corporate glitz if trajectories hold, so fuck all that AI safety.
On top of all of this, those efforts to tame and control outputs from the developer side could be abused to simply appease investors or totalitarian markets. So we might see a Disneyfication like we‘re seeing on other platforms like Youtube with their horrendous filters, spawning ridiculous terms like „unlifed“. And just imagine the level of censorship we‘d see if they ever try to get into the Chinese market because clearly, the ‚non‘ in non-profit is becoming more and more silent.
It's already easy to self host and we've optimized LLMs to run locally on not much serious hardware after we've trained them; I have GPT4ALL set up on my machine and it runs everything locally with my processor, no GPU or anything. Some of those datasets are uncensored, and I've seen what Stable Diffusion can do for image generation.
I tend to use the GPT-4 built into Edge with my O365 corporate plan, because it suits my needs better for day-to-day challenges. It can still audit code and summarize things, which is all I really need it to do here and there.
Nothing that runs on my GPU / CPU comes even close to GPT 3.5, GPT4 is not even in the same universe, and that’s with them running far more slowly.
In my tests, the self hosted options that have access to a 30xx or 40xx graphics card return results far faster than gpt4
Which model are you talking about?
Mistral for chatgpt, and i'm not saying it gives better answers, just that it's much faster than my web portal to gpt4
Oh, faster is easy. GPT 3.5 is also far faster than GPT 4. Faster at quality replies is the issue.
Pulled up a self hosted option last week to try it out. It’s not gpt4 level, but it’s damn close and I don’t worry giving access to my local documents
PrivateGPT for anyone interested
That’s an interface for models. Which model did you use?
Mistral-7B-Instruct-v0.1 is the default, i'm downloading the Llama2 model to test it with now, but many models on HuggingFace should still work
I do not believe any 7B model comes even close to 3.5 in quality. I used LLama V1 64B, and it was horrible in comparison. Are you really telling me that this tiny model gives better general answers? Or am I just misunderstanding what you are saying?
I didn’t say better, I said comparable
And faster, without handing over my data and conversations for monetization
Given the locally hosted benefits, and the ability to go to chatgpt for any answer minstrel gives that doesn’t satisfy you, makes it strong competition to chatgpt as the default tool
Hosting it yourself also means you can swap llm’s out based on context and what they’re trained on. Highly tuned models perform better than chatgpt at the things they are meant to excel in.
Prompt:
Mistral-7B-Instruct-v0.1
GPT 3.5-Turbo doesn’t support completion as it’s for chat, so I used an even worse one, text-davinci-003 which is far behind state of the art.
Mistral 7B might be okay for some very specific cases, but it’s not comparable to proper models at all.
edit: gave it a second chance, it’s a bit better (at least no complete nonsense anymore), but still terrible writing and doesn’t make much sense