Technology

34877 readers

45 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.

Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.

Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 5 years ago

MODERATORS

[email protected]

Learn in simple terms how ChatGPT works and why people are still needed (robbie.antenesse.net)

submitted 11 months ago by [email protected] to c/[email protected]

11 comments fedilink hide all child comments

It's more important than ever to understand what ChatGPT and other AI tools like it are actually doing when they talk to us and write for us.

I worked with some Large Language Models and GPTs and dug into what they're doing, and I wrote this article. I try to explain in the simplest terms possible what modern AIs actually are and how exactly they construct their content so we can move past the fear and confusion about what AI is capable of and start using it for what it's actually good for.

Please arm yourself with knowledge and understanding, and share this with someone who worries about AI taking over their job (or even the whole world)!

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago)

This article is full of errors!

At its core, an LLM is a big (“large”) list of phrases and sentences

Definitely not! An LLM is the combination of an architecture and its model parameters. It's just a bunch of numbers, no list of sentences, no database. (Seems like the author confused the word "LLM" with the dataset of the LLM???)

an LLM is a storage space (“database”) containing as many sample documents as possible

Nope. This applies to the dataset, not the model. I guess you can argue that memorization happens sometimes, so it might have some features of a database. But it isn't one.

Additional data (like the topic, mood, tone, source, or any number of other ways to categorize the documents) can be provided

LLMs are trained in an unsupervised fashion. Just sequences of tokens, no labels.

Typically, an LLM will cover a single context, e.g. only social media

I'm not aware of any LLM that does this. What's the "context" of GPT-4?

software developers have gone to great lengths to collect an unfathomable number of sample texts and meticulously categorize those samples in as many ways as possible

The closest real thing is the RLHF process that is used to fine tune an existing LLM for a specific application (like ChatGPT). The dataset for the LLM is not annotated or categorized in any way.

a GPT uses the words and proximity data stored in LLMs

This is confusing. "GPT" is the architecture of the LLM.

it is impossible for it to create something never seen before

This isn't accurate, depending on the temperature setting, an LLM can output literally any word at any time with a non-zero probability. It can absolutely produce things it hasn't seen.

Also I think it's too simple to just assert that LLMs are not intelligent. It mostly depends on your definition of intelligence and there are lots of philosophical discussions to be had (see also the AI effect).