this post was submitted on 29 Aug 2024
54 points (71.1% liked)

Technology

65819 readers
5214 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
54
LLMs have a strong bias against use of African American English (arstechnica.com)
submitted 6 months ago by [email protected] to c/[email protected]
33 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 25 points 6 months ago* (last edited 6 months ago) (1 children)

I'm not from USA, black, nor a native English speaker, but due to Linguistics I can give you guys some further info.

AAE (Afro-American English), in a nutshell, is a group of English varieties used by some speakers from USA and Canada. In a lot of aspects they resemble geographical varieties, like the ones you'd see in plenty other languages, but there's a key difference: it isn't used by people "of a certain region", but rather by people "of a certain race" (black people).

This is mostly but not completely spoken (cue to the term AAVE - the "V" stands for "vernacular"); it affects also the way that those people use the written language. So often you see AAE features in written English, like:

  • Negative concord - for example, "I don't want to hear nothing about this shit, man."
  • Habitual-be - for example, "They be talking about this everyday."
  • bits of non-standard spelling, due to phonetic differences
  • expressions and vocab typically used primarily by black people

What the article is saying is that LLMs are biased against those features. It's a rather strong bias, and not noticed for a geographical variety used as reference (Appalachian English). In other words: the LLM has been fed racist babble, and now it's regurgitating it.

[–] [email protected] 4 points 6 months ago (1 children)

I see, that's very different from most countries I imagine? People often speak on their own local dialect, here a northeastern would informally speak a completely different portuguese than someone from the south, doesn't matter the race.

[–] [email protected] 4 points 6 months ago

Yup, it's atypical even in the rest of the Americas. I think that the nearest equivalent in Portuguese would be the quilombola dialects, but even then it's way off - because those dialects are still geographically associated with their respective quilombos, not just with race.