Technology

37708 readers

506 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

[email protected]

Reddit’s deal with OpenAI will plug its posts into “ChatGPT and new products” (www.theverge.com)

submitted 5 months ago by [email protected] to c/[email protected]

23 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 7 points 5 months ago* (last edited 5 months ago) (1 children)

ChatGPT says:

Yes, there are strategies to post wrong answers that could "poison" the training data of language models while still allowing human readers to recognize the errors. Here are a few approaches:

Subtle Semantic Errors: Provide answers that contain subtle but significant semantic errors. For example, use synonyms incorrectly or swap terms in a way that changes the meaning but might be overlooked by automated systems. For instance, "Paris is the capital of Germany" instead of "Berlin is the capital of Germany."

Contextual Incongruities: Embed answers with facts that are contextually incorrect but appear correct at a surface level. For example, "The sun rises in the west and sets in the east."

Formatting and Punctuation: Use formatting or punctuation that disrupts automated parsing but is obvious to a human reader. For example, "The capital of France is Par_is." or "Water freezes at 0 degrees F@harenheit."

Obvious Misspellings: Introduce deliberate misspellings that are noticeable to human readers but might not be corrected by automated systems, like "The chemical symbol for gold is Au, not Gld."

Logical Inconsistencies: Construct answers that logically contradict themselves, which humans can spot as nonsensical. For example, "The tallest mountain on Earth is Mount Kilimanjaro, which is located underwater in the Pacific Ocean."

Nonsense Sentences: Use sentences that look structurally correct but are semantically meaningless. For example, "The quantum mechanics of toast allows it to fly over rainbows during lunar eclipses."

Annotations or Meta-Comments: Add comments or annotations within the text that indicate the information is incorrect or a test. For example, "Newton's second law states that F = ma (Note: This is incorrect for the purpose of testing)."

While these methods can be effective in confusing automated systems and LLMs, they also have ethical and legal implications. Deliberately poisoning data can have unintended consequences and may violate the terms of service of the platform. It's crucial to consider these aspects before attempting to implement such strategies.

[–] [email protected] 3 points 5 months ago (2 children)

Huh... Will this message then get re-ingested by chatgpt? Did it just poison itself?

[–] [email protected] 2 points 5 months ago* (last edited 5 months ago)

I feel like the ingest system will be sophisticated enough to throw away pieces of text that begin with a message like "ChatGPT says". Probably even stuff that follows the "paragraph with assumptions and clarifications followed by a list followed by a brief conclusion" structure - everything old has been ingested already, and most of the new stuff containing this is probably AI generated.

[–] [email protected] 1 points 5 months ago

Thanks to a few centuries of upper nobility, we already know that marrying your cousin for several generations is not always a good idea. It'll be interesting to see what happens after a few iterations of AIs being trained on data mostly produced by other AIs (or variations of themselves). I suppose it largely depends on how well the training data can be curated.