sisyphean

cross-posted from: https://lemmy.fmhy.ml/post/125116

The new wave of AI systems, ChatGPT and its more powerful successors, exhibit extraordinary capabilities across a broad swath of domains. In light of this, we discuss whether artificial INTELLIGENCE has arrived.

Paper available here: https://arxiv.org/abs/2303.12712 Video recorded at MIT on March 22nd, 2023

cross-posted from: https://programming.dev/post/154634

TL;DR (by GPT-4 🤖):

• Use of AI Tools: The author routinely uses GPT-4 to answer casual and vaguely phrased questions, draft complex documents, and provide emotional support. GPT-4 can serve as a compassionate listener, an enthusiastic sounding board, a creative muse, a translator or teacher, or a devil’s advocate.

• Large Language Models (LLM) and Expertise: LLMs can often persuasively mimic correct expert responses in a given knowledge domain, such as research mathematics. However, the responses often consist of nonsense when inspected closely. The author suggests that both humans and AI need to develop skills to analyze this new type of text.

• AI in Mathematical Research: The author believes that the 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process. With the integration of tools such as formal proof verifiers, internet search, and symbolic math packages, the author expects that 2026-level AI, when used properly, will be a trustworthy co-author in mathematical research, and in many other fields as well.

• Impact on Human Institutions and Practices: The author raises questions about how existing human institutions and practices will adapt to the rise of AI. For example, how will research journals change their publishing and referencing practices when AI can generate entry-level math papers for graduate students in less than a day? How will our approach to graduate education change? Will we actively encourage and train our students to use these tools?

• Challenges and Future Expectations: The author acknowledges that we are largely unprepared to address these questions. There will be shocking demonstrations of AI-assisted achievement and courageous experiments to incorporate them into our professional structures. But there will also be embarrassing mistakes, controversies, painful disruptions, heated debates, and hasty decisions. The greatest challenge will be transitioning to a new AI-assisted world as safely, wisely, and equitably as possible.

TL;DR (by GPT-4 🤖):

• Use of AI Tools: The author routinely uses GPT-4 to answer casual and vaguely phrased questions, draft complex documents, and provide emotional support. GPT-4 can serve as a compassionate listener, an enthusiastic sounding board, a creative muse, a translator or teacher, or a devil’s advocate.

• Large Language Models (LLM) and Expertise: LLMs can often persuasively mimic correct expert responses in a given knowledge domain, such as research mathematics. However, the responses often consist of nonsense when inspected closely. The author suggests that both humans and AI need to develop skills to analyze this new type of text.

• AI in Mathematical Research: The author believes that the 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process. With the integration of tools such as formal proof verifiers, internet search, and symbolic math packages, the author expects that 2026-level AI, when used properly, will be a trustworthy co-author in mathematical research, and in many other fields as well.

• Impact on Human Institutions and Practices: The author raises questions about how existing human institutions and practices will adapt to the rise of AI. For example, how will research journals change their publishing and referencing practices when AI can generate entry-level math papers for graduate students in less than a day? How will our approach to graduate education change? Will we actively encourage and train our students to use these tools?

• Challenges and Future Expectations: The author acknowledges that we are largely unprepared to address these questions. There will be shocking demonstrations of AI-assisted achievement and courageous experiments to incorporate them into our professional structures. But there will also be embarrassing mistakes, controversies, painful disruptions, heated debates, and hasty decisions. The greatest challenge will be transitioning to a new AI-assisted world as safely, wisely, and equitably as possible.

Original tweet: https://twitter.com/emollick/status/1671528847035056128

Screenshots (from the tweet):

cross-posted from: https://programming.dev/post/133153

Quote:

In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.

Related:

• Models (you can try them online):
  • TinyStories-1M
  • TinyStories-33M
• An interview with the authors (highly recommended): The Tiny Model Revolution with Ronen Eldan and Yuanzhi Li of Microsoft Research

Old but gold.