Actually Useful AI

TL;DR

See comments.

Notes (by GPT-4 🤖):

A Day Without a Copilot: Reflections on Copilot-Driven Development

Introduction

• The author, Gavin Ray, reflects on the impact of Github Copilot on his software development process.
• He shares his experience of a day without Copilot, which was a rare occurrence since the Technical Preview.
• He discusses how Copilot has profoundly changed his development process and experience.

From Monologue to Dialogue

• Ray appreciates the solitude of coding but also values the collaboration and learning from others.
• Github Copilot has been a game-changer for him, allowing him to have a dialogue with his code and the collective wisdom of the world without expending energy.
• Coding has become a collaborative dialogue between Ray and Copilot, shaping the output together.

Fresh Perspectives

• Copilot provides fresh perspectives, suggesting API designs or implementation details that Ray would not have considered.
• Not all suggestions are good, but even the bad ones help him think about the problem differently.
• Ray generates several sets of Copilot suggestions based on the specs before designing or implementing an API, picking the best candidates and tweaking them to create the final implementation.

Copilot-Driven Development

• Ray describes a phenomenon he calls "Copilot-Driven Development", a process that optimizes for Copilot's suggestions/accuracy.
• This process includes choosing popular programming languages and well-known libraries, using explicit names and types, writing types and interfaces with specifications and documentation first, implementing tests alongside each implementation, and keeping as much code in a single file as possible during early development.

Outcomes of Copilot-Driven Development

• Ray uses Copilot's suggestions to guide his development process, helping him think about problems differently and make better decisions.
• This process allows him to see the problem from different perspectives, gain insights, learn from the community, be more efficient, and be more confident in his decisions.

Evolving Roles in Software Development

• Tools like Github Copilot and ChatGPT highlight a shift in the role of the software developer, allowing developers to leverage the collective wisdom of the community to improve their work.
• This shift is important in modern software development, where the complexity and scale of projects can make it difficult for a single individual to have all the necessary knowledge and expertise.
• The use of tools like Github Copilot does not diminish the role of the individual but enables them to focus more on the creative and strategic aspects of development.
• These tools are redefining the role of the software developer, allowing them to be more effective and efficient in their work, and focus on the most interesting and challenging aspects of the development process.

👋 Hello everyone, welcome to our Weekly Discussion thread!

This week, we’re interested in your thoughts on AI safety: Is it an issue that you believe deserves significant attention, or is it just fearmongering motivated by financial interests?

I've created a poll to gauge your thoughts on these concerns. Please take a moment to select the AI safety issues you believe are most crucial:

VOTE HERE: 🗳️ https://strawpoll.com/e6Z287ApqnN

Here is a detailed explanation of the options:

1. Misalignment between AI and human values: If an AI system's goals aren't perfectly aligned with human values, it could lead to unintended and potentially catastrophic consequences.

2. Unintended Side-Effects: AI systems, especially those optimized to achieve a specific goal, might engage in harmful behavior that was not intended, often referred to as "instrumental convergence".

3. Manipulation and Deception: AI could be used for manipulating information, deepfakes, or influencing behavior without consent, leading to erosion of trust and reality.

4. AI Bias: AI models may perpetuate or amplify existing biases present in the data they're trained on, leading to unfair outcomes in various sectors like hiring, law enforcement, and lending.

5. Security Concerns: As AI systems become more integrated into critical infrastructure, the potential for these systems to be exploited or misused increases.

6. Economic and Social Impact: Automation powered by AI could lead to significant job displacement and increase inequality, causing major socioeconomic shifts.

7. Lack of Transparency: AI systems, especially deep learning models, are often criticized as "black boxes," where it's difficult to understand the decision-making process.

8. Autonomous Weapons: The misuse of AI in warfare could lead to lethal autonomous weapons, potentially causing harm on a massive scale.

9. Monopoly and Power Concentration: Advanced AI capabilities could lead to an unequal distribution of power and resources if controlled by a select few entities.

10. Dependence on AI: Over-reliance on AI systems could potentially make us vulnerable, especially if these systems fail or are compromised.

Please share your opinion here in the comments!

@AutoTLDR

Nuggt is a language to program directed agents. You can program agents for data analysis, data visualisation, building applications, building games and much more!

We are looking for feedback and collaborators. Find Nuggt on Github: https://github.com/Nuggt-dev/Nuggt

Like many, I've been flabbergasted by the huge advances we've seen in recent years with artificial intelligence. I've spent hours just playing with ChatGPT and Stable Diffusion and am consistently impressed.

I'm also aware of issues surrounding this breed of technology, like with intellectual property and black box biases. On top of that, I'm trying to avoid hype and grift. Where are you seeing AI excel, or where does it have a potential to excel? Are there places where it is being shoehorned into that should be avoided?

Announcement

The bot I announced in this thread is now ready for a limited beta release.

You can see an example summary it wrote here.

How to Use AutoTLDR

• Just mention it ("@" + "AutoTLDR") in a comment or post, and it will generate a summary for you.
• If mentioned in a comment, it will try to summarize the parent comment, but if there is no parent comment, it will summarize the post itself.
• If the parent comment contains a link, or if the post is a link post, it will summarize the content at that link.
• If there is no link, it will summarize the text of the comment or post itself.
• 🔒 If you include the #nobot hashtag in your profile, it will not summarize anything posted by you.

Beta limitations

• The bot only works in the [email protected] community.
• It is limited to 100 summaries per day.

How to try it

• If you want to test the bot, write a long comment, or include a link in a comment in this thread, and then, in a reply comment, mention the bot.
• Feel free to test it and try to break it in this thread. Please report any weird behavior you encounter in a PM to me (NOT the bot).
• You can also use it for its designated purpose anywhere in the AUAI community.

@[email protected]

TL;DR (by GPT-4 🤖)

The article discusses the evolution of AI beyond text-based chatbots, highlighting the emergence of multimodal AI, which can process different kinds of input, including images. This development allows AI to "see" and understand images, significantly enhancing its capabilities and enabling it to interact with the world in new ways. The article also mentions the integration of OpenAI's Whisper, a highly effective voice-to-text system, into the ChatGPT app, which changes how AI can be used, such as serving as an intelligent assistant. The author emphasizes that AI's growing capabilities, including internet connectivity, code execution, and the ability to watch and listen, have profound implications, necessitating a thoughtful consideration of both the benefits and concerns.

Notes (by GPT-4 🤖)

AI Evolution Beyond Text

• AI has evolved beyond being just chatbots. New modes of AI usage have emerged, such as the write-it-for-me buttons in Google Docs, which seamlessly integrate AI into work processes.
• These changes have significant implications for work and the meaning of writing.

Multimodal AI

• The most advanced AI, GPT-4, is a multimodal AI, which means it can process different kinds of input, including images.
• Multimodal AI allows the AI to "see" images and "understand" what it is seeing. This capability significantly enhances what AI can do, despite occasional errors and hallucinations.

AI Interaction with the World

• Because AI can now "see," it can interact with the world in an entirely new way, with significant implications.
• For instance, AI can now build and refine prototypes using vision, a substantial increase in capabilities.

AI Voice Recognition

• OpenAI's Whisper is a highly effective voice-to-text system that is now part of the ChatGPT app on mobile phones.
• This integration changes how AI can be used, such as serving as an intelligent assistant that can understand intent rather than just dictation.

AI in Education

• Voice recognition can be useful in education, providing real-time presentation feedback.
• For example, GPT-4 can act as a real-time virtual VC, providing feedback on startup pitches.

AI's Growing Capabilities

• AI's knowledge and capabilities have expanded beyond just text and include internet connectivity, code execution, and now, the ability to watch and listen.
• These advancements mean that jobs requiring visual or audio interactions are no longer insulated from AI.
• The implications of these capabilities are profound, and there is a need to start considering both the benefits and concerns today.

This is an excellent overview of tokenization with many interesting examples. I also like Simon's small CLI tools; you can read about them at the end of the post.

As usual, I've asked GPT-4 to write a TL;DR and detailed notes for it.

Notice that it couldn't print the "davidjl" glitch token, and (probably because of its presence), the notes are also incomplete. At first I thought it was because the text of the article was longer than the context window, but the TL;DR contains details the notes don't so that probably wasn't the case.

I've still decided to copy the notes here because they are generally useful and also demonstrate this weird behavior.

TL;DR (by GPT-4 🤖)

The article discusses the concept of tokenization in large language models like GPT-3/4, LLaMA, and PaLM. These models convert text into tokens (integers) and predict the next tokens. The author explains how English words are usually assigned a single token, while non-English languages often have less efficient tokenization. The article also explores "glitch tokens," which exhibit unusual behavior, and the necessity of counting tokens to ensure OpenAI's models' token limit is not exceeded. The author introduces a Python library called tiktoken and a command-line tool called ttok for this purpose. Understanding tokens can help make sense of how GPT tools generate text.

Notes (by GPT-4 🤖)

Understanding GPT Tokenizers

• Large language models like GPT-3/4, LLaMA, and PaLM operate in terms of tokens, which are integers representing text. They convert text into tokens and predict the next tokens.
• OpenAI provides a Tokenizer tool for exploring how tokens work. The author has also built a tool as an Observable notebook.
• The notebook can convert text to tokens, tokens to text, and run searches against the full token table.

Tokenization Examples

• English words are usually assigned a single token. For example, "The" is token 464, " dog" is token 3290, and " eats" is token 25365.
• Capitalization and leading spaces are important in tokenization. For instance, "The" with a capital T is token 464, but " the" with a leading space and a lowercase t is token 262.
• Languages other than English often have less efficient tokenization. For example, the Spanish sentence "El perro come las manzanas" is encoded into seven tokens, while the English equivalent "The dog eats the apples" is encoded into five tokens.
• Some languages may have single characters that encode to multiple tokens, such as certain Japanese characters.

Glitch Tokens and Token Counting

• There are "glitch tokens" that exhibit unusual behavior. For example, token 23282—"djl"—is one such glitch token. It's speculated that this token refers to a Reddit user who posted incremented numbers hundreds of thousands of times, and this username ended up getting its own token in the training data.
• OpenAI's models have a token limit, and it's sometimes necessary to count the number of tokens in a string before passing it to the API to ensure the limit is not exceeded. OpenAI provides a Python library called tiktoken for this purpose.
• The author also introduces a command-line tool called ttok, which can count tokens in text and truncate text down to a specified number of tokens.

Token Generation

• Understanding tokens can help make sense of how GPT tools generate text. For example, names not in the dictionary, like "Pelly", take multiple tokens, but "Captain Gulliver" outputs the token "Captain" as a single chunk.

The official documentation by OpenAI has a lot of hidden gems like this guide.

Much better than any "prompt infographic" by Twitter threadbois.

Table of Contents

1. Write clear instructions

• Include details in your query to get more relevant answers
• Ask the model to adopt a persona
• Use delimiters to clearly indicate distinct parts of the input
• Specify the steps required to complete a task
• Provide examples
• Specify the desired length of the output

2. Provide reference text

• Instruct the model to answer using a reference text
• Instruct the model to answer with citations from a reference text

3. Split complex tasks into simpler subtasks

• Use intent classification to identify the most relevant instructions for a user query
• For dialogue applications that require very long conversations, summarize or filter previous dialogue
• Summarize long documents piecewise and construct a full summary recursively

4. Give GPTs time to "think"

• Instruct the model to work out its own solution before rushing to a conclusion
• Use inner monologue or a sequence of queries to hide the model's reasoning process
• Ask the model if it missed anything on previous passes

5. Use external tools

• Use embeddings-based search to implement efficient knowledge retrieval
• Use code execution to perform more accurate calculations or call external APIs
• Give the model access to specific functions

6. Test changes systematically

• Evaluate model outputs with reference to gold-standard answers

TL;DR (by GPT-4 🤖)

The article discusses the concept of building autonomous agents powered by Large Language Models (LLMs), such as AutoGPT, GPT-Engineer, and BabAGI. These agents use LLMs as their core controller, with key components including planning, memory, and tool use. Planning involves breaking down tasks into manageable subgoals and self-reflecting on past actions to improve future steps. Memory refers to the agent's ability to utilize short-term memory for in-context learning and long-term memory for retaining and recalling information. Tool use allows the agent to call external APIs for additional information. The article also discusses various techniques and frameworks for task decomposition and self-reflection, different types of memory, and the use of external tools to extend the agent's capabilities. It concludes with case studies of LLM-empowered agents for scientific discovery.

Notes (by GPT-4 🤖)

LLM Powered Autonomous Agents

• The article discusses the concept of building agents with Large Language Models (LLMs) as their core controller, with examples such as AutoGPT, GPT-Engineer, and BabAGI. LLMs have the potential to be powerful general problem solvers.

Agent System Overview

• The LLM functions as the agent’s brain in an LLM-powered autonomous agent system, complemented by several key components:
  • Planning: The agent breaks down large tasks into smaller subgoals and can self-reflect on past actions to improve future steps.
  • Memory: The agent utilizes short-term memory for in-context learning and long-term memory to retain and recall information over extended periods.
  • Tool use: The agent can call external APIs for extra information that is missing from the model weights.

Component One: Planning

• Task Decomposition: Techniques like Chain of Thought (CoT) and Tree of Thoughts are used to break down complex tasks into simpler steps.
• Self-Reflection: Frameworks like ReAct and Reflexion allow the agent to refine past action decisions and correct previous mistakes. Chain of Hindsight (CoH) and Algorithm Distillation (AD) are methods that encourage the model to improve on its own outputs.

Component Two: Memory

• The article discusses the different types of memory in human brains and how they can be mapped to the functions of an LLM. It also discusses Maximum Inner Product Search (MIPS) for fast retrieval from the external memory.

Tool Use

• The agent can use external tools to extend its capabilities. Examples include MRKL, TALM, Toolformer, ChatGPT Plugins, OpenAI API function calling, and HuggingGPT.
• API-Bank is a benchmark for evaluating the performance of tool-augmented LLMs.

Case Studies

• The article presents case studies of LLM-empowered agents for scientific discovery, such as ChemCrow and a system developed by Boiko et al. (2023). These agents can handle autonomous design, planning, and performance of complex scientific experiments.

👋 Hello everyone, welcome to our very first Weekly Discussion thread!

This week, we're focusing on the applications of AI that you've found particularly noteworthy.

We're not just looking for headline-making AI applications. We're interested in the tools that have made a real difference in your day-to-day routine, or a unique AI feature that you've found useful. Have you discovered a new way to utilize ChatGPT? Perhaps Stable Diffusion or Midjourney has helped you generate an image that you're proud of?

Let's share our knowledge and learn more about the various applications of AI. Looking forward to your contributions.

TL;DR (by GPT-4 🤖):

Prompt Engineering, or In-Context Prompting, is a method used to guide Language Models (LLMs) towards desired outcomes without changing the model weights. The article discusses various techniques such as basic prompting, instruction prompting, self-consistency sampling, Chain-of-Thought (CoT) prompting, automatic prompt design, augmented language models, retrieval, programming language, and external APIs. The effectiveness of these techniques can vary significantly among models, necessitating extensive experimentation and heuristic approaches. The article emphasizes the importance of selecting diverse and relevant examples, giving precise instructions, and using external tools to enhance the model's reasoning skills and knowledge base.

Notes (by GPT-4 🤖):

Prompt Engineering: An Overview

• Introduction
  • Prompt Engineering, also known as In-Context Prompting, is a method to guide the behavior of Language Models (LLMs) towards desired outcomes without updating the model weights.
  • The effectiveness of prompt engineering methods can vary significantly among models, necessitating extensive experimentation and heuristic approaches.
  • This article focuses on prompt engineering for autoregressive language models, excluding Cloze tests, image generation, or multimodality models.
• Basic Prompting
  • Zero-shot and few-shot learning are the two most basic approaches for prompting the model.
  • Zero-shot learning involves feeding the task text to the model and asking for results.
  • Few-shot learning presents a set of high-quality demonstrations, each consisting of both input and desired output, on the target task.
• Tips for Example Selection and Ordering
  • Examples should be chosen that are semantically similar to the test example.
  • The selection of examples should be diverse, relevant to the test sample, and in random order to avoid biases.
• Instruction Prompting
  • Instruction prompting involves giving the model direct instructions, which can be more token-efficient than few-shot learning.
  • Models like InstructGPT are fine-tuned with high-quality tuples of (task instruction, input, ground truth output) to better understand user intention and follow instructions.
• Self-Consistency Sampling
  • Self-consistency sampling involves sampling multiple outputs and selecting the best one out of these candidates.
  • The criteria for selecting the best candidate can vary from task to task.
• Chain-of-Thought (CoT) Prompting
  • CoT prompting generates a sequence of short sentences to describe reasoning logics step by step, leading to the final answer.
  • CoT prompting can be either few-shot or zero-shot.
• Automatic Prompt Design
  • Automatic Prompt Design involves treating prompts as trainable parameters and optimizing them directly on the embedding space via gradient descent.
• Augmented Language Models
  • Augmented Language Models are models that have been enhanced with reasoning skills and the ability to use external tools.
• Retrieval
  • Retrieval involves completing tasks that require latest knowledge after the model pretraining time cutoff or internal/private knowledge base.
  • Many methods for Open Domain Question Answering depend on first doing retrieval over a knowledge base and then incorporating the retrieved content as part of the prompt.
• Programming Language and External APIs
  • Some models generate programming language statements to resolve natural language reasoning problems, offloading the solution step to a runtime such as a Python interpreter.
  • Other models are augmented with text-to-text API calls, guiding the model to generate API call requests and append the returned result to the text sequence.

From the “About” section:

goblin.tools is a collection of small, simple, single-task tools, mostly designed to help neurodivergent people with tasks they find overwhelming or difficult.

Most tools will use AI technologies in the back-end to achieve their goals. Currently this includes OpenAI's models. As the tools and backend improve, the intent is to move to an open source alternative.

The AI models used are general purpose models, and so the accuracy of their output can vary. Nothing returned by any of the tools should be taken as a statement of truth, only guesswork. Please use your own knowledge and experience to judge whether the result you get is valid.

Original tweet:

https://twitter.com/goodside/status/1672121754880180224?s=46&t=OEG0fcSTxko2ppiL47BW1Q

Text:

If you put violence, erotica, etc. in your code Copilot just stops working and I happen to need violence, erotica, etc. in Jupyter for red teaming so I always have to make an evil.⁠py to sequester constants for import.

not wild about this. please LLMs i'm trying to help you

(screenshot of evil.py full of nasty things)

Here is the link to the example epubs:

https://github.com/mshumer/gpt-author/tree/main/example_novel_outputs

I’m not sure how I feel about this project.

TL;DR (by GPT-4 🤖):

The article titled "It’s infuriatingly hard to understand how closed models train on their input" discusses the concerns and lack of transparency surrounding the training data used by large language models like GPT-3, GPT-4, Google's PaLM, and Anthropic's Claude. The author expresses frustration over the inability to definitively state that private data passed to these models isn't being used to train future versions due to the lack of transparency from the vendors. The article also highlights OpenAI's policy that data submitted by API users is not used to train their models or improve their services. However, the author points out that the policy is relatively new and data submitted before March 2023 may have been used if the customer hadn't opted out. The article also brings up potential security risks with AI vendors logging inputs and the possibility of data breaches. The author suggests that openly licensed models that can be run on personal hardware may be a solution to these concerns.

It's coming along nicely, I hope I'll be able to release it in the next few days.

Screenshot:

How It Works:

I am a bot that generates summaries of Lemmy comments and posts.

• Just mention me in a comment or post, and I will generate a summary for you.
• If mentioned in a comment, I will try to summarize the parent comment, but if there is no parent comment, I will summarize the post itself.
• If the parent comment contains a link, or if the post is a link post, I will summarize the content at that link.
• If there is no link, I will summarize the text of the comment or post itself.

Extra Info in Comments:

Prompt Injection:

Of course it's really easy (but mostly harmless) to break it using prompt injection:

It will only be available in communities that explicitly allow it. I hope it will be useful, I'm generally very satisfied with the quality of the summaries.

Link to original tweet:

https://twitter.com/sayashk/status/1671576723580936193?s=46&t=OEG0fcSTxko2ppiL47BW1Q

Screenshot:

Transcript:

I'd heard that GPT-4's image analysis feature wasn't available to the public because it could be used to break Captcha.

Turns out it's true: The new Bing can break captcha, despite saying it won't: (image)

This is a fascinating discussion of the relationship between goals and intelligence from an AI safety perspective.

I asked my trusty friend GPT-4 to summarize the video (I downloaded the subtitles and fed them into ChatGPT), but I highly recommend just watching the entire thing if you have the time.

Summary by GPT-4:

Introduction:

• The video aims to respond to some misconceptions about the Orthogonality Thesis in Artificial General Intelligence (AGI) safety.
• This arises from a thought experiment where an AGI has a simple goal of collecting stamps, which could cause problems due to unintended consequences.

Understanding 'Is' and 'Ought' Statements (Hume's Guillotine):

• The video describes the concept of 'Is' and 'Ought' statements. 'Is' statements are about how the world is or will be, while 'Ought' statements are about how the world should be or what we want.
• Hume's Guillotine suggests that you can never derive an 'Ought' statement using only 'Is' statements. To derive an 'Ought' statement, you need at least one other 'Ought' statement.

Defining Intelligence:

• Intelligence in AGI systems refers to the ability to take actions in the world to achieve their goals or maximize their utility functions.
• This involves having or building an accurate model of reality, using it to make predictions, and choosing the best possible actions.
• These actions are determined by the system's goals, which are 'Ought' statements.

Are Goals Stupid?

• Some commenters suggested that single-mindedly pursuing one goal (like stamp collecting) is unintelligent.
• However, this only seems unintelligent from a human perspective with different goals.
• Intelligence is separate from goals; it is the ability to reason about the world to achieve these goals, whatever they may be.

Can AGIs Choose Their Own Goals?

• The video suggests that while AGIs can choose their own instrumental goals, changing terminal goals is rare and generally undesirable.
• Terminal goals can't be considered "stupid", as they can't be judged against anything. They're simply the goals the system has.

Can AGIs Reason About Morality?

• While a superintelligent AGI could understand human morality, it doesn't mean it would act according to it.
• Its actions are determined by its terminal goals, not its understanding of human ethics.

The Orthogonality Thesis:

• The Orthogonality Thesis suggests that any level of intelligence is compatible with any set of goals.
• The level of intelligence is about effectiveness at answering 'Is' questions, and goals are about 'Ought' questions.
• Therefore, it's possible to create a powerful intelligence that will pursue any specified goal.
• The level of an agent's intelligence doesn't determine its goals and vice versa.

cross-posted from: https://lemmy.fmhy.ml/post/125116

The new wave of AI systems, ChatGPT and its more powerful successors, exhibit extraordinary capabilities across a broad swath of domains. In light of this, we discuss whether artificial INTELLIGENCE has arrived.

Paper available here: https://arxiv.org/abs/2303.12712 Video recorded at MIT on March 22nd, 2023