Actually Useful AI

I forget where I saw it now, but I ran across a story wherein a teacher gave an assignment to get ChatGPT to write an essay on whatever subject the students were learning and then the students were to write an essay on the accuracy and inaccuracy of the ChatGPT essay. I thought that was pretty genius.

Ethan Mollick has two recent articles related to this topic:

• The Homework Apocalypse
• Assigning AI: Seven Ways of Using AI in Class

It's been a while since I was in a formal classroom setting but as an engineer I'd assert that I'm constantly learning. So I'll offer my perspective on AI in education for those continuing education.

I find myself taking more risks at work and in my personal projects in experimenting with new technology and languages. AI's shortcomings grow exponentially as technical complexities of the prompt grow linearly, but for a beginner getting their feet wet in a subject it lowers the bar considerably in entry to the topic. I found I am not plagued so much by "analysis paralysis" after reading blogs and tutorials that are written by authors with varied understanding in the subject. With a few prompts I can effectively "filter down" the topics I need to read more about to produce something useful. No more fear that "what if i missed something."

Then there's the aspect of creating a tailored refresher for yourself on a class of "Stuff you have to relearn every time you have to use it" (love that comment). Or asking an AI to explain what a piece of content means. For example, if you wrote a really complex Makefile, dumping a tree of the repository and asking the AI to expand all the variables in the Makefile, I can now read what every step is doing.

But you definitely hit the nail on the head with pointing that "it opens a door to potential misuse". I become dependent on it for doing some tasks that I will only otherwise learn by doing. And in the context of data storage, in some ways I become less efficient and more error prone because I no longer access the knowledge I have cached in memory (my brain) and instead access data on disk (taking the time to ask an AI) that can retrieve incorrect data (data corruption, bad sectors, etc) that are difficult to catch.

As an educational tool, I think those that behave as "AI gluttons" and overindulge in use of AI to the point of excess or greed, risk eroding their critical thinking and creativity. And those that do not supplement their learning with AI risk of being left behind by those who use it responsibly. In the same way I think AI will not replace programmers, but programmers that use AI will replace programmers that don't.

"Potential misuse" is a bit of a weasel phrase... student use of AI assistants is rampant, the ways they use them are almost always academic misconduct, so it's actual misuse.

Our institution bans use of AI assistants during assessments, unless permitted by a subject's coordinator. This is because using ChatGPT in a way that's consistent with academic integrity is basically impossible. Fixing this means fixing ChatGPT etc, not reimagining academic integrity. Attribution of ideas, reliability of sources, and individual mastery of concepts are more important than ever in the face of LLMs' super-convincing hallucinations.

There are no Luddites where I teach. Our university prepares students for professional careers, and since in my field we use LLMs all day long for professional work, we also have to model this for students and teach them how it's done. I demonstrate good and bad examples from Copilot and ChatGPT, quite frequently co-answer student questions in conversation with ChatGPT, and always acknowledge LLM use in materials preparation.

I also have a side project that provides a chat interface to the subject contents (GPT4 synthesis over a vector store). It dramatically improves the quality of AI assistant answers, and makes it much easier to find where in the materials a concept was discussed. Our LMS search sucks even for plain text content. This thing fixes that and also indexes into code, lecture recordings, slides, screenshots, explainer videos... I'm still discovering new abilities that emerge from this setup.

I think the future is very uncertain. Students who are using ChatGPT to bluff their way through courses have no skills moat and will find their job roles automated away in very short order. But this realisation requires a two-year planning horizon and the student event horizon is "what's due tomorrow?" I haven't seen much discussion of AI in education that's grounded in educational psychology or a practical understanding of how students actually behave. AI educational tools will be a frothy buzzword-filled market segment where a lot of money is made/spent but overall learning outcomes remain unchanged.