We’re trialling an AI teaching assistant in our online Laws programmes to see how well it can support students with their academic questions. In this post I interview Jonathon Thomas, Associate Director, Learning Design and Production, to find out more about the project, why he’s doing this, and what he hopes to achieve.
Can you tell us about the AI teaching assistant?
We’re running a small pilot of Noodle Factory’s AI Teaching Assistant on a some of our online programmes to see if it can support our online learners with their academic questions.
An AI teaching assistant is an intelligent chatbot which is built-in to the learning platform. If students have questions while they are studying online, they can type in their questions and the AI teaching assistant provides an immediate answer. The answer is based on the information that has been previously supplied to it.
How does the AI teaching assistant work, and what can it do?
The process behind how is works is quite simple. We upload teaching material to the software and it then uses that content to formulate answers, create and mark formative assessments, recommend lesson plans and content and also help with grading exam papers. The Noodle Factory website has more information.
Additionally, and I think more interestingly, students can engage in conversational interactions with the software, similar to Open AI’s ChatGPT. This feature allows students to ask questions and receive answers in natural language. Importantly, unlike ChatGPT, which is not guaranteed to be accurate, ethical or up-to-date, the software provides contextualised tutoring, based on meticulously curated information.
How well does the teaching assistant support learning?
One of the things that impressed me the most about the software was its ability to provide students with accurate and meaningful feedback in real-time, which is crucial for successful learning. As much as we’d like to provide every online student with ample personal tutor time, it is not always feasible. Importantly, we don’t envisage that the software would, or could, replace a tutor. Instead, I believe AI can help bridge that gap by answering common academic questions, removing student frustration and freeing up tutors to focus on more complex problems.
The fact that it can also provide other functionality (grading, lesson plans, question creation and so on) was a welcome additional bonus!
Does the teaching assistant have a name?
As I’m writing this, I’m increasingly aware of the fact that we really need to find a name for this software quite soon!
So far, the project has been very unimaginatively called the AI Teaching Assistant Pilot project. I’m sure we can do better though – maybe EduBot…? The technology has a persona called Walter and I’ve noticed that this is slowly creeping into use which is nice. If the pilot proves successful, then maybe we could run a student competition to choose the name, assuming we can avoid another Boaty McBoatface situation!
What inspired the work on the project?
I was inspired about a year ago by a BBC Radio 4 programme that interviewed a start-up insurance company which was using AI to provide award-winning customer support. This got me thinking, and Googling, about how we could use this technology to improve our teaching and learning opportunities. Before long I came across Noodle Factory and reached out to them for a chat; our thinking turned out to be very much aligned and so we began to discuss a pilot.
This was about a year ago – before ChatGPT exploded on to the scene – when this all seemed new and innovative. The progression of this technology to mainstream is indicative of how powerful it can be in across many industries.
Who’s in the project team?
I’m working with a really good mix of professionals and academics which brings a variety of skills to the project including a Senior Product Innovation Manager, Senior Learning Designer, Senior Learning Technologist, an Associate Dean, Learning and Teaching, a Programme Director, and a Programmes Officer.
What are you most excited about?
Two things excite me about this project. First, I love technology that works well and solves a problem. In my slightly grumpy, middle-aged opinion, too much technology these days is over-engineered, over-designed and often just frustrating to use. ChatGPT isn’t any of those things. It’s beautifully simple to use – you stuff it full of information, ask it questions and it gives you answers. It has so many uses and is already becoming an essential tool I use day-to-day.
The other thing that excites me is helping students. I know, I know – it sounds terribly clichéd to say this, but I’ve not yet become so cynical and hollowed-out that I’ve stopped believing it! We all know how frustrating it is to not understand something or be unsure of a particular point – unable to progress, wasting precious time, unable to contact somebody to help… makes me angry just thinking about it. The purpose of the AI teaching assistant is that it can remove these blockers instantly. Stuck? Ask a question – get a response – get on with your study!
What challenges have you faced?
Setting up the product has been a breeze from a technical standpoint. It integrates seamlessly with our learning platform via LTI and is also available as a standalone website. Uploading content has been a speedy and hassle-free process.
However, the most significant hurdle we’ve encountered, which may continue to be a challenge for the foreseeable future, is related to the amount of learning material we can upload to the software. Large Language Model platforms like this one require extensive data to provide detailed and nuanced responses. Unfortunately, we haven’t obtained copyright permission to upload textbooks yet. As a result, we are currently limited to uploading only those materials where we own the copyright. Although this is substantial, it pales in comparison to the vast amount of text that students encounter throughout their studies.
We’re presently discussing potential solutions to this issue with our colleagues in the Online Library.
How will you know if the project is a success?
It will be hard to evaluate the success of the pilot from a quantitative standpoint in the timescales we are running it. We would need to see it over a number of academic cycles to establish whether it has any positive or negative impact on grades, retention or progression.
My primary hope is that both academics and students will use the software. I am so confident in the technology that I’m sure that if we can get enough content into the system and get people using it – then it will be a success! I say that as somebody who already counts ChatGPT as an essential day-to-day tool and can’t imagine life without it (or remember life before it).
What advice would you give to others looking to embark on a similar project?
I would recommend that you have a good understanding of the content that you have available, and be aware of copyright constraints around any third-party content. These models need volume to be successful – unfortunately uploading lecture notes and slides may not be sufficient to generate effective results.
Finally, what is next for the project?
The first students will begin using the teaching assistant in March 2023. We are scheduled to complete our evaluation in the Summer of 2023 and we will update colleagues on our findings. This is a commercial product, so this will need to be factored into any future decisions.
Some of the images used in this post were generated by AI based on the term ‘AI teaching assistant’.
