Stand-up comedians test ability of LLMs to write jokes

Stand-up comedians test ability of LLMs to write jokes

2024-06-21 13:00:24

A small team of AI researchers at Google's DeepMind project has found that LLMs are not very good at writing jokes that are funny. They asked stand-up comedians to use LLMs to write a stand-up routine for them and posted their findings on the arXiv preprint server.

comedians use LLMs to write a stand-up routine
Credit: AI-generated image

A small team of AI researchers at Google's DeepMind project has found that LLMs are not very good at writing jokes that are funny. They asked stand-up comedians to use LLMs to write a stand-up routine for them and posted their findings on the arXiv preprint server.

To be successful, most stand-up comedians have to write a stand-up routine and perform it on stage. Such routines, or monologs, typically involve both storytelling and jokes, or describe humorous situations. Many also employ surprising or incongruous remarks, giving the audience a sudden insight into something they may not have considered in a certain way before.

Most professional stand-up comedians spend a great deal of time polishing their routines and testing them on small audiences before performing in front of large crowds or on television specials.

In this new effort, the team at DeepMind wondered if LLMs might be capable of creating not just jokes, but entire stand-up routines. To find out, they recruited 20 professional stand-up comedians who had used LLMs in their work before. The performers used an LLM to help them write an entire routine and then rated the results.

The researchers found that LLMs were quite good at coming up with jokes; unfortunately, few if any were funny. Most, they suggested, were generic in nature and few offered anything in the way of a surprise.

Stand-up comedians test ability of LLMs to write jokes
Left: Evaluation of the instruction-tuned LLMs as creativity support tool for writing comedy using a Likert scale; each row corresponds to a question in a survey. Right box plots (box for quartiles and whiskers for min and max) show the break-down of the Creativity Support Index, respectively. Credit: arXiv (2024). DOI: 10.48550/arxiv.2405.20956

Overall, the comedians found the AI-generated jokes to lack the cutting edge that is typically needed for a joke to be funny. Many described the results as bland. But some of them did find the LLM to be useful in generating a routine that could be used for creating a basic structure around which they could build their own .

The research team suggests the results were not surprising considering that makers of LLMs use filters to prevent them from generating output that could be offensive or edgy.

More information: Piotr Wojciech Mirowski et al, A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians, arXiv (2024). DOI: 10.48550/arxiv.2405.20956

Journal information: arXiv

© 2024 Science X Network

Citation: Stand-up comedians test ability of LLMs to write jokes (2024, June 21) retrieved 21 June 2024 from https://techxplore.com/news/2024-06-comedians-ability-llms.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.