Authors
Liana Ermakova
Igor Kuzmin
Poojan Vachharajani
Tristan Miller
Anne-Gwenn Bosser
Jaap Kamps
Date (dd-mm-yyyy)
2027
Title
Overview of the CLEF 2026 JOKER Track
Subtitle
Humor Detection, Search, and Translation
Publication Year
2027
Number of pages
26
Publisher
ChamSpringer
ISBN
9783032391490
Document type
Conference contribution
Abstract

This paper presents an overview of the JOKER track at CLEF 2026 on automatic humour analysis introducing the following four tasks: Task 1 (Humor-aware Information Retrieval): Retrieve short humorous texts for a query in English and Hinglish, Task 2 (Pun Translation): translate puns from English to French, Task 3 (Onomastic Wordplay Translation): translate onomastic wordplay from English to French, and Task 4 (Humor Generation): generate humorous text from a given context in English, French, and Spanish. The 2026 JOKER Lab attracted a record 1,641 submissions from 22 teams and expanded previous editions with a Hinglish humor retrieval benchmark, a substantially larger pun translation corpus, and a new multilingual humor generation task. By annotating the locations of words carrying double meanings in our pun translation corpus, we enabled automatic evaluation of wordplay translation. Results show continued progress in humor-aware NLP: systems increasingly recreate wordplay rather than relying on literal translation, while frontier LLMs demonstrate promising capabilities for onomastic wordplay translation and pun generation. Despite these advances, creative language tasks remain challenging, highlighting the need for evaluation beyond reference matching and for benchmarks that capture multiple valid solutions.

URL
go to publisher's site
Permalink
https://hdl.handle.net/11245.1/b16d8dd1-ee95-4d89-9197-95d2a7d41541