ArabicNLP 2026 Shared Task

Task A (Task 1): Multimodal Hateful Meme Detection

Systems receive Arabic meme images with extracted text and predict hateful content labels across binary and fine-grained settings.

ArGuard Overview Task Repository

Definition

Task A (Task 1) focuses on multimodal harmful content detection in Arabic memes. Given a meme image and its extracted Arabic text, systems identify whether the meme is hateful and, when applicable, predict the fine-grained hateful content categories.

The task is designed for Arabic-specific multimodal challenges, including dialectal language, code-switching, sarcasm, image-text interaction, cultural references, and visually implied targets.

Subtasks

  • Subtask A1: Given an Arabic meme, classify it as Hateful or Not Hateful.
  • Subtask A2: Given an Arabic hateful meme, assign the relevant fine-grained hateful content categories.

Fine-grained labels include mocking, incitement, dehumanization, slurs, contempt, inferiority, exclusion, stereotyping, extremism, threat, insults, historical references, humor, sarcasm, and other categories.

Leaderboard

The CodaBench competitions for Task A (Task 1) are open for participation:

Datasets

The dataset contains Arabic memes annotated for hateful content detection. Each item includes the meme image, extracted text, and the task labels. The released training, development, and test splits will be the authoritative source for all counts and labels.

Evaluation

The official metric is macro-F1. Accuracy, macro-precision, macro-recall, and weighted F1 may also be reported for analysis.

Subtask A1 is evaluated as binary classification. Subtask A2 is evaluated as fine-grained category prediction.

Submission

Scorers, Format Checkers, and Baselines

Scorer scripts, format checkers, baseline systems, and starter-kit material will be released in the ArGuard repository.

Guidelines

The submission process will include a system development phase using the development set and a final evaluation phase using the blind test set.

  • Each team should maintain a single submission account.
  • The most recent valid submission before the deadline will be considered the final submission.
  • Output filenames and archive formats will be specified with the starter kit.
  • Teams should include their team name and a short method description with each submission.

Submission Site

The official submission site will be linked when the competition is opened.