ChatGPT, why can’t anyone afford a house? On the Effects of LLM pre-annotation on Annotator Subjectivity (2026)
Keywords: subjectivity, annotation, agreement, LLMs, social media analysis
Abstract: Large language models (LLMs) have often been proposed as substitutes for human annotators in a variety of tasks. At the same time, there has been increased focus on the role that human subjectivity and perspective plays in data annotation. To avoid eliminating the human role in annotation entirely, the use of LLMs for pre-annotation has been suggested as an alternative approach. In this paper, we explore to which degree this approach affects subjectivity of social media annotation in English. We focus on comments regarding the current status of the housing market and label them for concern level, factors affecting housing affordability, and aspects that authors claim either exacerbate or improve the situation. To investigate this, we design an experiment involving two rounds of annotation: the first, a dataset annotated by humans only; and the second, a dataset with LLM pre-annotations curated by the same human annotators. We observe that the second setting leads to much higher agreement, as well as significant changes in label distribution and co-occurrence. Similar shifts do not appear in the LLM labels. Our findings show that use of LLMs in the annotation process leads to convergence in annotations and, thus, to an erosion of human subjectivity.
Links: Proceedings of the fifth edition of NLPerspectives.

Master's Thesis by research
Morphosyntax of Luwo transitive verbs: tenses and aspects of the Luwo base and derived paradigms (September 2021 - September 2022)
Keywords: Fieldwork, Data Collection, Morphosyntax
In short: The Luwo language, spoken in South Sudan, has remained mostly unknown to Western Linguistics. But with more language disappearing each year, it is of crucial important to collect linguistic data on small languages, so that they can remain - even if the number of speakers decrease. This is a quest I have embarked on as part of the Nilotic Research group at the University of Edinburgh. For my Master's thesis, I worked with a native speaker of Luwo, and offered a comprehensive description of the base paradigm of the Luwo transitive verb.
Links: Edinburgh Research Archive.

Master's thesis (Part 1)
Untangling LLMs: Fine-tuning m-DeBERTa-V3 for Grammatical Error Detection (April 2023 - October 2023)
Keywords: m-DeBERTa-V3, Deep Learning, Grammatical Error Detection, Pytorch
In short: since its release in March 2023, m-DeBERTa-V3 has proven to be one of the best performing LLMs (see He et al., 2023). One area in which this brand new model has not yet been tested is Grammatical Error Detection (GED). We can hypothesise that m-DeBERTa-V3 should perform exceptionally well in GED, given the fact that it builds upon two SOTA LLMs for grammatical error checking: XLM-RoBERTa (Conneau et al., 2020) and ELECTRA (Clark et al., 2020). In this project, I fine-tuned m-DeBERTa-V3 for Grammatical Error detection, using a Swedish dataset provided by Spraakbanken.
Wanna know more? Check it out on Github.

Master's thesis (Part 2) (April 2023 - October 2023)
Untangling LLMs: how good is XLM-RoBERTa when it comes to error checking? (May 2023 - September 2023)
Keywords: LLMs, Deep Learning, Grammatical Error Detection, Linguistics, Data Analysis
In short: the LLM XLM-RoBERTa is the state-of-the-art of multilingual grammatical error detection. But in which areas of the grammar does the model struggle the most?
Results: the model missed more errors related to syntax, morphology and lexicon than punctuation and orthography. More specifically, the model seems to struggle with tense coordination, agreement, missing words and prepositions.
Wanna know more? Check it out on Github.

Master's thesis (June 2021 - August 2021)
Language complexity at the era of large typological databases (June 2021 - September 2021)
Keywords: Database, WALS, Language Complexity, Complexity Metrics
In short: What are the hardest languages on the planet? This is a question that has kept linguists awake at night for decades now. Over the yeras, researchers have attempted to create metrics to assess linguistic complexity, based on large typological databases such as WALS. But are these metrics reliable? How can we use large database to measure complexity in the most objective way? These are some of the questions I attempted to answer during my Master's thesis on Linguistic Complexity.

The Alphabet Challenge (October 2023)
Keywords: Python, Pygame, quiz, alphabets
In short: a fun side-project where I build a quiz using Pygame. And of course, it is about languages - and more precisely, about alphabets. Can you recognize them all?
Current tasks: working on the game interaction to make the interface more dynamics. Score counter is also coming soon!
Wanna play? Keep an eye on the progresses on Github.

Comparing LLMs (May 2023)
Keywords: LLMs, BERT, ELECTRA
In short: it is hard to believe that LLMs did not exist a decade ago, given the number of language models that have been released in the last few years. In fact, it is sometimes hard to pick the right language model for a given task, since the differences and similarities between them often reside in complex architectural properties. But as a LLM enthusiast, I happily delved into numerous academic articles to present an overview table of LLMs, from BERT to m-DeBERTa-V3, without forgetting non-BERT models such as ELECTRA.
Wanna check it out? Table available on Github.

A dialogue-based language game (March 2023 - April 2023)
Keywords: Azure, XState, JavaScript, Dialogue System
In short: what is even better than being passionate about languages? Sharing this passion with others. Here is a Swedish vocabulary game, coded in JavaScript using Microsoft Azure and the XState library. Choose a topic, and tell me what you see on the images - in Swedish, please! Note that this game requires basic knowledge of Swedish, but a demo will be uploaded soon if you don't speak it!
Wanna know more? code available on Github. Wanna play? click! Note hat you may need to clear the cache to see the images properly.