Polyglot conference 2024, Malta/Online

SESSION: Live experiment on GPTs ability to language identification

The digital transformation is ongoing, but without active participation, nothing will happen. The time is now!

Let us find out together how well GPT4-o is able to identify a language.

The credo is: a language GPT is able to generate in, is fine, but a language it can’t even recognize, it will not handle. So, if we ask ourselves for smaller languages or mixed language, will GPT be a genuine help for language learning, we could test it, but how? And this is what we will playfully, experimentally try to find out in this session…

Literature:

ACL paper 2024

ACL paper 2023

Meta NLLB paper

A structured approach

1. Initiate prompting: „You are a language expert…“
2. Test standard utterances in standard languages in their standard writing systems.
3. Test non-standard writing system writing in languages
4. Test dialects in a dialect continuum.
5. Test creoles and pidgins.
6. Test mixed language(s): 2, 3, 4, 5 … ok, let’s stay realistic 2, 3 (4)
7. Test robustness: gardenpaths, misleading the model non-words, logorrhea, etc.

https://etherpad.studiumdigitale.uni-frankfurt.de/p/PolyglotConferenceGPTExperiment