French Contextualized Word-Embeddings with a sip of CaBeRnet: a New French Balanced Reference Corpus
Fiche du document
16 mai 2020
- handle: 10670/1.rml9pn
- hal: hal-02678358
info:eu-repo/semantics/OpenAccess
Mots-clés
Language Models French BERT ELMo Tagging Parsing NER Balanced French CorpusSujets proches
Language (New words, slang, etc.)Citer ce document
Murielle Fabre et al., « French Contextualized Word-Embeddings with a sip of CaBeRnet: a New French Balanced Reference Corpus », HAL-SHS : linguistique, ID : 10670/1.rml9pn
Métriques
Partage / Export
Résumé
This paper describes and compares the impact of different types and size of training corpora on language models like ELMO. By asking the fundamental question of quality versus quantity we evaluate four French corpora for training on parsing scores, POS-tagging and named-entities recognition downstream tasks. The paper studies the relevance of a new corpus, CaBeRnet, featuring a representative range of language usage, including a balanced variety of genres (oral transcriptions, newspapers, popular magazines, technical reports, fiction, academic texts), in oral and written styles. We hypothesize that a linguistically representative and balanced corpora will allow the language model to be more efficient and representative of a given language and therefore yield better evaluation scores on different evaluation sets and tasks.