11 mai 2021
https://creativecommons.org/licenses/by-nc-nd/4.0/ , info:eu-repo/semantics/openAccess
Gabriele Sarti, « UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations », Accademia University Press, ID : 10.4000/books.aaccademia.7733
This work describes a self-supervised data augmentation approach used to improve learning models’ performances when only a moderate amount of labeled data is available. Multiple copies of the original model are initially trained on the downstream task. Their predictions are then used to annotate a large set of unlabeled examples. Finally, multi-task training is performed on the parallel annotations of the resulting training set, and final scores are obtained by averaging annotator-specific head predictions. Neural language models are fine-tuned using this procedure in the context of the AcCompl-it shared task at EVALITA 2020, obtaining considerable improvements in prediction quality.