Abstract
Automatic hate speech detection is hampered by the scarcity of labeled datasetd, leading to poor generalization. We employ pretrained language models (LMs) to alleviate this data bottleneck. We utilize the GPT LM for generating large amounts of synthetic hate speech sequences from available labeled examples, and leverage the generated data in fine-tuning large pretrained LMs on hate detection. An empirical study using the models of BERT, RoBERTa and ALBERT, shows that this approach improves generalization significantly and consistently within and across data distributions. In fact, we find that generating relevant labeled hate speech sequences is preferable to using out-of-domain, and sometimes also within-domain, human-labeled examples.
| Original language | English |
|---|---|
| Title of host publication | Findings of the Association for Computational Linguistics, Findings of ACL |
| Subtitle of host publication | EMNLP 2021 |
| Editors | Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-Tau Yih |
| Publisher | Association for Computational Linguistics (ACL) |
| Pages | 4699-4705 |
| Number of pages | 7 |
| ISBN (Electronic) | 9781955917100 |
| State | Published - 2021 |
| Event | 2021 Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021 - Punta Cana, Dominican Republic Duration: 7 Nov 2021 → 11 Nov 2021 |
Publication series
| Name | Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021 |
|---|
Conference
| Conference | 2021 Findings of the Association for Computational Linguistics, Findings of ACL: EMNLP 2021 |
|---|---|
| Country/Territory | Dominican Republic |
| City | Punta Cana |
| Period | 7/11/21 → 11/11/21 |
Bibliographical note
Publisher Copyright:© 2021 Association for Computational Linguistics.
ASJC Scopus subject areas
- Language and Linguistics
- Linguistics and Language