TY - CONF
T1 - Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Models
AU - van de Kar, Mozes
AU - Xia, Mengzhou
AU - Chen, Danqi
AU - Artetxe, Mikel
N1 - Funding Information:
We thank Ves Stoyanov, Jingfei Du, Timo Schick and Sewon Min for their feedback. Mozes van de Kar received a travel grant from ELLIS and Qualcomm to attend the conference.
Funding Information:
for those of you with the ability to book your Cuban Rent A Car in advance, all the above official websites still offer availability at the writing of this article. in 1989, Tufts University School of Medicine received a grant from the National Institutes of Health (NIH) to start the Minority High School Research Apprenticeship Program. when all is said and done, the legislature should approve the project aimed at repairing and upgrading seats and improving lighting and drainage at the facility. consciousness is a form of pain, originally, definitely.
Publisher Copyright:
© 2022 Association for Computational Linguistics.
PY - 2022
Y1 - 2022
N2 - Masked language models like BERT can perform text classification in a zero-shot fashion by reformulating downstream tasks as text infilling. However, this approach is highly sensitive to the template used to prompt the model, yet practitioners are blind when designing them in strict zero-shot settings. In this paper, we propose an alternative mining-based approach for zero-shot learning. Instead of prompting language models, we use regular expressions to mine labeled examples from unlabeled corpora, which can optionally be filtered through prompting, and used to finetune a pretrained model. Our method is more flexible and interpretable than prompting, and outperforms it on a wide range of tasks when using comparable templates. Our results suggest that the success of prompting can partly be explained by the model being exposed to similar examples during pretraining, which can be directly retrieved through regular expressions.
AB - Masked language models like BERT can perform text classification in a zero-shot fashion by reformulating downstream tasks as text infilling. However, this approach is highly sensitive to the template used to prompt the model, yet practitioners are blind when designing them in strict zero-shot settings. In this paper, we propose an alternative mining-based approach for zero-shot learning. Instead of prompting language models, we use regular expressions to mine labeled examples from unlabeled corpora, which can optionally be filtered through prompting, and used to finetune a pretrained model. Our method is more flexible and interpretable than prompting, and outperforms it on a wide range of tasks when using comparable templates. Our results suggest that the success of prompting can partly be explained by the model being exposed to similar examples during pretraining, which can be directly retrieved through regular expressions.
UR - http://www.scopus.com/inward/record.url?scp=85149444426&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85149444426&partnerID=8YFLogxK
M3 - Paper
AN - SCOPUS:85149444426
SP - 7508
EP - 7520
T2 - 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
Y2 - 7 December 2022 through 11 December 2022
ER -