Foundational Challenges in Assuring Alignment and Safety of Large Language Models

Usman Anwar; Abulhair Saparov; Javier Rando; Daniel Paleka; Miles Turpin; Peter Hase; Ekdeep Singh Lubana; Erik Jenner; Stephen Casper; Oliver Sourbut; Benjamin L. Edelman; Zhaowei Zhang; Mario Günther; Anton Korinek; Jose Hernandez-Orallo; Lewis Hammond; Eric Bigelow; Alexander Pan; Lauro Langosco; Tomasz Korbak; Heidi Zhang; Ruiqi Zhong; Seán Héigeartaigh; Gabriel Recchia; Giulio Corsi; Alan Chan; Markus Anderljung; Lilian Edwards; Aleksandar Petrov; Christian Schroeder de Witt; Sumeet Ramesh Motwani; Yoshua Bengio; Danqi Chen; Philip H.S. Torr; Samuel Albanie; Tegan Maharaj; Jakob Foerster; Florian Tramer; He He; Atoosa Kasirzade; Yejin Choi; David Krueger

Foundational Challenges in Assuring Alignment and Safety of Large Language Models

Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco, Tomasz KorbakHeidi Zhang, Ruiqi Zhong, Seán Héigeartaigh, Gabriel Recchia, Giulio Corsi, Alan Chan, Markus Anderljung, Lilian Edwards, Aleksandar Petrov, Christian Schroeder de Witt, Sumeet Ramesh Motwani, Yoshua Bengio, Danqi Chen, Philip H.S. Torr, Samuel Albanie, Tegan Maharaj, Jakob Foerster, Florian Tramer, He He, Atoosa Kasirzade, Yejin Choi, David Krueger

Research output: Contribution to journal › Article › peer-review

12 Scopus citations

Abstract

This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose 200+ concrete research questions.

Original language	English (US)
Journal	Transactions on Machine Learning Research
Volume	2024
State	Published - 2024

All Science Journal Classification (ASJC) codes

Computer Vision and Pattern Recognition
Artificial Intelligence

Cite this

Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., Günther, M., Korinek, A., Hernandez-Orallo, J., Hammond, L., Bigelow, E., Pan, A., Langosco, L., ... Krueger, D. (2024). Foundational Challenges in Assuring Alignment and Safety of Large Language Models. Transactions on Machine Learning Research, 2024.

@article{bd276d493547496f969bc020ee5a8287,

title = "Foundational Challenges in Assuring Alignment and Safety of Large Language Models",

abstract = "This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose 200+ concrete research questions.",

author = "Usman Anwar and Abulhair Saparov and Javier Rando and Daniel Paleka and Miles Turpin and Peter Hase and Lubana, \{Ekdeep Singh\} and Erik Jenner and Stephen Casper and Oliver Sourbut and Edelman, \{Benjamin L.\} and Zhaowei Zhang and Mario G{\"u}nther and Anton Korinek and Jose Hernandez-Orallo and Lewis Hammond and Eric Bigelow and Alexander Pan and Lauro Langosco and Tomasz Korbak and Heidi Zhang and Ruiqi Zhong and Se{\'a}n H{\'e}igeartaigh and Gabriel Recchia and Giulio Corsi and Alan Chan and Markus Anderljung and Lilian Edwards and Aleksandar Petrov and \{de Witt\}, \{Christian Schroeder\} and Motwani, \{Sumeet Ramesh\} and Yoshua Bengio and Danqi Chen and Torr, \{Philip H.S.\} and Samuel Albanie and Tegan Maharaj and Jakob Foerster and Florian Tramer and He He and Atoosa Kasirzade and Yejin Choi and David Krueger",

year = "2024",

language = "English (US)",

volume = "2024",

journal = "Transactions on Machine Learning Research",

issn = "2835-8856",

publisher = "Transactions on Machine Learning Research",

}

Anwar, U, Saparov, A, Rando, J, Paleka, D, Turpin, M, Hase, P, Lubana, ES, Jenner, E, Casper, S, Sourbut, O, Edelman, BL, Zhang, Z, Günther, M, Korinek, A, Hernandez-Orallo, J, Hammond, L, Bigelow, E, Pan, A, Langosco, L, Korbak, T, Zhang, H, Zhong, R, Héigeartaigh, S, Recchia, G, Corsi, G, Chan, A, Anderljung, M, Edwards, L, Petrov, A, de Witt, CS, Motwani, SR, Bengio, Y, Chen, D, Torr, PHS, Albanie, S, Maharaj, T, Foerster, J, Tramer, F, He, H, Kasirzade, A, Choi, Y & Krueger, D 2024, 'Foundational Challenges in Assuring Alignment and Safety of Large Language Models', Transactions on Machine Learning Research, vol. 2024.

TY - JOUR

T1 - Foundational Challenges in Assuring Alignment and Safety of Large Language Models

AU - Anwar, Usman

AU - Saparov, Abulhair

AU - Rando, Javier

AU - Paleka, Daniel

AU - Turpin, Miles

AU - Hase, Peter

AU - Lubana, Ekdeep Singh

AU - Jenner, Erik

AU - Casper, Stephen

AU - Sourbut, Oliver

AU - Edelman, Benjamin L.

AU - Zhang, Zhaowei

AU - Günther, Mario

AU - Korinek, Anton

AU - Hernandez-Orallo, Jose

AU - Hammond, Lewis

AU - Bigelow, Eric

AU - Pan, Alexander

AU - Langosco, Lauro

AU - Korbak, Tomasz

AU - Zhang, Heidi

AU - Zhong, Ruiqi

AU - Héigeartaigh, Seán

AU - Recchia, Gabriel

AU - Corsi, Giulio

AU - Chan, Alan

AU - Anderljung, Markus

AU - Edwards, Lilian

AU - Petrov, Aleksandar

AU - de Witt, Christian Schroeder

AU - Motwani, Sumeet Ramesh

AU - Bengio, Yoshua

AU - Chen, Danqi

AU - Torr, Philip H.S.

AU - Albanie, Samuel

AU - Maharaj, Tegan

AU - Foerster, Jakob

AU - Tramer, Florian

AU - He, He

AU - Kasirzade, Atoosa

AU - Choi, Yejin

AU - Krueger, David

PY - 2024

Y1 - 2024

N2 - This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose 200+ concrete research questions.

AB - This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose 200+ concrete research questions.

UR - https://www.scopus.com/pages/publications/105000284817

UR - https://www.scopus.com/inward/citedby.url?scp=105000284817&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:105000284817

SN - 2835-8856

VL - 2024

JO - Transactions on Machine Learning Research

JF - Transactions on Machine Learning Research

ER -

Foundational Challenges in Assuring Alignment and Safety of Large Language Models

Abstract

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this