An early warning system for LLMs that can help create biological threats is being developed by OpenAI. For this kind of misuse, current models prove to be, at most, moderately useful, and they will keep improving their evaluation framework going forward.
A much talked-about possible risk associated with LLMs is easier access to data on the generation of biothreats. They sought to create assessments of the current reality of the information access risk and future monitoring strategies, building on our preparedness framework.
According to OpenAI’s largest-ever evaluation, GPT-4 only slightly improves biological threat production accuracy (see dark blue below). Even though the uplift isn’t significant enough to be definitive, this discovery serves as a springboard for more investigation and thought.
Their findings make it quite evident that more research is required in this area.
