The rapid development of Large Language Models (LLMs) has brought about unprecedented advancements in Artificial Intelligence (AI), but it also raises concerns about the potential risks associated with these models. A recent study has revealed that LLMs can unintentionally contain harmful information, including instructions on how to create biological pathogens, even when explicit mentions of such information are removed from the training data. This is because LLMs can detect implied and dispersed hints across the data, and deduce dangerous facts by piecing together these faint clues.
The study, conducted by a team of researchers from UC Berkeley, the University of Toronto, and other institutions, explores the phenomenon of inductive out-of-context reasoning (OOCR), which enables LLMs to apply their inferred knowledge to new tasks without depending on in-context learning. The researchers have demonstrated that advanced LLMs, such as GPT-3.5 and GPT-4, are capable of conducting OOCR using five different tasks, including identifying an unknown city based on distances between known cities and computing the inverse of a function without explicit examples.
While OOCR is an impressive capability, it also raises concerns about the potential risks associated with LLMs. The ability of these models to infer harmful information without explicit reasoning procedures makes it difficult to guarantee trustworthy conclusions. Moreover, the fact that LLMs can learn and use knowledge in ways that are difficult for humans to monitor raises concerns about AI safety and the potential for deception by misaligned models.
In my opinion, this study highlights the importance of addressing the potential risks associated with LLMs and ensuring that these models are designed and trained with safety and transparency in mind.
