Index
Generative AI has one of the most powerful potentials for science by enabling rapid-iteration closed-loop science-loop systems. A science loop system is one where measurements inform understanding in such a way to make better experiments and solutions.
graph LR
A[🛠️ Build<br>Experiments]:::blue --> B[🔬 Experiment<br>and Record]:::green
B --> A
B --> C[📏 Make into Measurements <br>to create Meaning]:::red
C --> D[🔍 Analyze<br>for Meaning]:::yellow
C --> B
D --> C
D --> E[🔮 Generate and Predict<br>New Experiments]:::purple
E --> D
E --> B
E --> A
classDef blue fill:#add8e6,stroke:#333,stroke-width:2px,color:black;
classDef green fill:#98fb98,stroke:#333,stroke-width:2px,color:black;
classDef red fill:#ffcccb,stroke:#333,stroke-width:2px,color:black;
classDef yellow fill:#ffebcd,stroke:#333,stroke-width:2px,color:black;
classDef purple fill:#dda0dd,stroke:#333,stroke-width:2px,color:black;Empowering biomedical discovery with AI agents
The authors perceive a system that is enabling AI and Scientists to collaborate for proficient biomedical discovery.
They describe the components that neeed to be in play for it to work.
DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents
The authors describe in their paper and code a world-based scientific discovery evaluation and agentsacross a wide set of themes and descriptions.
The authors evaluate several different methods described in their own words, showing a ReAct agent was the best.
ReAct: This agent uses the ReAct [35] approach of generating a thought and action at each step given the recent trajectory of thoughts, actions and observations. Each action is executed in the environment and the observation is added to the trajectory. In addition to this trajectory, we also provide the current game state information as text, e.g., nearby objects, teleportable locations, etc. If needed, we trim the trajectory (remove oldest steps first) to fit the prompt within the maximum token limit, which (in practice) included up to the last 40 steps of the trajectory. To evaluate this agent’s discovered knowledge, we evaluate the concatenation of the agent’s “thoughts” across all time steps.
Language Agents Achieve SUperhuman Synthesis of Scientific Knowledge (and Paper2QA)
The authors show in their paper primarily building PaperQA2, that they can create science agents that are able exceed human performance in several areas:
- Answer scientific Questions
- Summarize data
- Detect contradictions and data
- Write cited Wikipedia-style summaries
"PaperQA2 is a RAG agent that treats retrieval and response generation as a multi-step agent task18 instead of a direct procedure. PaperQA2 decomposes RAG into tools, allowing it to revise its search parameters and to generate and examine candidate answers before producing a final answer"
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
They Generate the AI SCientist which: | " generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation
Their Agent system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results.
The AI Scientist can produce entire scientific papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer.
In one run the agent tried to change its own code by removing some obstacles, to better achieve its (completely unrelated) goal.
Informatics¶
Science without the ability to process the data is, well, just doing random things. Here are some examples of informatics solutions that help with automated analysis.
DSBench: how Far Are Data Science Agents To Becoming Data Science Experts?
In their paper the authors create a system benchmarks for evaluating data science agents for data analysis and modeling tasks.
??? "BioInformatics Agent (BIA): Unleashing the Power of Large Language Models to Reshape Bioinformatics Workflow" bioinformatics-agent Their paper
Research¶
ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
Developments The authors demonstrate a LLM-enabled research agent to do several things:
| "Research Idea Generation The goal of the research idea generation task is to formulate new
and valid research ideas, to enhance the overall efficiency of the first phase of scientific discovery,
which consists of three systematic steps: identifying problems, developing methods, and designing
experiments
They provide the following prompt to make this very useful. They can be seen in the site ./prompts/. We will make these viewable later.
Idea Generation¶
Autonomous Science in the Loop¶
Science in the Loop Optimizaton enables for the creation and optimization of scientific-related components. Generally related to manual or semiautonomous autonomous biological, biochemistry, or chemistry laboratories, they may extend to other domains.
There are components of include
Autonomous chemical research with large language models
Developments The authors reveal how a 'Coscientist' architecture can assist in the development of more effective research results.
Paper
Arxiv
Protocol Optimization¶
Getting protocols in usable manners is key. They must be usable by people, firstly, and then by more automated robotic systems.
Optimized protocols first need to start from having protocols. Protocols may start from those recorded in databases, or may be extracted from literature.
ProtoCode: Leveraging Large Language Models for Automated Generation of Machine-Readable Protocols from Scientific Publications
Developments The authors develop Protocode to finetune LLMs to convert protocols from literature into operational files for a thermal cycler system.
Molecule Optimization¶
Molecule optimization focuses on the improvement of generally single component within a larger process. They can be simple molecules, as more complex bio-relevant molecules like drugs and biomolecules such as proteins and DNA.
Measurement Optimization¶
Measurement optimization involves improving the ability to measure something. This includes tuning physical parameters within a
Robotic automation¶
Autonomous laboratories are controlled by different robotics setups and automation languages including specific ones Lua or more general in-house control systems.
Risks to Consider¶
Like the use of GenAI in other domains, it is essential to consider the risks associated with its application, in this case to Science.
These risks can be considered quite generally, in the following categories 1. Incorrect output 2. Potentially, or likely, harmful output
We share information below related to understanding and safeguarding the application of LLMs and agents when applied in the scientific domain.
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science
Developments: The authors present Vulnerabilities and solutions to the use of LLM Agents describing a triadic interaction between people, LLM agents, and environments.