Evaluation of AI-Generated Lab Notebook Entries
Assess AI's capability to accurately and completely document scientific work, and understand the limitations and risks of AI-assisted documentation.
Recommended for: Laboratory courses emphasizing scientific documentation and reproducibility
Procedure
- Students complete an experimental procedure or laboratory activity, maintaining detailed contemporaneous notes following standard laboratory notebook practices
- Students input their raw laboratory notes into an AI system and request generation of a formal, complete laboratory notebook entry
- Students systematically evaluate whether the AI-generated entry:
- Accurately represents procedures performed: Are all experimental steps correctly described without fabrication or misrepresentation?
- Maintains completeness: Are all procedural steps, observations, measurements, and results included without omission?
- Preserves scientific precision: Are quantities, units, conditions, concentrations, and technical terminology exact and unambiguous?
- Follows documentation standards: Does the entry conform to laboratory notebook requirements and scientific documentation best practices?
- Retains significant details: Are potentially important observations, anomalies, or contextual information preserved?
- Introduces errors or fabrications: Does the AI add information that was not present in the original notes or misinterpret recorded data?
- Students produce revised notebook entries correcting all identified deficiencies
- Students compare their corrected version with their original handwritten notes to identify information lost or distorted through the AI intermediation
Discussion questions
- What types of information did the AI systematically add, omit, or misrepresent during the documentation process?
- What are the research integrity and reproducibility risks of using AI to document experimental work?
- What role, if any, might AI appropriately play in scientific documentation while maintaining standards of accuracy and integrity?
- How might AI-assisted documentation affect the long-term reproducibility and verifiability of scientific research?
Assessment opportunity
Students submit both the AI-generated entry and their corrected version, along with a detailed analysis of discrepancies and their implications for scientific integrity.