Summary of the RBB-LLM Framework Methodology
The RBB-LLM framework comprises three key components: peer review document collection and filtering, reflection bank construction, and LLM-assisted reasoning. Initially, peer-reviewed documents from Nature Communications (January 1, 2024 – September 25, 2024) are collected and structured via a custom algorithm that efficiently matches reviewer comments and author responses, converting unstructured data into JSON format. The reflection bank is constructed as the LLM employs chain-of-thought (CoT) reasoning to produce responses while reflecting critically on its output against human responses, generating key quadruples (reviewer comment, LLM response, human response, and reflection). During the response-writing phase, the LLM retrieves relevant information from the reflection bank to enhance response quality. Evaluation methodologies include comparison across different LLMs and human assessments, focusing on producing responses that align with the essence of human feedback, ensuring a balance between accuracy and usability for researchers. This lightweight framework emphasizes accessibility for any researcher with standard computational capabilities.