The analysis utilized a publicly available eye-tracking and fMRI dataset from 52 native English speakers reading STEM articles. After exclusions, the eye-tracking analysis involved 51 participants (average age 22.6), while the fMRI analysis included 50 participants (average age 22.5). Article texts were drawn from credible sources and rigorously revised. Each participant read sentences sequentially in an fMRI scanner, responding to comprehension questions post-reading. Employing state-of-the-art LLMs, particularly LLaMA and its instruction-tuned variants, the study investigated the models’ alignment with human behavior and neural data. Scaling and fine-tuning effects were assessed through next-word prediction loss and attention matrix comparisons. Ridge regression was used to evaluate model sensitivity to specific instructions and trivial attention patterns. The study’s findings suggest a significant relationship between LLM representations and human cognitive processes while employing robust statistical methods to ensure reproducibility and accurate data interpretation.
Source link
Share
Read more