Subjects: Psychology >> Psychological Measurement Subjects: Psychology >> Statistics in Psychology submitted time 2023-05-11
Abstract： Cognitive diagnostic models (CDMs) are psychometric models which have received increasing attention within the field of psychological, educational, social, biological, and many other disciplines. It has been argued that an inappropriate convergence criterion for MLE-EM (maximum likelihood estimation using the expectation maximization) algorithm could result in unpredictably distorted model parameter estimates, and thus may yield unstable and misleading conclusions drawn from the fitted CDMs. Although several convergence criteria have been developed, it remains an unexplored question, how to specify the appropriate convergence criterion for the fitted CDMs.
A comprehensive method for assessing convergence is proposed in this study. To minimize the impact by the model parameter estimation framework, a new framework adopting the multiple starting values strategy mCDM is introduced. To examine the performance of the convergence criterion for MLE-EM in CDMs, a simulation study under various conditions was conducted. Five convergence assessment methods were examined: the maximum absolute change in model parameters, the maximum absolute change in item endorsement probabilities and structural parameters, the absolute change in log-likelihood, the relative log-likelihood, and the comprehensive method. The data generating models were the saturated CDM and the hierarchical CDM. The number of items was set to J = 16 and 32. Three levels of sample sizes were considered: 500, 1000, and 4000. Three convergence tolerance value conditions were: 10-4 , 10-6 , and 10-8 . The simulated response data were fitted by the saturated CDM using the mCDM and the R package GDINA. And the maximum number of iterations was set to 50000.
Simulation results suggest that:
(1) The saturated CDM converged under all conditions. However, the actual number of iterations exceeded 30000 under some conditions, which implies that when predefined maximum iteration number is less than 30000, the MLE-EM algorithm might mistakenly stop.
(2) The model parameter estimation framework affected the performance of the convergence criteria. The performance of the convergence criteria under the mCDM framework was comparable or superior to that of the GDINA framework.
(3) Regarding the convergence tolerance values considered in this study, 10-8 consistently had the best performance in providing the maximum value of the log-likelihood and 10-4 had the worst as suggested by the higher log-likelihood value. Compared to all other convergence assessment methods, the comprehensive method in general had the best performance, especially under the mCDM framework. The performance of the maximum absolute change in model parameters was similar to the comprehensive method, however, its good performance was not guaranteed. On the contrary, the relative log-likelihood had the worst performance under the mCDM or GDINA framework.
The simulation results showed that, the most appropriate convergence criterion for MLE-EM in CDMs was the comprehensive method with tolerance 10-8 under the mCDM framework. Results from the real data analysis also demonstrated the good performance of the proposed comprehensive method and mCDM framework.
Subjects: Psychology >> Statistics in Psychology Subjects: Psychology >> Psychological Measurement submitted time 2022-07-15
Abstract：
A Q-matrix, which defines the relations between latent attributes and items, is a central building block of the cognitive diagnostic models (CDMs). In practice, a Q-matrix is usually specified subjectively by domain experts, which might contain some misspecifications. The misspecified Q-matrix could cause several serious problems, such as inaccurate model parameters and erroneous attribute profile classifications. Several Q-matrix validation methods have been developed in the literature, such as the G-DINA discrimination index (GDI), Wald test based on an incomplete information matrix (Wald-IC), and Hull methods. Although these methods have shown promising results on Q-matrix recovery rate (QRR) and true positive rate (TPR), a common drawback of these methods is that they obtain poor results on true negative rate (TNR). It is important to note that the worse performance of the Wald-IC method on TNR might be caused by the incorrect computation of the information matrix.
A new Q-matrix validation method is proposed in this paper that constructs a Wald test with a complete empirical cross-product information matrix (XPD). A simulation study was conducted to evaluate the performance of the Wald-XPD method and compare it with GDI, Wald-IC, and Hull methods. Five factors that may influence the performance of Q-matrix validation were manipulated. Attribute patterns were generated following either a uniform distribution or a higher-order distribution. The misspecification rate was set to two levels: $QM\text{=}0.15$and$QM\text{=}0.3$. Two sample sizes were manipulated: 500 and 1000. The three levels of IQ were defined as high IQ, ${{P}_{j}}\left( 0 \right)\sim U(0,0.2)$and${{P}_{j}}\left( 1 \right)\sim U(0.8,1)$; medium IQ, ${{P}_{j}}\left( 0 \right)\sim U(0.1,0.3)$ and ${{P}_{j}}\left( 1 \right)\sim U(0.7,0.9)$; and low IQ, ${{P}_{j}}\left( 0 \right)\sim U(0.2,0.4)$ and ${{P}_{j}}\left( 1 \right)\sim U(0.6,0.8)$. The number of attributes was fixed at $K\text{=}4$. Two ratios of the number of items to attribute were considered in the study: $J=16$$\left[ (K\text{=}4)\times (JK\text{=}4) \right]$ and $J=32$$\left[ (K\text{=}4)\times (JK\text{=}8) \right]$.
The simulation results showed the following.
(1) The Wald-XPD method always provided the best results or was close to the best-performing method across the different factor levels, especially in the terms of the TNR. The HullP and Wald-IC methods produced larger values of QRR and TPR but smaller values of TNR. A similar pattern was observed between HullP and HullR, with HullP being better than HullR. Among the Q-matrix validation methods considered in this study, the GDI method was the worst performer.
(2) The results from the comparison of the HullP, Wald-IC, and Wald-XPD methods suggested that the Wald-XPD method is more preferred for Q-matrix validation. Even though the HullP and Wald-IC methods could provide higher TPR values when the conditions were particularly unfavorable (e.g., low item quality, short test length, and low sample size), they obtain very low TNR values. The practical application of the Wald-XPD method was illustrated using real data.
In conclusion, the Wald-XPD method has excellent power to detect and correct misspecified q-entry. In addition, it is a generic method that can serve as an important complement to domain experts’ judgement, which could reduce their workload.
Peer Review Status:
Awaiting Review
Subjects: Psychology >> Social Psychology Subjects: Psychology >> Cognitive Psychology submitted time 2022-01-14
Abstract：
People sometimes behave unethically, which may threaten their self-concept of being moral. To cope with the threat to the moral self, people would forget these past unethical actions or related information more easily. Recent research using autobiographical memory paradigm, game paradigm, take-in paradigm, and self-reference memory paradigm provides evidence for this memory bias. Moreover, research suggest that this memory bias results from people’s need to cope with the threat to their moral self. That is, people selectively forget their unethical behaviors to maintain a positive moral self. Notably, it is only under certain conditions that this memory bias occurs. Future research should provide much more convergent evidence for the phenomenon, examine its underlying neural and cognitive mechanisms, and explore its relationships with other strategies that people use to cope with the threat to their moral self.
Peer Review Status:Awaiting Review
Subjects: Psychology >> Cognitive Psychology submitted time 2021-07-25
Abstract： Embodied language cognition highlights that language processing is not only involves internal representation of purely abstract symbols but also body and environment which play an important role. The majority of the evidences are from the field of the first language (L1), and there is still debate on whether this effect exists in second language (L2) cognition or not. We proposed two hypotheses, “automatic activation” and “native language mediation”, after summarizing the evidences for the L2 embodiment effect from behavioral and neurophysiological studies. And we analyzed the advantages, disadvantages, and influencing factors. Finally, the existing issues in this field and the research orientation in the future are discussed, studies on these issues are helpful to enrich the theory of language embodied cognition and further clarify the mechanism of bilingual representation, and effectively improve L2 teaching.
Subjects: Psychology >> Cognitive Psychology submitted time 2020-11-23
Abstract： Consensus on the origin of creativity has not been reached. Taking the advantage of high temporal resolution, electroencephalography can accurately reveal the neural oscillation mechanism in the process of creative production, which facilitates a deep understanding on the nature of creativity. In recent years, studies have revealed that alpha rhythm of neural oscillation increases along with increased creativity, which reflects an increased demand for internal processing and the top-down cognitive control during creativity generation. Meanwhile, cross-frequency coupling of neural oscillation reflects the dynamic exchange of information among multiple brain regions, such as frontal, temporal, and parietal lobes, during creative production. Future research, based on integrated theoretical framework as well as multi-level and multi-approach research tools, needs to be conducted to introduce more ecological mathematical calculation methods, and to effectively predict the trend of individual creativity development through computational neuroscience modeling, which facilitates a more comprehensively and profoundly understanding of creativity.
Hits
YES
Hits
Downloads
Comment