Prof McGill in another criticism of CHC

    

McGill, R. J. (2017). Re (Examining) Relations between CHC Broad and Narrow Cognitive Abilities and Reading Achievement. Journal of Educational and Developmental Psychology, 7(1), 265.‏  http://www.ccsenet.org/journal/index.php/jedp/article/viewFile/66066/36510

In June 2016 I published a post which discussed a paper by Prof. Ryan McGill.  In that paper McGill argued against the PSW operational definition of learning disability (usually using CHC model).  I disagreed with him on a few points, and was delighted that he read the post and responded to it. 

This post discusses a new paper by Prof. McGill.  In this paper he deals with the question of what predicts reading better – the general ability (the IQ score) or broad CHC abilities.  Before I present his arguments (and my response to them), I think it's time to introduce Prof. McGill, especially since he is our (educational psychologists) colleague.

 Prof. Ryan J. McGill

  

Ryan J. McGill is Assistant Professor and Director of the School Psychology Program at the College of William & Mary. His research and teaching interests include psychological assessment and measurement, judgement and decision-making in school psychology, assessment and identification of specific learning disability, and quantitative psychology.   Previously, he was a faculty member in the school psychology programs at Texas Woman’s University (2014-2016) and Chapman University (2011-2014). Additionally, he was a practicing school psychologist in Southern California from 2009-2014.

These are the main points in McGill's paper as I understood them: 

With regard to CHC cognitive-achievement relationships, a series of highly cited predictive validity investigations provide evidence of differential predictive effects for CHC-related broad abilities for reading, mathematics, and written language achievement across the age span. With regard to reading achievement, Evans and colleagues (2001) examined relations between CHC broad and narrow abilities and various reading abilities across 14 age groups. Their findings suggest that several abilities (i.e., Phonological Awareness, Processing Speed [Gs], and Long-Term Retrieval [Glr]) consistently accounted for significant effects in reading achievement across the age span. As a result, they encouraged primary consideration of these and other related CHC broad and narrow abilities when investigating reading skill development. However, Evans et al. (2001) did not to include an estimate of general intelligence in their prediction models, a variable that has a rich history of accounting for meaningful levels of academic achievement variance.

Thorndike (1986) noted that 85% to 90% of predictable variance in measures of achievement may be accounted for by a single general score (i.e., FSIQ), that is thought to estimate general intellectual ability. As a consequence of  Evans et al.'s (2001) omission of an estimate of general intelligence, the unique contributions of CHC broad and narrow abilities in predicting reading abilities above and beyond a more parsimonious general intelligence dimension is unclear.

Many researchers debated which factor has more influence on reading – the general ability or broad abilities – and found evidence for both sides of the argument.  Floyd, Keith, Taub, and McGrew (2007) examined the latent predictive effects of CHC-related abilities on reading decoding. In contrast, to Evans et al. (2001), the authors chose to model a general intelligence factor and found that its influence on reading was mediated through the Stratum II broad abilities. Utilizing the same methodological approach, Benson (2007) later concluded that g had direct and significant effects on reading abilities and that the effects associated with the broad and narrow abilities was mostly small. Alternatively, Floyd, Meisinger, Gregg, and Keith (2012) suggested that an integrative model with both direct and indirect effects from g best predicted reading comprehension across development. 

Given the multidimensionality inherent in contemporary measures of CHC-related abilities, researchers have consistently recommended that the effects of the higher-order g-factor should be partialed out or controlled for prior to making inferences regarding the relative importance of lower-order cognitive variables. Failure to do so, may risk overestimating the effects of lower-order variables at the expense of the higher-order dimension.

McGill attempted to measure the influence of broad abilities on reading above and beyond the influence of the general ability.  The participants were children and adolescents ages 6-0 to 18-11 (N = 4,722) drawn from the standardization sample for the WJ III.  The WJ III COG is a multidimensional test of general intelligence for ages 2 to 90 years. The measure is comprised of 30 subtests, 14 of which contribute to the measurement of seven CHC-based Stratum II broad cluster scores: Comprehension-Knowledge (Gc), Fluid Reasoning (Gf), Auditory Processing (Ga), Visual-Spatial Thinking (Gv), Short-Term Memory (Gsm), Long-Term Retrieval (Glr), and Processing Speed (Gs). Additionally, six clinical cluster scores (Phonemic Awareness, Working Memory, Broad Attention, Cognitive Fluency, Executive Processes, and Delayed Recall) thought to reflect more narrow CHC dimensions are also available through different configurations of the subtests.  McGill wanted to know to what extent each of these cognitive measures predicts reading. 

Reading was assessed with the WJ3ACH.  The WJ III-ACH is a comprehensive academic assessment battery designed to measure five academic domains: Reading, Written Language, Mathematics, Oral Language, and Academic Knowledge. The WJ III ACH is comprised of 22 subtests that combine to provide 17 broad clusters and a total achievement composite score.

McGill found that the general ability score (GIA) accounted for statistically significant (p < .05) portions of the Basic Reading scores in all of the age brackets that were assessed. Across the 13 regression models utilized to predict Basic Reading, the GIA accounted for 40% (age 12) to 63% (age 17; M = 49%) of the criterion variance.  Broad clusters entered jointly into the second block of the regression equations accounted for 2% (age 17) to 6% (ages 6; M = 4%) variance beyond g.  Narrow clusters entered jointly into the second block of the regression equations accounted for 1% (ages 8-10, 16, 17) to 7% (age 7; M = 2%) additional variance beyond g.

The GIA accounted for statistically significant (p < .05) portions of the Reading Comprehension scores in all of the age brackets that were assessed. Across the 13 regression models utilized to predict Reading Comprehension, the GIA accounted for 46% (age 6) to 67% (age 17; M = 61%) of the criterion variance.  Broad clusters entered jointly into the second block of the regression equations accounted for 3% (age 6) to 10% (ages 17; M = 8%) additional variance beyond g.  The incremental variance coefficients attributed to individual WJ III COG broad clusters ranged from 0% to 9%. With only the variance coefficient associated with the Comprehension-Knowledge cluster at age 17 ( = .09) accounting for meaningful amounts of achievement variance on its own  Narrow clusters entered jointly into the second block of the regression equations accounted for 0% (age 14) to 9% (age 7; M = 4%) additional variance beyond g.

In light of these findings, McGill argues that the failur

e to replicate the broader results produced by Evans et al. (2001), suggest that a more circumspect appraisal of the importance of CHC dimensions in relationship to the development of reading skills may be needed in the professional literature.

McGill notes that, due to the hierarchical structure of the measurement instrument, the importance of order of entry when utilizing Hierarchical Multiple Regression Analysis (HMRA) to assess the incremental effects of independent variables must also be considered. Hale, Fiorello, Kavanaugh, Holdnak, and Aloe (2007) demonstrated that by entering the first-order factor scores from a previous iteration of the Wechsler Intelligence Scale prior to entering the FSIQ score, the predictive effects of FSIQ were diminished to the point of being inconsequential. As a result, Hale and colleagues argued that order of entry arbitrarily determines whether scores such as the GIA mean everything or nothing due to the long established fact that variables entered first into a regression equation capture greater criterion variance than variables entered later. However, order of entry is not an arbitrary process and must be determined a priori according to the expected theoretical relationships between variables.  The proposed indirect hierarchical structural model for the WJ III COG (as per CHC theory) support entering the GIA score prior to the broad and narrow clusters due to the fact that these scores are subordinate to the GIA. Further, reverse entry conflicts with CHC theory and constitutes a violation of the scientific law of parsimony.

These are McGill's arguments.

But I think this seems trivial.

There is independent evidence that broad and narrow abilities contribute to reading.  For instance it's an established fact that phonological awareness contributes to reading.  It's also known that rapid naming contributes to reading.  I think it's indisputable that Comprehension-Knowledge contributes to basic reading as well as to reading comprehension.  Since g is comprised of these and other abilities that contribute to reading, it's obvious that g will be the best predictor of reading.

This doesn't mean that from now on we'll use only the general ability as a predictor of reading.  This is of no interest to us.  What we want to know is which broad abilities within the general ability predict reading.  This is the knowledge that will lead us to better interventions. We can learn that by not including an estimate of general intelligence in the prediction model.   We don't need these broad abilities to be better predictors of reading than the general ability.  But if some broad abilities contribute more to reading than others, we want to know that.