Ortiz's method of multicultural assessment

Who is Samuel Ortiz?

Dr. Ortiz is a school psychologist.  He is also an Associate Professor of Psychology and former Director of the School Psychology Program at St. John's University, Queens, New York.  Dr. Ortiz trains and consults nationally and internationally on topics ranging from nondiscriminatory assessment to contemporary evaluation of learning disabilities. Dr. Ortiz is bilingual (Spanish) and bicultural (Puerto Rican).

Dr. Ortiz works and publishes a lot with Dawn Flanagan, who developed the application of CHC theory to learning disability definition.

The material here is taken from this chapter:

Flanagan, D. P., Ortiz, S. O., & Alfonso, V. C. (2007). Use of the cross-battery approach in the assessment of diverse individuals. Essentials of cross-battery assessment second edition, 146-205.‏

 Ortiz developed a system that helps determine whether low scores obtained by a multilingual/multicultural child result mainly from the child's cultural and linguistic difference or reflect real cognitive difficulties the child has.  Ortiz emphasizes that this method does not stand alone, but should be taken as one of many assessment procedures that also consider the child's cultural and linguistic history.

Ortiz classified tests from IQ and other batteries into a table mapping their linguistic and cultural loading.  This is the  CULTURE LANGUAGE INTERPRETIVE MATRIX  C – LIM.

A few words before we look at the tables.

Intelligence tests are usually developed from a western point of view.  The following amusing story by Alan Kaufman about his work with David Wechsler on the WISC test can attest that:

"He would usually respond calmly but occasionally I'd strike a raw nerve, and his grandfatherly smile would evaporate.  His temples would start to pulse, and his entire face and scalp would turn crimson.  I'd unconsciously move my chair back in self protection, the way I did when I tested hard core prisoners on the old WAIS and had to ask the question: "Why should we keep away from bad company?" I struck that exposed nerve when I urged him to eliminate the Comprehension item about walking away from a fight if someone much smaller starts to fight with you.  The argument that you can't walk away from any fight in a black ghetto just added fuel to his rage.  When I suggested, at a later meeting, that he just had to get rid of the item, "Why should women and children be saved first in a shipwreck?" or incur the wrath of the new wave of militant feminists, his response was instant.  With red face and a pulsing head, he stood up, leaned on his desk with extended arms, and said as if he were firing a semiautomatic, "Chivalry may be dying, chivalry may be dead.  But it will not die on the WISC."

The Wechsler-Kaufman argument shows the extent to which the content of tests reflects the beliefs and values of the people who create them.  What is considered intelligent behavior, and what is considered a right answer to a question in an intelligence test, is not something objective.  All intelligence and cognitive ability tests reflect the culture in which they were created.  The extent of the familiarity of a person with the culture in which the test was created affects his performance on the test.

Even tests that are presented as free from cultural bias are not entirely so.  Lowering the oral language requirements of a test does not entirely erase the potential linguistic bias and does not lessen much of the cultural bias.  Tests with visual content sometimes have even more culture dependent content than verbal tests (pictures of a baseball glove or bat are meaningless to Israeli children, for example).  Sometimes the visual stimuli are clear, but the test has lengthy and complex verbal instructions that can be burdensome to bilingual children. 

For the purpose of classifying tests according to their cultural and linguistic loading, Ortiz collected data of the performance of bilingual children on intelligence tests.  To the best of my understanding, most of the information he managed to glean was about the Wechsler tests.    When no data was available, he used expert consensus to classify test batteries.

 Here are Ortiz's test classifications.

How do we work with these tables?

		Degree of linguistic demand
		low	moderate	high
Degree of cultural loading	low	Performance least affected 1	2	Increasing effect of language difference 3
	Moderate	2	3	4
	high	Increasing effect of cultural difference 3	4	Performance most affected by linguistic and cultural differences 5

For each cell in the table, we compute an average score of the tests administered belonging to this cell.

Next, we ask ourselves three questions:

1.     Is the highest cell average in cell 1?

2.    Is the lowest cell average in cell 5?

3.    Do the remaining cell averages fall between the highest and lowest scores and follow a relative decline in value from cell 1 to cell 5?

 If the answer to all questions is "yes", then it is very likely that the test results are invalid and reflect lack of acculturation and limited English proficiency more so than true ability.  If the answer to any question is "no",  then the data may be valid and uncompromised by cultural or linguistic factors and can be used, in conjunction with other converging data, to support hypotheses regarding the presence of a disability.  It doesn't mean that the child has disability.  It means that we can go on to see if the child's data satisfy the criteria for LD.

I think Ortiz's method is a good starting point to multicultural assessment and leaves room for optimism about our ability to reduce cultural and linguistic bias in assessment.