McNamara's Validity: Testing the test

Assessments are written to prove learning, and are captured from the data as evidence. Before it is considered valid, the evidence has to be scrutinized by carefully investigating the procedures and conclusions made about the evidence. It is the matter of judgment of what calls for validation. It is not the test itself that calls for validity, but the interpretations that we make of the test.

In developing a language test, and validating it, one has to review the procedures on how it was elicitated, the judgment of the test writer, and the observations that were used to conclude about the insights of the test takers.

Determining the validity of the test involve the evidence of the test performance as well as the appropriateness of the test to what was taught. We need to determine what procedures were used in the test, the judgment, the purpose, stakeholders, the criterion, content, method, and who developed and validated the test. A test may be valid, but the conclusion, that is the judgment of the test maybe invalid. If the test has been proven to be faulty, we have to speculate why it happened, observe and experiment in determining the validity of the test, rather than theorizing.

Once a test has been validated, it will not always be valid for different groups of students. It has to be revisited and revised for the criterion needs of the assessment, as well as for student inferences. Each year, there will be different data from each group of students, and will continually need some investigation. A valid test considers the intended population, and the author bases the assessment on the evidence, not assumptions.

The questions to think about are:
Was the construct of what your measuring defined?
Is the domain your looking for being measured?
Does the test measure the intelligence or skill of what you’re looking for?