A Study of Reliability and Consequential Validity of a Mathematics Diagnostic Assessment System in Georgia

Several school districts in Georgia (including Forsyth and Jefferson Counties) currently use the Individual Knowledge Assessment of Number (IKAN; New Zealand Ministry of Education, 2011) and the Global Strategy Stage (GloSS; New Zealand Ministry of Education, 2012) as part of a diagnostic assessment system within their multi-tiered systems of support. The IKAN and GloSS assessments were designed for use in Grades K–8 (GaDOE, 2015). IKAN provides information on students’ number knowledge (that is, magnitude comparisons, knowledge of base 10 system and meaning of decimals and fractions), and the GloSS provides information on strategies students use when solving mathematical problems. When used together, the IKAN and GloSS assessments furnish teachers with information on how students solve mathematics problems and students’ understanding of whole and rational number concepts. Teachers can then use the information from the assessments to tailor their instruction to students’ levels of understanding and address problems that underlie lack of success with grade-level curriculum. The Georgia Department of Education (GaDOE) has received positive feedback through testimonials from district-level personnel and math coaches supporting the use of IKAN/GloSS. Yet, very limited psychometric data exists to support the use of these measures. GaDOE has not conducted reliability or validity studies using its student population. While many studies have been conducted in New Zealand by the Ministry of Education, participating Georgia school districts and the GaDOE are interested in psychometric data using teachers and students in their state in the context of their state system of standards, assessments, and accountability. Thus, through the Improving Mathematics Research Alliance, the GaDOE requested REL Southeast conduct a study to determine the reliability and validity of the IKAN/GloSS diagnostic assessments. The three research questions guiding the project relate to the inter-assessor reliability of the GloSS and IKAN assessments when administered by two assessors within a one-week period and the consequential validity of using IKAN and GloSS diagnostic assessments (that is, teachers’ perception of their instructional utility for providing intervention). Student data from the IKAN and GloSS assessments administered by teachers and mathematics coaches will be used to answer the two research questions related to the inter-assessor reliability studies.

The latest form for A Study of Reliability and Consequential Validity of a Mathematics Diagnostic Assessment System in Georgia expires 2022-03-31 and can be found here.