AimswebPlus Technical Manual Plus
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 78
| Download | |
| Open PDF In Browser | View PDF |
Technical Manual Table of Contents Introduction ........................................................................................................................................................ 1 Standardization Sample ..................................................................................................................................... 1 Norm Sample ...................................................................................................................................................... 3 Standardization Sample Descriptives .............................................................................................................. 5 Norm Sample Descriptives ............................................................................................................................... 9 Reliability ........................................................................................................................................................... 14 Early Numeracy Equivalency Studies............................................................................................................. 16 Early Literacy Equivalency Studies ................................................................................................................. 17 Math Equivalency Studies............................................................................................................................... 18 Reading Equivalency Studies .......................................................................................................................... 18 Equivalency of Progress Monitoring Forms .................................................................................................. 23 Validity ............................................................................................................................................................... 45 Early Numeracy Criterion Validity ................................................................................................................ 46 Early Literacy Criterion Validity..................................................................................................................... 46 Math Criterion Validity .................................................................................................................................. 47 Reading Criterion Validity .............................................................................................................................. 49 Classification Accuracy .................................................................................................................................... 51 Early Numeracy Classification Accuracy ....................................................................................................... 52 Early Literacy Classification Accuracy ........................................................................................................... 53 Math Classification Accuracy ......................................................................................................................... 55 Reading Classification Accuracy .................................................................................................................... 58 References ......................................................................................................................................................... 61 Appendix: aimswebPlus Measures ................................................................................................................. 63 Tables Table 1 Demographic Characteristics of the Standardization Sample ........................................................ 2 Table 2 Demographic Characteristics of the Norm Sample, Kindergarten and Grade 1 ........................... 3 Table 3 Demographic Characteristics of the Norm Sample for Math, Grades 2 Through 8 ..................... 4 Table 4 Demographic Characteristics of the Norm Sample for Reading, Grades 2 Through 8 ................. 4 Table 5 Descriptive Statistics for Early Numeracy Measures in the Standardization Sample, Kindergarten and Grade 1 ................................................................................................................ 6 Table 6 Descriptive Statistics for Early Literacy Measures in the Standardization Sample, Kindergarten and Grade 1 ................................................................................................................ 6 Table 7 Descriptive Statistics for Math Measures in the Standardization Sample, Grades 2 Through 8.......................................................................................................................... 7 aimswebPlus For more information visit aimswebPlus.com | i | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table of Contents cont. Tables cont. Table 8 Descriptive Statistics for Reading Measures in the Standardization Sample, Grades 2 Through 8.......................................................................................................................... 8 Table 9 Descriptive Statistics for Early Numeracy Measures and Composites in the Norm Sample, Kindergarten and Grade 1 ................................................................................................................ 9 Table 10 Descriptive Statistics for Early Literacy Measures and Composites in the Norm Sample, Kindergarten and Grade 1 ............................................................................................................ 10 Table 11 Descriptive Statistics for Math Measures and Composites in the Norm Sample, Grades 2 Through 8 ...................................................................................................................... 11 Table 12 Descriptive Statistics for Reading Measures and Composites in the Norm Sample, Grades 2 Through 8 ...................................................................................................................... 12 Table 13 Concepts & Applications Total Score Means and Quantiles, by Grade and Season ................. 13 Table 14 Oral Reading Fluency Words Read Correctly (WRC) Means and Lexiles, by Grade and Season .................................................................................................................... 13 Table 15 Benchmark Composite Scoring Rules, by Subject, Grade, and Season ...................................... 15 Table 16 Reliability of Early Numeracy Measures and Composites, Kindergarten and Grade 1.............. 19 Table 17 Reliability of Early Literacy Measures and Composites, Kindergarten and Grade 1 .................. 20 Table 18 Reliability of Math Measures and Composites, Grades 2 Through 8 ......................................... 21 Table 19 Reliability of Reading Measures and Composites, Grades 2 Through 8 ..................................... 22 Table 20 Average Alternate-Form Reliability of Single ORF Stories, by Grade and Season ..................... 22 Table 21 Average Alternate-Form Reliability of Single SRF Stories, by Grade and Season ...................... 23 Table 22 Reliability of the Median of Three SRF Story Reading Rates, by Grade ..................................... 23 Table 23 Statistical Equivalency of Progress Monitoring Forms, Number Naming Fluency ...................... 24 Table 24 Statistical Equivalency of Progress Monitoring Forms, Quantity Total Fluency.......................... 25 Table 25 Statistical Equivalency of Progress Monitoring Forms, Quantity Difference Fluency ................. 26 Table 26 Statistical Equivalency of Progress Monitoring Forms, Number Comparison Fluency–Pairs ............................................................................................ 27 Table 27 Statistical Equivalency of Progress Monitoring Forms, Math Facts Fluency–1 Digit ................... 28 Table 28 Statistical Equivalency of Progress Monitoring Forms, Math Facts Fluency–Tens ...................... 29 Table 29 Statistical Equivalency of Progress Monitoring Forms, Letter Word Sounds Fluency ............... 29 Table 30 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 1) ........... 30 Table 31 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 2) ........... 31 Table 32 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 3) ........... 32 Table 33 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 4) ........... 33 aimswebPlus For more information visit aimswebPlus.com | ii | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table of Contents cont. Tables cont. Table 34 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 5) ........... 34 Table 35 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 6) ........... 35 Table 36 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 7) ........... 36 Table 37 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 8) ........... 37 Table 38 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 2) ....................................................................................................................................... 38 Table 39 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 3) ....................................................................................................................................... 39 Table 40 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 4) ....................................................................................................................................... 40 Table 41 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 5) ....................................................................................................................................... 41 Table 42 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 6) ....................................................................................................................................... 42 Table 43 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 7) ....................................................................................................................................... 43 Table 44 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 8) ....................................................................................................................................... 44 Table 45 Early Numeracy Composite Score Predictive Validity Coefficients, by Grade, Season, and Criterion Measure .................................................................................................... 46 Table 46 Early Numeracy Spring Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure ................................................................................................. 46 Table 47 Early Literacy Predictive Validity Coefficients, by Grade, Season, and Criterion Measure ....... 47 Table 48 Early Literacy Concurrent Validity Coefficients, by Grade and Criterion Measure ................... 47 Table 49 Math Composite Score Predictive Validity Coefficients, by Grade and Criterion Measure ......................................................................................................................... 48 Table 50 Math Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure ......................................................................................................................... 49 Table 51 Reading Composite Score Predictive Validity Coefficients, by Grade and Criterion Measure ......................................................................................................................... 50 Table 52 Reading Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure ......................................................................................................................... 50 Table 53 Classification Accuracy, Two-by-Two Model .............................................................................. 51 aimswebPlus For more information visit aimswebPlus.com | iii | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table of Contents cont. Tables cont. Table 54 Classification Accuracy Statistics................................................................................................... 52 Table 55 Classification Accuracy of Early Numeracy Composite Scores and TCAP ................................ 53 Table 56 Classification Accuracy of Fall Letter Naming Fluency with Winter Early Literacy Composite Scores and Word Reading Fluency (Kindergarten) ................................................. 54 Table 57 Classification Accuracy of Fall Letter Naming Fluency with Winter Early Literacy Composite Scores and R–CBM (Kindergarten) .......................................................................... 54 Table 58 Classification Accuracy of Fall Oral Reading Fluency Scores and Spring ITBS (Grade 1) .......... 55 Table 59 Classification Accuracy of Math Composite Scores and ITBS (Grade 2) and ISAT (Grades 3–8) ........................................................................................................................ 56 Table 60 Classification Accuracy of Math Composite Scores and NWEA–MAP (Grades 2–6) .............. 56 Table 61 Classification Accuracy of Math Composite Scores and STAAR (Grades 3–8)......................... 57 Table 62 Classification Accuracy of Math Composite Scores and NMSBA (Grades 6–8) ....................... 57 Table 63 Classification Accuracy of Reading Composites Scores and ISAT (Grades 3–8)....................... 58 Table 64 Classification Accuracy of Reading Composites Scores and NWEA–MAP (Grades 2–6) ........ 59 Table 65 Classification Accuracy of Reading Composites Scores and MAP–GLA (Grades 3–8)............. 59 Table 66 Classification Accuracy of Reading Composites Scores and STAAR (Grades 3–5) .................. 60 Table A1 Early Numeracy Measure Descriptions ....................................................................................... 63 Table A2 Early Literacy Measure Descriptions ........................................................................................... 65 Table A3 Math Measure Descriptions ......................................................................................................... 67 Table A4 Reading Measure Descriptions .................................................................................................... 69 aimswebPlus For more information visit aimswebPlus.com | iv | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Introduction aimswebPlus® is an assessment, data management, and reporting system that combines standards-aligned assessments of math and reading achievement with brief curriculum-based measurement (CBM) of critical math and reading basic skills for Kindergarten through Grade 8 students. This system provides reliable, valid, and nationally normed scores for Fall, Winter, and Spring benchmark assessments, and provides all of the features and content for multi-tiered systems of support (MTSS). This manual discusses the technical characteristics of the aimswebPlus measures, including demographic characteristics, descriptive statistics, reliability and validity data, and classification accuracy information. In addition, an overview of the aimswebPlus measures available at each grade level is provided in the appendix. Note that this manual is dynamic document—additional information will be added as new research is conducted and new evidence is collected. Also, this manual is designed to be utilized in conjunction with the aimswebPlus Development Manual, which provides detailed information regarding the rationale for the aimswebPlus measures, descriptions of the developmental stages, and the supporting scientific research. Standardization Sample Over 31,000 students participated in the aimswebPlus standardization study, with data collected during the 2013–2014 school year. Most participating students completed testing in each of the Fall, Winter, and Spring test sessions. Table 1 provides a summary of the demographic characteristics of the standardization sample for the math and reading measures at each grade level. Characteristics are reported for sex, race/ethnicity, and English language learner status. Sampling was conducted at the school level, by grade. Schools indicated the grade(s) that would participate in testing and were then assigned to reading, math, or both content areas. Participating schools were required to assess to all students in the selected grades except those with moderate to severe intellectual disabilities or moderate to severe motor impairment and those who are blind, deaf, or had an English Language Proficiency score of less than 3. The standardization sample at each grade level reflects adequate representation across each demographic category, enabling the selection of normative samples that are representative of the U.S. population. aimswebPlus For more information visit aimswebPlus.com | 1 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 1 Demographic Characteristics of the Standardization Sample aimswebPlus For more information visit aimswebPlus.com | 2 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Norm Sample Tables 2 through 4 present the demographic characteristics of the normative samples for the math and reading measures at each grade level. To be included in the norm sample, students had to complete the set of measures assigned to them (reading, math, or both). The percentage of students completing all assigned measures in all three seasons generally exceeded 90% in Math (Grades 2–8) and Early Literacy (Kindergarten and Grade 1). Approximately 85% of students completed all Early Numeracy measures (Kindergarten and Grade 1) and all Reading measures (Grades 2–8) in all three seasons. The dropout pattern was unrelated to demographic characteristics and was generally consistent across participating schools, with two exceptions. First, one school dropped out after the Winter testing session in the Early Numeracy study. Second, Oral Reading Fluency was administered on two separate platforms during Fall testing, which then had to be combined by matching various student characteristics, including student name. About 15% of the cases could not be matched and were excluded from the remaining data analyses. Although the standardization samples were reasonably representative of the U.S. student population across demographic categories (even after listwise deletion of students not completing all assigned measures), a resampling method was used to generate the final norm samples. By using this resampling method, a perfect match to the U.S. Census Bureau American Community Survey (U.S. Census Bureau, 2013) by sex, race/ethnicity, and ELL status was obtained. This matching was done to improve precision and reduce bias in the norms. The resampling algorithm used a target total sample size by grade and subject, resulting in target counts for each demographic based on U.S. census percentages. The total target sample size for each grade was identified such that it did not exceed more than twice the original sample and no student would be resampled more than eight times. This resampling technique is analogous to weighting each student. Table 2 summarizes the characteristics of the Early Literacy and Early Numeracy norm samples (Kindergarten and Grade 1). Note that SES percentages are based on free and reduced lunch data for Tables 2 through 4. Table 2 Demographic Characteristics of the Norm Sample, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 3 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Tables 3 and 4 summarize the characteristics of the Math (Grades 2–8) and Reading (Grades 2–8) norm samples, respectively. For Reading, note that Oral Reading Fluency sample selections based on sex are generally close to an even split between males and females, and characteristics based on race/ethnicity are closely matched to U.S. population estimates. The race/ethnicity sample characteristics for Reading Comprehension, Vocabulary, and Silent Reading Fluency exactly match those of the U.S. population estimates, per the previous weighting discussion. Table 3 Demographic Characteristics of the Norm Sample for Math, Grades 2 Through 8 Table 4 Demographic Characteristics of the Norm Sample for Reading, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 4 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Standardization Sample Descriptives For most Early Numeracy and Early Literacy measures (Kindergarten and Grade 1), descriptive statistics are based on the total number correct score (e.g., number of correctly answered items). The two exceptions are Phoneme Segmentation (total number of correct phonemes) and Oral Reading Fluency (mean number of words read correctly in two stories, each read for 1 minute). The descriptive statistics for Math and Reading are based on the following: • • • • Oral Reading Fluency: Mean number of words read correctly in two stories, each read for 1 minute. Silent Reading Fluency: Words read silently per minute. Number Comparison Fluency–Triads and Mental Computation Fluency: Adjusted total scores, in which 0.5 points are subtracted for every item answered incorrectly and the result rounded to the nearest whole number. Concepts & Applications, Vocabulary, and Reading Comprehension: Scores reported on a vertical standardized scale that spans Grades 2 through 8, is centered on Spring of Grade 5, and has a mean of 200. Each benchmark form within a given grade was developed from a common blueprint, with the resulting forms nearly equivalent in difficulty. Thus, score gains can be interpreted as actual achievement growth. One way to interpret the magnitude of the gain is to express it in Fall (or Winter) standard deviation (SD) units. Doing so enables direct comparison of gains across measures and grades. As expected, scores for each grade tend to increase across seasons. Large annual gains (>0.7 SD units) are common in Kindergarten through Grade 3. In Grades 4 through 8, gains are more modest, generally ranging from about 0.3 to 0.5 SD units. Two measures—Initial Sounds and Auditory Vocabulary—show very small gains. These two Early Literacy measures were designed to support diagnostic interpretation of results for the lowest performing students; as such, they are relatively easy and are not expected to be sensitive to growth for the average performing student. Tables 5 through 8 provide standardization sample sizes, means, and standard deviations by season and grade for all of the aimswebPlus reading and math measures. Note that the results shown are based on students with a valid score on each measure in a given season, while dashed lines indicate that a given measure is not administered in a particular grade or season. aimswebPlus For more information visit aimswebPlus.com | 5 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 5 Descriptive Statistics for Early Numeracy Measures in the Standardization Sample, Kindergarten and Grade 1 Table 6 Descriptive Statistics for Early Literacy Measures in the Standardization Sample, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 6 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 7 Descriptive Statistics for Math Measures in the Standardization Sample, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 7 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 8 Descriptive Statistics for Reading Measures in the Standardization Sample, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 8 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Norm Sample Descriptives Tables 9 through 12 provide norm sample sizes, means, and standard deviations by season and grade for all of the aimswebPlus reading and math measures. Scores for each measure are reported on the same scale described in the Standardization Sample Descriptives section of this manual. Table 13 presents the total score means and corresponding Quantile® scores for Concepts & Applications, by grade and season. Similarly, Table 14 presents the words read correctly means and corresponding Lexile® levels for Oral Reading Fluency, by grade and season. Quantile and Lexile scores represent MetaMetrics’s proprietary developmental math and reading scales, respectively, that span Kindergarten through Grade 12. These scores were obtained as part of extensive linking studies conducted by MetaMetrics (see Linking aimswebPlus Concepts & Applications (Grades 2–8) With the Quantile® Framework for Mathematics and Linking aimswebPlus Oral Reading Fluency (Grade 1) With the Lexile® Framework for Reading for descriptions of each study). As previously discussed, the norm samples are based on a resampling method used to improve representation of certain student and school demographics as compared to U.S. census data. Note that the results shown in the following tables are based on norm sample students with valid scores in all three testing seasons, while dashed lines indicate that a given measure is not administered in a particular grade or season. Table 9 Descriptive Statistics for Early Numeracy Measures and Composites in the Norm Sample, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 9 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 10 Descriptive Statistics for Early Literacy Measures and Composites in the Norm Sample, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 10 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 11 Descriptive Statistics for Math Measures and Composites in the Norm Sample, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 11 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 12 Descriptive Statistics for Reading Measures and Composites in the Norm Sample, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 12 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 13 Concepts & Applications Total Score Means and Quantiles, by Grade and Season Table 14 Oral Reading Fluency Words Read Correctly (WRC) Means and Lexiles, by Grade and Season aimswebPlus For more information visit aimswebPlus.com | 13 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Reliability Tables 16 through 44 present test reliability results for each aimswebPlus measure. Two types of reliability are reported: alternate form and internal consistency. Alternate form reliability is reported for all timed measures, while internal consistency reliability is reported for untimed measures. Reliability is an estimate of the consistency or stability of test scores. Consistency is affected by random error (which can be caused by many factors including variations in student motivation and attentiveness), imperfect and incomplete specification of the achievement domain, and guessing. The choice of reliability method depends on how the test is administered and scored, as well as how the results will be used. For untimed tests that assess student achievement at a single point in time, internal consistency reliability is most appropriate. Among the various internal consistency methods, Cronbach’s alpha is the most commonly utilized and it is the one reported for all aimswebPlus untimed measures. Note that for untimed measures, items that were skipped/unanswered were scored as zero. To be included in the analysis, a minimum of five valid item scores were required for any given measure. This number of items was chosen because the administration guidelines for standardization testing indicating that testing should be discontinued if the student failed each of the first five items of a given measure. This occurred, on average, during about 1% of test administrations. Cronbach’s alpha is not appropriate for aimswebPlus timed measures because this type of reliability requires a score on all items in a given measure. The time limits used for aimswebPlus fluency measures are designed to provide strong reliability and growth sensitivity; however, these time limits also have the effect of ensuring that most students will not complete all of the items in a given measure. As such, alternate form reliability is most appropriate for aimswebPlus timed measures. Another important reason for using alternate form reliability for these measures is how scores from the timed measures are used. aimswebPlus timed measures are used for benchmark screening and for frequent (e.g., weekly) monitoring of student progress. The timed measures have either 12 or 23 alternate forms for each grade, depending on benchmark seasons administered. Two (fall/winter or winter/spring) or three (fall/winter/spring) of the forms are used for universal screening, with the remaining 10 or 20 used for progress monitoring. All alternate forms for each measure were constructed from a common test blueprint and are nearly equivalent in difficulty. Progress monitoring scores are used to estimate rate of growth and to determine whether that rate is sufficient to meet the performance goal set for a student. Therefore, it is important to know how variations in test content and occasion affect score consistency. Alternate form reliability is designed for that purpose. aimswebPlus uses composite scores (sums of scores from two or more measures) to determine risk classification. A measure’s influence on the composite depends on the magnitude of its variance relative to the variances of the other tests in the composite. The greater the variance, the greater its influence on the composite. In order to equalize the contribution of each test to the overall composite, a weighting method was used. However, because this process can be perceived as complicating the interpretation of scores, weighting was applied only when a measure’s variance was greater than twice the variance of the other measures in the composite. aimswebPlus For more information visit aimswebPlus.com | 14 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. To equalize the contribution of each test in a composite, the total score for a measure was either multiplied by 1.0 or by a fractional weight (see Table 15). An exception was made for the Grade 1 Early Literacy Fall composite because ORF is such a strong predictor of end-of-year reading performance. Table 15 Benchmark Composite Scoring Rules, by Subject, Grade, and Season Subject Grade Season Early Literacy K W, S Early Literacy 1 F Reading 2–3 F, W, S (1/2*ORF) + VO + RC Reading 4–8 F, W, S (1/2*SRF) + VO + RC Early Numeracy K F Early Numeracy K W, S Early Numeracy 1 F Early Numeracy 1 W, S 2–8 F, W, S Math Composite LNF + LWSF + PS LWSF + ORF (1/3*NNF) + QTF + CA (1/3*NNF) + QTF + CA + QDF NCF–P + MFF–1D + CA NCF–P + MFF–1D + CA + MFF–T (NCF–T + MCF) + CA Composite reliabilities are based on Feldt & Brennan’s (1989) stratified alpha method. Stratified alpha uses observed reliabilities and variances for each measure contributing to the composite to estimate the error variance of the composite. Using this method, reliability is computed as: 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝛼𝛼 = 1 − ∑ 𝜎𝜎𝑖𝑖2 (1−∝𝑖𝑖 ) 𝜎𝜎𝑥𝑥2 Where i is a component (i.e., measure) in the composite, α is the reliability of each component, and the denominator is the total composite variance. Note that sample sizes are not shown in the tables reporting stratified alpha values because the individual measure reliabilities come from different studies with varying sample sizes. As such, no single sample size is appropriate. Reliability results are presented in table organized by domain: Early Literacy, Early Numeracy, Reading, and Math. Reliability coefficients are provided for each measure, season, and grade within these domains. Cronbach’s alpha coefficients were derived using all standardization cases with valid test scores for a given season. Alternate form reliability coefficients were derived from data collected in separate equivalency studies. These alternate form equivalency studies are briefly described below, followed by the tables reporting reliability for each measure, grade, and season. aimswebPlus For more information visit aimswebPlus.com | 15 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Early Numeracy Equivalency Studies NNF, QTF, and QDF Alternate form reliability data of the Fall, Winter, and Spring benchmark forms were gathered as part of the national item calibration field test study. For this study, four alternate forms each of NNF and QTF and three alternate forms of QDF were evaluated, with all alternate forms developed from a common blueprint, summarized as follows: • • • NNF: Each form consisted of 80 Arabic numerals, ranging from 0 to 20. The total score equaled the number of numerals correctly named in 1 minute. QTF: Each form consisted of 38 items presenting a box or a pair of boxes containing dots. Dots were arranged like the dots on dice, with up to six dots in each box and the maximum total number of dots displayed per item was 10. The student indicated the total number of dots for each item. The total score equaled the number of items answered correctly in 1 minute. QDF: Each form consisted of 24 items presenting two boxes containing dots, one with blue dots and one with red dots. Dots were arranged like the dots on dice, with the blue dots ranging between 1 and 5 and red dots ranging from 2 to 6. For each item, the box with blue dots always contained fewer dots than the box with red dots. The student indicated how many more blue dots were needed to match the number of red dots for each item. The total score equaled the number of items answered correctly in 1 minute. For this study, six test sets were used. Each set consisted of two NNF, two QTF, and two QDF forms, as well as 25 Concepts & Applications items. The order of measures for each set was: NNF(1), QTF(1), QDF(1), CA, NNF(2), QTF(2), QDF(2). Each fluency measure was assigned to two test sets in counterbalanced sequence such that if a fluency form (e.g., NNF) appeared before CA in the first set, then it appeared after CA in the second set. A spiraling approach was used to assign students to test sets. In total, 635 students completed all seven test forms in all the sets. With approximately 105 students completing each set, this resulted in about 210 students completed each of the four alternate forms per fluency measure. Note that an administration error occurred with two of the QTF forms, which resulted in a loss of about 100 cases. NCF–P, MFF–1D, and MFF–T Alternate form reliability data of the Fall, Winter, and Spring benchmark forms were gathered as part of the national item calibration field test study. For this study, four alternate forms each of NCF–P and MFF–1D and three alternate forms of MFF–T were evaluated, with all alternate forms developed from a common blueprint, summarized as follows: • NCF–P: Each form consisted of 50 pairs of Arabic numerals, with numbers ranging from 0 to 99. The total score equaled the number of items answered correctly in 1 minute. • MFF–1D: Each form consisted of 40 addition and subtraction problems involving numbers 0 through 10. The total score equaled the number of items answered correctly in 1 minute. • MFF–T: Each form consisted of 32 items involving the addition and subtraction of 10. The total score equaled the number of items answered correctly in 1 minute. aimswebPlus For more information visit aimswebPlus.com | 16 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. For this study, six test sets were used. Each set consisted of two NCF–P, two MFF–1D, and two MFF–T forms, as well as 25 Concepts & Applications items. The order of measures for each set was: NCF–P(1), MFF–1D(1), MFF–T(1), CA, NCF–P(2), MFF–1D(2), MFF–T(2). Each fluency measure was assigned to two test sets in counter-balanced sequence such that if a fluency form (e.g., NCF–P) appeared before CA in the first set, then it appeared after CA in the second set. A spiraling approach was used to assign students to test sets. In total, 606 students completed all seven test forms in all the sets. With approximately 100 students completing each set, this resulted in about 200 students completed each of the four alternate forms per fluency measure. Early Literacy Equivalency Studies LWSF In the Winter testing season, 536 Kindergarten students completed one set of four alternate LWSF forms. Each of the 10 sets included the Grade 1 Fall LWSF benchmark form as the anchor form, with the remaining three forms per set being drawn from the 14 alternate forms developed for LWSF. Note that the Grade 1 Fall LSWF benchmark form was developed from the same blueprint used in the Winter and Spring of Kindergarten. Each group of three alternate forms was assigned to two of the ten equivalency study sets, with the order of the first and third forms reversed across the sets. In each set, the anchor form was always administered first. This approach was used to control for order effects and sampling variation. Approximately 50 students completed each LWSF set. The correlation of the scores from the anchor form and the alternate forms was used to estimate reliability. The coefficient reported was computed from the weighted mean of the Fisher’s z-transformed correlation coefficients. LNF Alternate form reliability for this measure was computed from Fall, Winter, and Spring LNF benchmark scores obtained during the 2007–2008 school year. Due to the 4-month interval between benchmark administrations (i.e., fall to winter, winter to spring), the correlation coefficients represent lower bound estimates of reliability. WRF To assess the equivalency of the six WRF forms, an equivalency study was conducted in which each form was assigned to two sets and each set comprised three forms. The order of forms was counterbalanced such that if a form appeared in the first position in one set, then it appeared in the third position in another set and vice versa. Forms assigned to the second position were assigned to that position in both sets it appeared in. This approach was used to control for order effects and sampling variation. For this study, 355 Grade 1 students completed three forms during the Spring testing window. ORF Alternate form reliability coefficients for ORF were derived from benchmark data obtained during standardization. In each season, Grade 1 students read two passages aloud, each for 1 minute. The Spearman-Brown Prophecy formula was used to estimate reliability of the mean reading rate from the correlation of reading rates for the two passages. aimswebPlus For more information visit aimswebPlus.com | 17 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Math Equivalency Studies NCF–T and MCF Alternate form reliability data of the three NCF–T and MCF benchmark forms was obtained as part of a larger study of the equivalency of NCF–T and MCF progress monitoring forms. For this study, students in Grades 2 through 8 completed a set of three NCF–T forms and three MCF forms. Fifteen sets were used for this study, with each set randomly assigned to students by spiraling sets within grade at each testing site. Sets 13 through 15 each contained all three benchmark forms, with the order of the forms completely counterbalanced across these three sets to control for order effects and sampling variation. NSF Number Sense Fluency (NSF) is a composite derived from the sum of NCF–T and MCF scores. As such, NSF alternate form reliabilities are based on this sum and include only students who had a valid score on both NCF–T and MCF. The NSF score is the basis for all progress monitoring decisions. Reading Equivalency Studies ORF Alternate form reliability coefficients for ORF were derived from benchmark data obtained during standardization. In each season, Grade 2 through 8 students read two passages aloud, each for 1 minute. The Spearman-Brown Prophecy formula was used to estimate reliability of the mean reading rate from the correlation of reading rates for the two passages. Results of single story reliabilities are shown in Table 20. SRF Silent Reading Fluency reading rates are based on the median rate from three stories. Because there is no formulaic approach to estimate the reliability of a median, a simulation study was conducted. For this study, 10 replications of 1,000 cases per grade were simulated, using the observed correlations between pairs of stories for each grade and benchmark period and variance. Six scores were simulated for each student using the MVTNORM package in R. The median score on variables 1 to 3 was then correlated with the median score on variables 4 to 6 to yield the alternate-form reliability of the median of three stories. The average pairwise correlation of reading rates among single stories in SRF across Grades 4 through 8 is 0.75 (see Table 21), while the average reliability of the median of three stories is 0.87 (see Table 22). aimswebPlus For more information visit aimswebPlus.com | 18 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 16 Reliability of Early Numeracy Measures and Composites, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 19 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 17 Reliability of Early Literacy Measures and Composites, Kindergarten and Grade 1 aimswebPlus For more information visit aimswebPlus.com | 20 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 18 Reliability of Math Measures and Composites, Grades 2 Through 8 aimswebPlus For more information visit aimswebPlus.com | 21 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 19 Reliability of Reading Measures and Composites, Grades 2 Through 8 Table 20 Average Alternate-Form Reliability of Single ORF Stories, by Grade and Season aimswebPlus For more information visit aimswebPlus.com | 22 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 21 Average Alternate-Form Reliability of Single SRF Stories, by Grade and Season Table 22 Reliability of the Median of Three SRF Story Reading Rates, by Grade Equivalency of Progress Monitoring Forms Tables 23 through 44 report the average difficulty of each progress monitoring form obtained from data collected during the equivalency studies described above. Each table includes the measure abbreviation, grade level, the progress monitoring form number that is used in the aimswebPlus system, the sample size of students taking each form in the equivalency study, the mean score as a measure of difficulty, the standard deviation (SD), and the effect size (ES). For each form, the reported effect size is the standard deviation unit difference between the form’s mean and the overall mean presented below the table. This method is computed as follows: (𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 − 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀) 𝑆𝑆𝑆𝑆 Where the SD is the average SD reported below each measure’s table. Each table also reports the percentage of variation in test scores attributed to form. This percentage is the ratio of the variance of the means divided by the total score variance. The ratio is multiplied by 100 to generate the reported percentage provided in each table. aimswebPlus For more information visit aimswebPlus.com | 23 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. For all measures, test form variance accounts for less than 5% of the total variance and most are less than 1%. Most effect sizes are less than 0.1 and nearly all are less than 0.3, which is the commonly used threshold indicating a small effect. Table 23 Statistical Equivalency of Progress Monitoring Forms, Number Naming Fluency aimswebPlus For more information visit aimswebPlus.com | 24 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 24 Statistical Equivalency of Progress Monitoring Forms, Quantity Total Fluency aimswebPlus For more information visit aimswebPlus.com | 25 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 25 Statistical Equivalency of Progress Monitoring Forms, Quantity Difference Fluency aimswebPlus For more information visit aimswebPlus.com | 26 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 26 Statistical Equivalency of Progress Monitoring Forms, Number Comparison Fluency–Pairs aimswebPlus For more information visit aimswebPlus.com | 27 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 27 Statistical Equivalency of Progress Monitoring Forms, Math Facts Fluency–1 Digit aimswebPlus For more information visit aimswebPlus.com | 28 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 28 Statistical Equivalency of Progress Monitoring Forms, Math Facts Fluency–Tens Table 29 Statistical Equivalency of Progress Monitoring Forms, Letter Word Sounds Fluency aimswebPlus For more information visit aimswebPlus.com | 29 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 30 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 1) aimswebPlus For more information visit aimswebPlus.com | 30 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 31 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 2) aimswebPlus For more information visit aimswebPlus.com | 31 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 32 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 3) aimswebPlus For more information visit aimswebPlus.com | 32 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 33 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 4) aimswebPlus For more information visit aimswebPlus.com | 33 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 34 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 5) aimswebPlus For more information visit aimswebPlus.com | 34 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 35 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 6) aimswebPlus For more information visit aimswebPlus.com | 35 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 36 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 7) aimswebPlus For more information visit aimswebPlus.com | 36 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 37 Statistical Equivalency of Progress Monitoring Forms, Oral Reading Fluency (Grade 8) aimswebPlus For more information visit aimswebPlus.com | 37 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 38 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 2) aimswebPlus For more information visit aimswebPlus.com | 38 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 39 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 3) aimswebPlus For more information visit aimswebPlus.com | 39 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 40 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 4) aimswebPlus For more information visit aimswebPlus.com | 40 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 41 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 5) aimswebPlus For more information visit aimswebPlus.com | 41 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 42 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 6) aimswebPlus For more information visit aimswebPlus.com | 42 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 43 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 7) aimswebPlus For more information visit aimswebPlus.com | 43 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 44 Statistical Equivalency of Progress Monitoring Forms, Number Sense Fluency (Grade 8) aimswebPlus For more information visit aimswebPlus.com | 44 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Validity During the 2013–2014 standardization study, Pearson obtained achievement scores for participating students from other reading and math tests used by each school. As a condition of participation, schools provided spring test scores from interim assessments, state NCLB tests or other formative assessments. A secure file transfer protocol was used to share data, with test scores being provided to Pearson without individually identifiable information. A unique, randomly derived student ID assigned by Pearson was used to match each participant’s scores to standardization data. This section presents the concurrent and predictive validity coefficients obtained from these data from criterion measures and aimswebPlus. Concurrent validity represents the correlation of aimswebPlus composite scores and criterion measure scores, both from the Spring testing season. Predictive validity represents the correlation of Fall aimswebPlus composite scores and Spring scores from the criterion measures. Predicting student achievement in the Spring from Fall benchmark scores is the basis for determining a student’s risk status. The National Center on Intensive Intervention (NCII) requires predictive validity coefficients of 0.70 or higher to obtain the maximum rating (i.e., providing convincing evidence) for screeners. However, there is not a single universally accepted standard for defining success and many different tests are used across U.S. schools; thus, it is important to evaluate predictive validity with several criterion measures. When a test shows strong prediction with several different criterion measures, there is greater confidence that results can be generalized to other standardized and validated measures of student achievement. In the sections that follow, concurrent and predictive validity coefficients for aimswebPlus Early Numeracy, Early Literacy, Math, and Reading benchmark composites are provided. Each validity table presented shows the unadjusted and adjusted validity coefficients, as well as the mean adjusted coefficients by grade. The adjusted coefficients represent an estimate of the true population coefficient, which takes into account the effects that variation of sample characteristics has on the score variance of the predictor. All things being equal, an increase in score variance will result in larger coefficients. As such, the adjusted validity coefficient is a more accurate estimate of the true population coefficient. This adjustment is computed as: 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝑟𝑟𝑥𝑥𝑥𝑥 𝑟𝑟𝑥𝑥𝑥𝑥 𝑆𝑆𝑆𝑆𝑢𝑢 𝑆𝑆𝑆𝑆𝑢𝑢2 2 2 � =� � / 1 − 𝑟𝑟𝑥𝑥𝑥𝑥 + 𝑟𝑟𝑥𝑥𝑥𝑥 ( 2 ) 𝑆𝑆𝑆𝑆𝑟𝑟 𝑆𝑆𝑆𝑆𝑟𝑟 Where SDu is the population standard deviation of the aimswebPlus composite from the national norms, and SDr is the sample standard deviation of the aimswebPlus composite. The average adjusted validity coefficient is the mean of the adjusted validity coefficients, by grade. The mean is weighted by the sample size of each coefficient. aimswebPlus For more information visit aimswebPlus.com | 45 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Early Numeracy Criterion Validity Table 45 shows the predictive validity coefficients of the aimswebPlus Early Numeracy composite scores with the Tennessee Comprehensive Achievement Program (TCAP) math scores. TCAP assesses math skills aligned to Tennessee’s state learning standards. The characteristics of the sample upon which the coefficient was obtained are also provided. Table 46 shows the concurrent validity coefficients for the aimswebPlus Early Numeracy composites with TCAP math scores. The aimswebPlus Early Numeracy scores were collected in May 2014, while TCAP scores were obtained in late April 2014. Table 45 Early Numeracy Composite Score Predictive Validity Coefficients, by Grade, Season, and Criterion Measure Table 46 Early Numeracy Spring Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure Early Literacy Criterion Validity An important outcome of Kindergarten early literacy instruction is to move students from elementary phonological awareness, such as letter identification and letter sounds, to word reading and eventually to reading connected text in the form of sentences and short stories. Thus, the aimswebPlus measure Word Reading Fluency is used as the predictive criterion measure of Fall and Winter Kindergarten scores. Word Reading Fluency assesses a student’s automaticity with reading high frequency and highly decodable words. Students are given 1 minute to read as many words as possible. In the Fall testing season of Kindergarten, aimswebPlus requires only Letter Naming Fluency for assessing risk status. This measure was selected because research shows it to be a strong predictor of end-of-year oral reading fluency ability (Clemens et al., 2015) and because it is a very appropriate measure of foundational reading skills in the beginning Kindergarten. By midyear, Kindergarten students typically have had formal instruction on letter identification, letters sounds, and parsing simple words into phonemes. As such, the aimwebPlus Early Literacy Winter composite for Kindergarten also includes Letter Word Sounds Fluency and Phoneme Segmentation. The composite of these three measures is used to identify risk and predict end-of-grade performance on Word Reading Fluency. aimswebPlus For more information visit aimswebPlus.com | 46 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. In Grade 1, early literacy instruction continues with a greater emphasis on word reading, as well as reading and comprehending connected text. For Grade 1 students, Oral Reading Fluency has been shown to provide strong prediction of end-of-grade performance on broad measures of reading. The Iowa Test of Basic Skills Level 6 measures vocabulary, word reading, and reading comprehension at the end of Grade 1, making it an appropriate criterion measure for ORF. Table 47 shows the unadjusted and adjusted predictive validity coefficients of aimswebPlus LNF (Kindergarten, Fall), the composite comprised of LNF, LWSF, and PSF (Kindergarten, Winter), and ORF (Grade 1, Fall). The characteristics of the sample upon which the coefficient was obtained are also provided. Because WRF was administered to all Kindergarten students in the Spring testing season, data from this measure were used to obtain the validity coefficient. Table 48 shows the concurrent validity coefficients for the composite comprised of LNF, LWSF, and PSF (Kindergarten, Spring) and ORF (Grade 1, Spring). ITBS scores were obtained in April 2014. Table 47 Early Literacy Predictive Validity Coefficients, by Grade, Season, and Criterion Measure Table 48 Early Literacy Concurrent Validity Coefficients, by Grade and Criterion Measure Math Criterion Validity Five criterion measures were used to calculate criterion validity for aimswebPlus Math: • • • • • Iowa Tests of Basic Skills®–Total Math (ITBS®) Illinois Standards Achievement Test (ISAT) New Mexico Standards Based Assessment (NMSBA) Northwest Evaluation Association Measures of Academic Progress® (NWEA–MAP®) State of Texas Academic Assessment of Readiness (STAAR) The ITBS is a comprehensive, group-administered, paper-based assessment of reading and math achievement. ITBS’s Total Math score reflects performance on standards-based math concepts, problem solving, and computation. The ISAT is the end-of-year achievement test assessing Illinois learning standards covering five math strands: Number Sense, Measurement, Algebra, Geometry, and Data Analysis and Probability. The NMSBA is used to measure student proficiency on New Mexico’s reading and math learning standards. NWEA–MAP is a computer-adaptive test that assesses achievement in reading and aimswebPlus For more information visit aimswebPlus.com | 47 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. mathematics. Results are reported on an RIT scale, which is then linked to each state’s performance standards. Finally, the STAAR assesses student performance on Texas’s mathematics and reading learning standards. Table 49 shows the predictive validity coefficients of the aimswebPlus Math composite with each criterion measure. Weighted mean validity coefficients, by grade, are also shown, which provides an estimate of the overall predictive validity. The characteristics of the sample upon which the coefficient was obtained are also provided. Table 50 shows the concurrent validity coefficients for the aimswebPlus Math composite with each criterion measure, as well as the mean adjusted coefficients by grade. aimswebPlus Math scores were collected in May 2014, while the criterion measures scores were obtained in March through May 2014. Table 49 Math Composite Score Predictive Validity Coefficients, by Grade and Criterion Measure aimswebPlus For more information visit aimswebPlus.com | 48 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 50 Math Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure Reading Criterion Validity Four criterion measures were used to calculate criterion validity for aimswebPlus Reading: • • • • Illinois Standards Achievement Test (ISAT) Missouri Assessment Program Grade Level Assessment (MAP–GLA) Northwest Evaluation Association Measures of Academic Progress (NWEA–MAP) State of Texas Academic Assessment of Readiness (STAAR) The ISAT is the end-of-year achievement test assessing Illinois learning standards, including reading comprehension. The MAP–GLA is the end-of-year achievement test that assesses Missouri reading and math standards, including reading comprehension. NWEA–MAP is a computer-adaptive test that assesses achievement in reading and mathematics. Results are reported on an RIT scale, which is then linked to each state’s performance standards. Finally, the STAAR assesses student performance on Texas’s mathematics and reading learning standards. Table 51 shows the predictive validity coefficients of the aimswebPlus Reading composite with each criterion measure. Weighted mean validity coefficients, by grade, are also shown, which provides an estimate of the overall predictive validity. The characteristics of the sample upon which the coefficient was obtained are also provided. Table 52 shows the concurrent validity coefficients for the aimswebPlus Reading composite with each criterion measure, as well as the mean adjusted coefficients by grade. aimswebPlus Math scores were collected in May 2014, while the criterion measures scores were obtained in March through May 2014. aimswebPlus For more information visit aimswebPlus.com | 49 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 51 Reading Composite Score Predictive Validity Coefficients, by Grade and Criterion Measure Table 52 Reading Composite Score Concurrent Validity Coefficients, by Grade and Criterion Measure aimswebPlus For more information visit aimswebPlus.com | 50 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Classification Accuracy Educators want to know how well scores collected in the fall identify who is at risk of not attaining proficiency in the spring, so that they can provide those students with the resources and interventions to improve learning, close achievement gaps, and ultimately move them to proficiency by the end of the school year. Classification accuracy is a way to quantify how accurately scores on one test predict scores on a different, criterion test. More specifically, it refers to how accurately the predictor test classifies students as proficient or not proficient, according to the criterion test. In this section, classification accuracy results, based on the same data used for predictive validity, are provided. Classification accuracy is an alternative means of expressing criterion validity that is appropriate when there is interest in predicting a dichotomous criterion (e.g., passing or not passing an end-of-year state test). A cut score on the predictor test (in this case, a given aimswebPlus measure) is chosen such that those who score at or above the cut score are considered likely to pass the criterion, while those who score below the cut score are likely to fail. A classification accuracy analysis indicates how frequently these expectations prove correct, and the results are reported in a variety of statistics. Table 53 shows a two-by-two classification table. The columns indicate classification of proficiency based on the criterion (e.g., spring reading achievement test), and the rows indicate classification of proficiency based on the predictor (e.g., fall or winter aimswebPlus Reading composite). The four possible outcomes listed (TP, FP, FN, and TN) are defined as follows: • • • • TP is a true positive, meaning a student who passed the test was correctly predicted to pass. FP is a false positive, meaning a student who failed the test was incorrectly predicted to pass. FN is a false negative, meaning a student who passed the test was incorrectly predicted to fail. TN is a true negative, meaning a student who failed the test was correctly predicted to fail. Table 53 Classification Accuracy, Two-by-Two Model Criterion Proficiency Predictor Proficiency Yes (positive) No (negative) Row totals Yes TP FP R1 No FN TN R2 Column totals P N Total From each of these four prediction outcomes, several statistics can be derived and used to evaluate the accuracy of prediction. Table 54 lists the various classification accuracy statistics reported for aimswebPlus Reading and Math composite scores. One key statistic is the overall accuracy rates, representing the percentage of students correctly classified by the predictor. This statistic directly answers the question of how accurately a test score classifies a student; however, overall accuracy rates depend on other statistics, such as base rate and the cut score chosen for the predictor, and even small changes in these values can significantly change overall accuracy rates. aimswebPlus For more information visit aimswebPlus.com | 51 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Another statistic, known as the area under the curve (AUC), does not depend on base rates and cut scores; as such, this statistic can be used to compare the predictive accuracy of different predictors. AUC represents the total area under a curve formed from the relationship between the false positive rate and the true positive rate at each point from 0 to 1.0. (Note that AUC cannot be described with a simple formula.) AUCs greater than or equal to 0.85 are considered strong evidence of classification accuracy by the National Center on Intensive Intervention. Table 54 Classification Accuracy Statistics Statistic Formula False positive rate FP ÷ N False negative rate FN ÷ P Sensitivity TP ÷ P Specificity TN ÷ N Positive predictive power TP ÷ R1 Negative predictive power TN ÷ R2 Overall accuracy rate (TP + TN) ÷ Total Base rate N ÷ Total Early Numeracy Classification Accuracy This section describes the classification accuracy of the aimswebPlus Fall and Winter Early Numeracy composite scores in Kindergarten and Grade 1 with Tennessee Comprehensive Achievement Program (TCAP) performance in the spring of 2014. TCAP assesses math and reading skills aligned to Tennessee’s state learning standards. Because TCAP does not report proficiency levels below Grade 3, a cut score was defined such that the proficiency rate would approximate the proficiency rate observed in Grade 3, which was approximately 40% of students. Using aimswebPlus national percentiles to approximate this rate, the 40th national percentile was selected. Students scoring below the 40th national percentile were considered not proficient. Classification accuracy results are shown in Table 55, by grade level. The base rate indicates the percentage of students not proficient. Using the criterion described above, 35% of students were not proficient in Kindergarten and 16% were not proficient in Grade 1. The relatively low base rate observed in Grade 1 indicates that the overall ability of the sample was above average. The overall classification accuracy rates range from 81% to 100%. The AUC is also very high, ranging from 0.90 to 1.00. aimswebPlus For more information visit aimswebPlus.com | 52 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 55 Classification Accuracy of Early Numeracy Composite Scores and TCAP Early Literacy Classification Accuracy This section describes the classification accuracy of the aimswebPlus Fall and Winter Early Literacy composite scores in Kindergarten, as well as for Fall Grade 1 Oral Reading Fluency (ORF) scores. In Kindergarten, two criterion measures were used: Spring Word Reading Fluency (WRF) scores and Spring R–CBM scores. WRF is a new aimswebPlus word reading CBM, while R–CBM is the original aimsweb oral reading fluency CBM. In Grade 1, spring scores on the Iowa Test of Basic Skills (ITBS) reading composite were used as the criterion. This total reading composite score includes foundational reading skills, listening comprehension, and reading comprehension. The 25th Spring national percentile on WRF and the 25th Fall national percentile on R–CBM were defined as the criterion cut scores designating proficiency. For ITBS, a grade equivalent score of 1.5 was defined as the criterion cut score designating proficiency. This grade equivalent was chosen because it represents the median performance of students at the end of Grade 1. Tables 56 and 57 show classification accuracy results for Kindergarten. In the Fall testing window, Letter Naming Fluency (LNF) is the predictor; meanwhile, in Winter, the predictor is a composite based on the sum of LNF, Letter Word Sounds Fluency (LWSF), and Phoneme Segmentation (PS) scores. Overall, classification accuracy rates range from 76% to 97%. The AUC is also very high, ranging from 0.82 to 0.99. Table 58 shows results for Grade 1. The overall accuracy rate for Fall ORF scores is 75%, with an AUC of 0.85. aimswebPlus For more information visit aimswebPlus.com | 53 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 56 Classification Accuracy of Fall Letter Naming Fluency with Winter Early Literacy Composite Scores and Word Reading Fluency (Kindergarten) Table 57 Classification Accuracy of Fall Letter Naming Fluency with Winter Early Literacy Composite Scores and R–CBM (Kindergarten) aimswebPlus For more information visit aimswebPlus.com | 54 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 58 Classification Accuracy of Fall Oral Reading Fluency Scores and Spring ITBS (Grade 1) Math Classification Accuracy This section describes the classification accuracy of the aimswebPlus Fall Math composite score for Grades 2 through 8. To extend the generalizability of results, classification accuracy was evaluated with the following five different criterion measures: ITBS, ISAT, NMSBA, NWEA–MAP, and STAAR. Note that these measures are described in the Validity section of this manual. Tables 59 through 62 show the classification accuracy results, by grade level. For the three state accountability criterion assessments (ISAT, STAAR, and NMSBA), spring benchmark performance levels were based on the cut score at or above which a student was designated as proficient in that state assessment system during the 2013–2014 school year, by grade level. Base rates, which range from the mid-0.20s to mid-0.40s, indicate the percentage of students who were not proficient on the state test. The NWEA–MAP math cut scores were based on results provided in the NWEA linking study reports. NWEA conducts linking studies using data from students with MAP scores and state test scores. The linking study aligns NWEA’s Rasch Unit (RIT) scale to the state test scale using equipercentile equating. For each state proficiency level, a RIT cut score is defined. AUC values range from the upper-0.70s to the mid-0.90s. Approximately half of the AUCs exceed 0.85 and 85% exceed 0.80, the threshold for good classification accuracy. aimswebPlus For more information visit aimswebPlus.com | 55 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 59 Classification Accuracy of Math Composite Scores and ITBS (Grade 2) and ISAT (Grades 3–8) Table 60 Classification Accuracy of Math Composite Scores and NWEA–MAP (Grades 2–6) aimswebPlus For more information visit aimswebPlus.com | 56 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 61 Classification Accuracy of Math Composite Scores and STAAR (Grades 3–8) Table 62 Classification Accuracy of Math Composite Scores and NMSBA (Grades 6–8) aimswebPlus For more information visit aimswebPlus.com | 57 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Reading Classification Accuracy This section describes the classification accuracy of the aimswebPlus Fall Reading composite score for Grades 2 through 8. To extend the generalizability of results, classification accuracy was evaluated with the following four different criterion measures: ISAT, MAP–GLA, NWEA–MAP, and STAAR. Note that these measures are described in the Validity section of this manual. Tables 63 through 66 show the classification accuracy results, by grade level. For the three state accountability criterion assessments (ISAT, STAAR, and NMSBA), spring benchmark performance levels were based on the cut score at or above which a student was designated as proficient in that state assessment system during the 2013–2014 school year, by grade level. Base rates, which range from the low-0.20s to mid-0.60s, indicate the percentage of students who were not proficient on the state test. The NWEA–MAP reading cut scores were based on results provided in the NWEA linking study reports. NWEA conducts linking studies using data from students with MAP scores and state test scores. The linking study aligns NWEA’s RIT scale to the state test scale using equipercentile equating. For each state proficiency level, a RIT cut score is defined. AUC values range from the upper-0.70s to the mid-0.90s, with 14 of the 20 reported AUCs exceeding 0.85 and 19 exceeding 0.80, the threshold for good classification accuracy. Table 63 Classification Accuracy of Reading Composites Scores and ISAT (Grades 3–8) aimswebPlus For more information visit aimswebPlus.com | 58 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 64 Classification Accuracy of Reading Composites Scores and NWEA–MAP (Grades 2–6) Table 65 Classification Accuracy of Reading Composites Scores and MAP–GLA (Grades 3–8) aimswebPlus For more information visit aimswebPlus.com | 59 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table 66 Classification Accuracy of Reading Composites Scores and STAAR (Grades 3–5) aimswebPlus For more information visit aimswebPlus.com | 60 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. References Clemens, N.H., Hagan-Burke, S., Luo, W., Cerda, C., Blakely, A., Frosch, J., … Jones, M. (2015). The predictive validity of kindergarten and first-grade reading skills. School Psychology Review, 44(1), 76–97. Feldt, L.S., & Brennan, R.L. (1989). Reliability. In R.L. Linn (Ed.), Educational Measurement (3rd ed., pp. 105–146). New York: Macmillan. Hoover, H. D., Dunbar, S. B., & Frisbie, D. A. (2007). Iowa Tests of Basic Skills. Itasca, IL: Houghton Mifflin Harcourt. Northwest Evaluation Association. Measures of Academic Progress. Portland, OR: Author. U.S. Census Bureau (2013). American community survey, 2013: 1-year period estimates. Washington, DC: Author. aimswebPlus For more information visit aimswebPlus.com | 61 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Appendix aimswebPlus Measures Early Numeracy (Kindergarten and Grade 1) aimswebPlus Early Numeracy comprises the individually administered math measures developed for students in Kindergarten and Grade 1. Note that these measures are also available in Spanish. Table A1 presents the grades and seasons available, tasks, and scoring criteria for these measures, followed by brief descriptions of each measure. Table A1 Early Numeracy Measure Descriptions Measure Number Naming Fluency (NNF) Quantity Total Fluency (QTF) Quantity Difference Fluency (QDF) Concepts & Applications (CA) Number Comparison Fluency–Pairs (NCF–P) Grade Season K F, W, S Verbally name numbers up to 20 for 1 minute. Number of items correctly answered F, W, S Boxes containing blue dots are presented. Students state the total number of dots within each box or each pair of boxes for 1 minute. Number of items correctly answered Pairs of boxes containing dots (one with blue dots, one with red dots) are presented. Students state how many more blue dots are needed to match the number of red dots for 1 minute. Number of items correctly answered K What students do Score K W, S K, 1 F, W, S Mentally solve various types of math problems and state the correct answers. Number of items correctly answered F, W, S Pairs of numbers are presented. Students identify which of two numbers is larger for each pair for 1 minute. Number of items correctly answered Number of items correctly answered Number of items correctly answered 1 Math Facts Fluency–1 Digit (MFF–1D) 1 F, W, S Mentally solve simple addition and subtraction problems involving numbers 0 through 10 and state the correct answers for 1 minute. Math Facts Fluency–Tens (MFF–T) 1 W, S Mentally add or subtract 10 to/from given numbers and state the correct answers for 1 minute. Number Naming Fluency (NNF) • Grade: Kindergarten • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student points to and names visually presented numbers for 1 minute. Each form contains 80 items. • Scoring: 1 point for each correctly named number • Time Limit: 1 minute aimswebPlus For more information visit aimswebPlus.com | 63 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Quantity Total Fluency (QTF) • Grade: Kindergarten • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student states the total number of dots in each box or pair of boxes for 1 minute. Each form contains 38 items. • Scoring: 1 point for each correctly answered item • Time Limit: 1 minute Quantity Difference Fluency (QDF) • Grade: Kindergarten • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The states how many more blue dots are needed to match the number of red dots for each box pair for 1 minute. Each form contains 24 items. • Scoring: 1 point for each correctly answered item • Time Limit: 1 minute Concepts & Applications (CA) • Grades: Kindergarten and Grade 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), untimed • Test Content: The student solves one- and two-step math word problems, each addressing an aspect of grade-appropriate CCSS domains. The examiner reads each item to the student and the student states the correct answer, using the corresponding visual stimulus to solve the problem. The student attempts all 25 items in a given form. • Scoring: 1 point for each correctly answered item • Administration time: 7–12 minutes (approximate) Number Comparison Fluency–Pairs (NCF–P) • Grade: 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student points to and names the larger number in each pair for 1 minute. Each form contains 50 items. • Scoring: 1 point for each correctly answered item • Time Limit: 1 minute Math Facts Fluency–1 Digit (MFF–1D) • Grade: 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student solves addition and subtraction problems involving numbers 0 through 10 for 1 minute. Each form contains 40 items. • Scoring: 1 point for each correctly answered item • Time Limit: 1 minute aimswebPlus For more information visit aimswebPlus.com | 64 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Math Facts Fluency–Tens (MFF–T) • Grade: 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student solves problems involving the addition and subtraction of 10 for 1 minute. Each form contains 32 items. • Scoring: 1 point for each correctly answered item • Time Limit: 1 minute Early Literacy (Kindergarten and Grade 1) aimswebPlus Early Literacy comprises the individually administered reading measures developed for students in Kindergarten and Grade 1. Table A2 presents the grades and seasons available, tasks, and scoring criteria for these measures, followed by brief descriptions of each measure. Table A2 Early Literacy Measure Descriptions Measure Grade Season Print Concepts (PC) K F Letter Naming Fluency (LNF) K What students do Score Show understanding of purpose, use, and contents (letters, pictures) of a book. Number of questions answered correctly F, W, S Say the names of visually presented letters for 1 minute. Number of letters named correctly K F, W Look at four pictures and either point to the one that begins with a given letter sound or make the sound that begins the word. Number of correct letter sounds and picture names K, 1 F, W, S Point to the one of four pictures that matches an orally presented word. Number of pictures chosen correctly Letter Word Sounds Fluency (LWSF) K 1 W, S F Say the sounds of visually presented letters, syllables, and words for 1 minute. Number of sounds or words said correctly Phoneme Segmentation (PS) K 1 W, S F Say the phonemes in orally presented words. Number of phonemes said correctly Word Reading Fluency (WRF) K 1 S F, W, S Read a word list aloud for 1 minute. Number of words read correctly Oral Reading Fluency* (ORF) 1 F, W, S Read two stories aloud, each for 1 minute. Average number of words read correctly Initial Sounds (IS) Auditory Vocabulary (AV) *Note. The ORF information in this table applies to the screening seasons of Fall, Winter, and Spring. When using ORF to progress monitor, students read one story aloud for 1 minute per testing session and the reported score is the number of words read correctly for that single story. aimswebPlus For more information visit aimswebPlus.com | 65 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Print Concepts (PC) • Grade: Kindergarten • Test Format: Individual, student storybook and examiner digital record form (online), untimed • Test Content: The student shows understanding of the purpose, use, and contents (letters, pictures) of a book (specific criteria for selecting appropriate books are provided in the aimswebPlus Early Literacy Administration and Scoring Guide). The student attempts all 9 items. • Scoring: 1 point for each correctly answered item • Administration time: 2–3 minutes (approximate) Letter Naming Fluency (LNF) • Grade: Kindergarten • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student says the names of visually presented letters for 1 minute. Each form contains 100 letters (mix of upper- and lower-case) presented in a student-friendly font. • Scoring: 1 point for each correctly named letter • Time limit: 1 minute Initial Sounds (IS) • Grade: Kindergarten • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), untimed • Test Content: The student looks at four pictures and either points to the one that begins with a given letter sound or makes the sound that begins the word. The student attempts all 12 items in a given form. • Scoring: 1 point for each correctly answered item • Administration time: 2–3 minutes (approximate) Auditory Vocabulary (AV) • Grades: Kindergarten and Grade 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), untimed; all items have four response options • Test Content: The student looks at four pictures and points to the picture that matches an orally presented word. The student attempts all 25 items in a given form. • Scoring: 1 point for each correctly answered item • Administration time: 2–4 minutes (approximate) Letter Word Sounds Fluency (LWSF) • Grades: Kindergarten and Grade 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student says the sounds of visually presented letters, syllables, and words for 1 minute. Each form contains 45 letters and 10 three-letter words. • Scoring: 1 point for each letter or word sound correctly made • Time limit: 1 minute aimswebPlus For more information visit aimswebPlus.com | 66 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Phoneme Segmentation (PS) • Grades: Kindergarten and Grade 1 • Test Format: Individual, examiner digital record form (online), untimed • Test Content: The student says the phonemes of orally presented words that are made up of up to four phonemes. The student attempts all 15 items in a given form. • Scoring: 1 point for each phoneme correctly made • Administration time: 2–3 minutes (approximate) Word Reading Fluency (WRF) • Grades: Kindergarten and Grade 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student reads read words aloud for 1 minute. Each form contains two pages of word lists, totaling 99 words. • Scoring: 1 point for each word correctly read • Time limit: 1 minute Oral Reading Fluency (ORF) • Grade: 1 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed Test Content: The student reads one or two stories aloud, each for 1 minute. Each screening form contains two stories, while each progress monitoring forms contains one story. • Scoring: Mean number of words read correctly in the two stories (screening) or words read correctly in one story (progress monitoring) • Time Limit: 1 minute per story Math (Grades 2–8) aimswebPlus Math comprises the measures developed for students in Grades 2 through 8. Note that these measures are also available in Spanish.Table A3 presents the grades and seasons available, tasks, and scoring criteria for these measures, followed by brief descriptions of each measure. Table A3 Math Measure Descriptions Measure Grade Season What students do Score Number Comparison Fluency–Triads (NCF–T) 2–8 F, W, S Compare three numbers within and across number systems to determine the relative distance between each number for 3 minutes. Mental Computation Fluency (MCF) 2–8 F, W, S Solve multiple-choice math computation problems for 4 minutes. Number of items correctly answered, corrected for guessing* Concepts & Applications (CA) 2–8 F, W, S Solve multiple-choice math word problems. Number of items correctly answered Number of items correctly answered, corrected for guessing* *Note. NCF–T and MCF employ a correction for guessing when calculating the total score. Items not attempted (skipped) and items not reached are ignored in the calculation of the corrected total score. Together, NCF–T and MCF combine into the Number Sense Fluency (NSF) score, which is the simple sum of the NCF–T and MCF corrected scores. This NSF score is the basis for progress monitoring decisions. aimswebPlus For more information visit aimswebPlus.com | 67 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Number Comparison Fluency–Triads (NCF–T) • Grades: 2–8 • Test Format: Group, online, timed • Test Content: The student answers multiple-choice math items, comparing numbers within and across number systems, for 3 minutes. Each item is presented as a triad of numbers, with the student determining whether the top number in the triad is closer in value to the bottom left number, the bottom right number, or exactly between the two numbers. Each form contains 40 items. • Scoring: 1 point for each correctly answered item, total score then adjusted for guessing • Time Limit: 3 minutes Mental Computation Fluency (MCF) • Grades: 2–8 • Test Format: Group, online, timed • Test Content: The student answers multiple-choice math items, each requiring one- or two-step mental computation of a math expression, for 4 minutes. The use of friendly (e.g., round) numbers facilitates the mental computation of answers. Each form contains 42 items. • Scoring: 1 point for each correctly answered item, total score then adjusted for guessing • Time Limit: 4 minutes Concepts & Applications (CA) • Grades: 2–8 • Test Format: Group, online, untimed; audio is available for all students at all grade levels • Test Content: The student answers multiple-choice math word problems, each addressing an aspect of grade-appropriate CCSS domains. Each form contains between 29 and 31 items, depending on grade and season. The student attempts all items in a given form. • Scoring: 1 point for each correctly answered item, total score then converted to a developmental scale score • Administration time: 15–25 minutes (approximate) Reading (Grades 2–8) aimswebPlus Reading comprises the measures developed for students in Grades 2 through 8. Table A4 presents the grades and seasons available, tasks, and scoring criteria for these measures, followed by brief descriptions of each measure. aimswebPlus For more information visit aimswebPlus.com | 68 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Table A4 Reading Measure Descriptions Measure Grade Season What students do Score Vocabulary (VO) 2–8 F, W, S Identify the meanings of target words by selecting from multiplechoice options. Number of items correctly answered Reading Comprehension (RC) 2–8 F, W, S Read six passages of text and answer multiple-choice questions about each passage. Number of items correctly answered Silent Reading Fluency (SRF) 4–8 F, W, S Read three stories divided into brief sections and answer multiple-choice questions about each story. Median reading rate of three stories Oral Reading Fluency* (ORF) 2–8 F, W, S Read two stories aloud, each for 1 minute. Average number of words read correctly *Note. The ORF information in this table applies to the screening seasons of Fall, Winter, and Spring. When using ORF to progress monitor, students read one story aloud for 1 minute per testing session and the reported score is the number of words read correctly for that single story. Vocabulary • Grades: 2–8 • Test Format: Group, online, untimed; audio is available for all students at all grade levels • Test Content: The student answers multiple-choice vocabulary items, choosing the response that best matches the meaning of a target word. Each form contains 16 (Grade 2) or 22 items (Grades 3–8), presented one per screen. The student attempts all items in a given form. • Scoring: 1 point for each correctly answered item • Administration time: 4–7 minutes (approximate) Reading Comprehension (RC) • Grades: 2–8 • Test Format: Group, online, untimed • Test Content: The student reads passages (three literary and three informational) and answers multiple-choice questions about each passage to demonstrate comprehension of the text. The student attempts all 24 items in a given form. • Scoring: 1 point for each correctly answered item • Administration time: 15–25 minutes (approximate) aimswebPlus For more information visit aimswebPlus.com | 69 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Silent Reading Fluency (SRF) • Grades: 4–8 • Test Format: Group, online, untimed • Test Content: The student reads story segments and answers multiple-choice questions about each segment, receiving immediate correct/incorrect feedback after each question before moving on to the next segment and question. The time spent reading each passage is captured to compute the student’s reading rate for each story. Each form contains three stories broken into four segment/question pairs, resulting in 12 questions per form. The student attempts all items in a given form. • Scoring: Median reading rate of three stories, if sufficient comprehension demonstrated (i.e., at least three of four questions correctly answered on at least two stories) • Administration time: 4–6 minutes (approximate) Oral Reading Fluency (ORF) • Grades: 2–8 • Test Format: Individual, student stimulus book (print) and examiner digital record form (online), timed • Test Content: The student reads one or two stories aloud, each for 1 minute. Each screening form contains two stories, while each progress monitoring forms contains one story. • Scoring: Mean number of words read correctly in the two stories (screening) or words read correctly in one story (progress monitoring) • Time limit: 1 minute per story aimswebPlus For more information visit aimswebPlus.com | 70 | Technical Manual Copyright © 2017 NCS Pearson, Inc. All rights reserved. Pearson Executive Office 5601 Green Valley Drive Bloomington, MN 55437 800.627.7271 www.aimsweb.com Copyright ©2017 NCS Pearson, Inc. All rights reserved. Warning: No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the copyright owner. Pearson, aimswebPlus, and the aimswebPlus logo are trademarks in the U.S. and/or other countries, of Pearson Education, Inc., or its affiliate(s). Produced in the United States of America. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.7 Linearized : Yes Author : Clark, Megan R Company : PEARSON Content Type : Document Create Date : 2017:08:01 16:09:22-05:00 Modify Date : 2019:06:17 13:52:58-07:00 Source Modified : D:20170801210812 Subject : Language : EN-US Tagged PDF : Yes XMP Toolkit : Adobe XMP Core 5.2-c001 63.139439, 2010/09/27-13:37:26 Metadata Date : 2019:06:17 13:52:58-07:00 Creator Tool : Acrobat PDFMaker 11 for Word Document ID : uuid:04fe6594-0b53-4976-8793-71d2f2101251 Instance ID : uuid:a9e9d5af-f772-4804-967e-7db8c80019a7 Format : application/pdf Title : aimswebPlus Technical Manual Description : Creator : Clark, Megan R Producer : Adobe PDF Library 11.0 Page Layout : SinglePage Page Count : 78EXIF Metadata provided by EXIF.tools