Report for Detroit Public Schools Community District and Wayne Regional Educational Services Agency Audit of Alignment of Existing Instructional Materials in Mathematics and Recommendations to Consider in Adopting New Materials by Jason Zimba, Ph.D. with Elizabeth Meier January 8, 2017 Contents 1. Overview and Methodology 2. Alignment Findings: Mathematics A. Grades K–6 B. Pre-Algebra C. Algebra 1, Geometry, and Algebra 2 3. Recommendations to Consider in Adopting New Materials Appendices A. Materials Alignment Criteria as Applied to Existing Instructional Materials (Grades K–6) B. Materials Alignment Criteria as Applied to Existing Instructional Materials (Pre-Algebra) C. Materials Alignment Criteria as Applied to Existing Instructional Materials (Algebra 1, Geometry, and Algebra 2) D. Blank Materials Alignment Criteria for Detroit Public School Community District Use in Procurement (Grades K–8) E. Blank Materials Alignment Criteria for Detroit Public School Community District Use in Procurement (High School) 1 1. Overview and Methodology Detroit Public Schools Community District (DPSCD) commissioned David Liben (English Language Arts) and Jason Zimba (Mathematics) to audit instructional materials currently and most commonly used in the district. The purpose of the examination was to evaluate the degree to which instructional materials were or were not aligned to Michigan’s adopted educational standards for English Language Arts and Mathematics. That audit has now been completed, and this report is the audit of the materials in current use for Mathematics. Mathematics instructional materials for Kindergarten through Pre-Algebra were provided by the Office of Mathematics Education of DPSCD. The materials were examined in relation to the alignment criteria listed in the appendices. In outline, the criteria include: Section I. Preliminary Alignment Check Section II. Alignment: Concepts, Fluency, Applications Section III. Alignment: Standards for Mathematical Practice Section IV. Support for All Students Section V. Fit to Your District Each of these sections includes several sub-criteria. The alignment criteria have been adapted to the present purpose from the Instructional Materials Evaluation Tool (IMET),i which is part of an authoritative alignment toolkit applicable to the Michigan mathematics standards. The alignment criteria isolate essential high-level features of the standards, allowing fundamental misalignments to be detected. To launch the project, the authors also participated in one day of classroom walkthroughs at a variety of grade levels. These walkthroughs were not part of a study of teachers or of teaching, but rather provided context for the review of instructional materials. The authors wish to thank the dedicated teachers who opened their classrooms to us. The authors also wish to thank the principals, school and district staff, and Wayne RESA staff who facilitated the walkthroughs and this instructional materials audit. Description of Instructional Materials and Examined Components The mathematics instructional materials examined for Kindergarten through Pre-Algebra consisted of EnVisionMATH Common Core 2012 (briefly: EnVision, but note well the year), Holt Pre-Algebra 2008, and Pearson Algebra 1/Geometry/Algebra 2. Description of EnVision In grades K–2, the school year is structured as a sequence of 16 Topics. Each Topic opens with an Interactive Math Story, a diagnostic (Review What You Know), and a Game (which could be done in school or at home).1 Centers relevant to the Topic are suggested, and teachers could also assign a Math 1 Topic 1 in grade 1 begins with a Readiness assessment. 2 Project for the Topic. After the Topic opening, there is a run of several Lessons. Each Topic then ends with Reteaching resources and a Topic Test, followed by a Performance Assessment. Available assessments for the grade include a Placement Test, Benchmark Tests for each four-Topic period, and an End-of-Year Test. One Lesson in each Topic is a Problem Solving Lesson;2 these Lessons have titles like “Act It Out” or “Use Reasoning.” A Lesson in grades K–2 has standard components that flow in this order:       Common Core Review (skill review in multiple-choice format) Develop the Concept: Interactive (problem-based learning phase) Develop the Concept: Visual (problem-based learning phase bridging into practice) In Kindergarten, an Additional Activity as time permits Close/Assess and Differentiate (using a Quick Check) Resources for differentiation in categories of Intervention, On-Level, and Advanced (for instruction) and Reteaching, Practice, and Enrichment (for homework). In grades 3–6, the school year is structured as a sequence of 16 Topics.3 Each Topic opens with a diagnostic (Review What You Know!). Teachers could also assign a Math Project for the Topic. After the Topic Opener, there is a run of several Lessons. Each Topic then ends with Reteaching resources and a Topic Test, followed by a Performance Task. Available assessments for the grade include a Placement Test, Benchmark Tests for each four-Topic period, and an End-of-Year Test. One or more Lessons in each Topic is a Problem Solving Lesson; these Lessons have titles like “Multiple-Step Problems” or “Work Backward.” A Lesson in grades 3–6 has standard components that flow in this order:         Common Core Review4 (skill review in multiple-choice format) 5 Develop the Concept: Interactive (problem-based learning phase) Develop the Concept: Visual6 (problem-based learning phase bridging into practice) Possibly one or more Other Examples Guided Practice, Independent Practice, and Problem Solving Close Assess (using a Quick Check) Resources for differentiation in categories of Intervention, On-Level, and Advanced (for instruction) and Reteaching, Practice, and Enrichment (for homework). In the Develop the Concept: Interactive phase of a Lesson, the teacher poses the problem, sometimes with manipulatives, grid paper, or other Teaching Tools. The phase begins with a minute or two of Engage (Set the Purpose and Connect to real life or other mathematics). Then comes Pose the Problem followed by several teaching modes, such as Instruct in Small Steps, Model and Demonstrate, Small 2 An exception is Topic 15 in grade 1, which has two Problem Solving Lessons. 19 Topics in grade 6 4 This feature is called “Daily Spiral Review” in grade 6. 5 Students are able to see the Develop the Concept: Interactive problems for each Lesson ahead of time in the Topic overview section of the Student Edition. 6 This is the first page of the Lesson the students see in the Student Edition; in the previous phase (Develop the Concept: Interactive), the teacher poses the problem. 3 3 Group Interaction, Link to Prior Knowledge, Academic Vocabulary, etc. The phase ends with Extend. The teacher-facing materials for this phase include questions to ask and answers to expect. In the Develop the Concept: Visual phase of a Lesson, the teacher instructs with a problem (Visual Learning Bridge) followed by Guided Practice, Independent Practice, and Problem Solving. The teacherfacing materials for this phase include questions to ask, answers to expect, and regular or occasional features such as Prevent Misconceptions, Error Intervention, Reteaching, Test-Taking Tip, and ELL Strategy. The materials emphasize both “Big Ideas” and “Essential Understandings,” part of a widespread approach to curriculum development known as Understanding by Design. The components examined included Teacher Editions, Student Lesson Packets/Student Editions, and Ready-Made Centers for Differentiated Instruction. Teacher Editions include facsimiles of student-facing Lessons as well as teacher-facing content. Topic-level teacher-facing content includes mathematical background, considerations for supporting special populations and below- or above-level students, notes about reading comprehension and vocabulary, notes for facilitating the Interactive Math Story and the Topic Opener, and other material. Lesson-level teacher-facing content includes notes for facilitating Lesson components, facsimiles of assessments and differentiation resources, and other material. Other program components, such as the Math Diagnosis and Intervention System, Math Library, Animated Glossary, Examview Assessment Suite, and other ancillary materials were not examined. It is apparent from the audit that the program provides an expansive array of components for teachers and students. Artwork in student-facing components could sometimes be confusing or misleading, but incorrect answers and typographical and printing errors were rarely observed. Description of Pre-Algebra Each of the 14 Chapters (divided into two Sections each) begins with an opener relating to careers and a diagnostic assessment (Are You Ready?). There follows a run of several Lessons, then a Mid-Chapter Quiz and a Focus on Problem Solving; then a run of several more Lessons. Interspersed among Lessons are such features as Hands-On Labs, Technology Labs, Math-ables (game-like), and Extensions. Chapters close with a Study Guide and Review, Chapter Test, Performance Assessment, and Standardized Test Prep. The standard Lesson flow is as follows:         Warm Up, Problem of the Day, possibly a bit of Math Humor; Introduction by the teacher of the topic at hand; Guided instruction centering on a sequence of example problems; Student Think and Discuss phase; Close; Exercises; Lesson Quiz; Lesson Resources with additional practice and resources for differentiation in the categories of Reteach, Practice, and Challenge. The Teacher Edition was examined for this project. The Teacher Edition includes facsimiles of studentfacing Lessons, assessments and reproducible masters, as well as teacher-facing content, including Warm Up, Problem of the Day, Math Background, Reaching All Learners, Common Error Alerts, Test Prep 4 Doctor, notes for facilitating the Lesson, answers to problems, and other material. Other program components, such as Student Handbook, Homework and Practice Workbook, Spanish Resources, digital Lesson Presentations, Countdown to Testing, and other ancillary materials were not examined. Description of Algebra 1/Geometry/Algebra 2 The program as a whole consists of three yearlong courses.7 Each course unfolds as a sequence of Chapters, numbering from 12 to 14 depending on the course. Each Chapter begins with a diagnostic assessment (Get Ready!), followed by a Common Core Performance Task that opens the Chapter; then a run of several Lessons; then a Mid-Chapter Quiz; then a run of several more Lessons; and finally a section of Assessment and Test Prep consisting of synthesis and review materials, a chapter test, and a cumulative test. There is also an end-of-course assessment. A Lesson has standard components that flow in this order:       Solve It!, a problem that tempts or initiates students into the concepts of the Lesson Guided Instruction Lesson Check Practice (of various kinds) Lesson Quiz Lesson Resources with, among other things, support for special populations plus resources for differentiation in the categories of Intervention, On-Level, and Extension. The materials prominently foreground both “Big Ideas” and “Essential Understandings,” part of a widespread approach to curriculum development known as Understanding by Design. Pedagogically, the program might be described as problem-based (recognizing that this term has many meanings, not all of them compatible with each other). The design is for students to learn the new mathematics of each Lesson by working their way through problems. Students are guided by the teacher who poses each problem and directs the students as they engage with it. There are also features of the written materials (such as the Know-Need-Plan boxes and the Think-Write boxes) that aim to support students’ growth as problem solvers. The components examined for this project included Teacher Editions for the Algebra 1 course (Volume 1 and Volume 2), for the Geometry course (Volume 1 and Volume 2), and for the Algebra 2 course (Volume 1 and Volume 2). The Teacher Editions include facsimiles of student-facing Lessons as well as extensive teacher-facing content, including mathematical background, notes for facilitating Lesson components, assessments, answers to problems, facsimiles of intervention resources, and other material. Other program components, such as Student Companions, Geometry Companion, Skills Handbooks, PowerAlgebra.com, Portable Study Center, Virtual Nerd Videos, and other ancillary materials were not examined. It is apparent from the audit that the program provides an expansive array of components for teachers and students. The physical form and organization of the materials makes them pleasant and efficient to navigate. Incorrect answers or other typographical errors were rarely observed. The diagrams are clean, and the artwork generally adds to the mathematics. The profusion of boldface headings, all-caps, and text icons generally does not. 7 Although the materials don’t explicitly describe the prerequisite relationship between the Geometry course and the two Algebra courses, at various points the mathematics in the materials reveals a traditional sequence in which Geometry follows Algebra 1 and precedes Algebra 2. 5 2. Alignment Findings: Mathematics A. Grades K–6 In this section we present the overall finding for grades K–6, followed by some general themes. Additional details are included in Appendix A. Overall finding: Based on the materials reviewed, the curriculum in grades K–6 is best described as Far from aligned or infeasible to modify to reach alignment. Several themes in alignment emerged from the review. Content Misalignments Although the Topics at each grade are organized according to the content domains of the standards, the program still covers content not required by the standards—for example, combinations, certain kinds of pattern work, and certain statistical topics. In grade 4, a single content standard about patterns is heavily overemphasized by devoting an entire Topic to it. Making matters worse, most of this Topic doesn’t align to the standard cited. At any given grade, the program often includes substantial content that aligns to previous or later grades. In grades 4 and 5, pattern work balloons out to become a substantial study of quantitative relationships, not required by the standards until the middle grades. An already full year in grade 6 is additionally burdened by geometry material that isn’t required in the standards until after grade 6. The misrepresentation of the Kindergarten standards in the materials is especially pronounced: content is taught in Kindergarten that isn’t required until later grades (sometimes much later), and the Kindergarten materials give a misaligned emphasis to sorting, classification, and measurement that draws instructional time away from the number core emphasized in the standards. (Focus on the number core is better in the Kindergarten Center Activities than in the Lessons themselves.) A motivated and expert reader at any given grade level may consult the Standards Skills Trace, but grade-level discrepancies are sometimes obscured in program metadata by creative or incorrect codings to the standards. In some cases, superfluous or off-grade content might be easy to correct for; in other cases, doing so would likely be a complicated undertaking. Problems of Coherence Within a given Topic, the Lessons often address learnable chunks of material in a sensible order—an inestimable virtue in a mathematics curriculum. However, at times the ordering of Lessons or Topics is highly questionable, as for example when distance-time graphs are studied before the coordinate plane itself is, or when area is defined in terms of square units before square units themselves are defined. The Understanding by Design approach does not always succeed. For example, despite the salutary decision to have a Big Idea that “Some attributes of objects are measurable and can be quantified using 6 unit amounts,” the conceptual connection between, say, area measurement and measurement of other quantities remains weak. Moreover, the Essential Understandings for this Big Idea do not include fundamental concepts that are explicit in the standards. Other Essential Understandings can be picayune or procedural. Concepts of multiplication and division are coherently introduced in grade 3, consistent with the standards. There are passages of elegance here. But the program in grades 4, 5, and 6 extends those concepts beyond their initial whole-number meanings only incompletely and inconsistently, creating alignment problems in the progression from arithmetic to algebra.  By grade 5, the standards call for students to be able to think about multiplication as scaling. Multiplication as scaling is indeed addressed in the materials, but not as part of the main content architecture, which positions multiplication as repeated addition all the way into middle school. For example, a grade 5 problem reads: “... If the number of drips from a faucet is 30 per minute, how many drips is this for 10 minutes? Use repeated addition.” In this problem, the grade 5 student is explicitly asked not to think multiplicatively in ways that the grade 5 standards expect.  In a “What You Think” box for the problem 0.36  4, students aren’t to think anything about fractions; nor are they to think about multiplication. They are to think, “Multiplying 0.36  4 is like adding 0.36 four times....” And yet, in the “What You Write” box that follows, students don’t write 0.36 + 0.36 + 0.36 + 0.36; they write 0.36  4 and use the multi-digit multiplication algorithm. The next segment of this Lesson, Another Example, considers the problem 0.5  0.3. Here again students don’t think about fractions, or about multiplication; and since repeated addition suddenly fails as an idea about multiplication, students are asked to think now about shading. The representation is a kind of Venn diagram: the product is the overlap when two regions are shaded. What this has to do with multiplying isn’t told. Instead, it is simply announced that “The product is the area where the shading overlaps.” The representation isn’t being used to reflect mathematical thinking; it’s being used as an analog calculator. (In fact, right next to it is a suggestion to multiply using a real calculator, with pictures showing which buttons to press.) If the additive view of multiplying decimals is problematic in grade 5, it’s even worse in grade 6, where the students are a year older and yet the very same example (0.36  4) is given the very same treatment. Even the artwork for the problem is identical. The standards in grades 4 and 5 closely connect fractions and decimals, grounding decimals in fraction concepts. For example, the first appearance of decimals in the standards isn’t in the multi-digit number domain (which would have been 4.NBT) but instead in the fraction domain (4.NF). Indeed in the standards, the first mention of decimals is in the phrase “decimal notation for fractions.” And by grade 6, the standards unite whole numbers, fractions and decimals under a single heading called The Number System. Yet in the materials examined, whole numbers, fractions, and decimals remain stubbornly separate.  Sometimes this disjointedness is due to missed connections across grades. In grade 5, the materials don’t connect multiplication of a decimal by a whole number to multiplication of a fraction by a whole number, even though multiplication of a fraction by a whole number had been taught in grade 4.  Sometimes, connections within a single grade are missed. In a grade 5 problem, students determine how many 1/4s are in 3, and there is no lookback for teachers or students to a closely related problem in a previous Lesson where the context was money and students determined 7    how many quarters are in $15.50. (Observe that in both cases, dividing a quantity by a fourth multiplies the quantity by four.) Whereas the standards ask students to apply and extend previous whole-number understandings of multiplication to multiply fractions, the materials reviewed portray fraction multiplication as a separate enterprise from whole number multiplication. Even in grade 5, where the standards set a culminating expectation for multiplying fractions, fraction multiplication in the materials still hasn’t become a single thing, but persists as three distinct things: (A) multiplying a whole number by a fraction; (B) multiplying a fraction by a whole number; and (C) multiplying a fraction by a fraction. In teacher-facing material, the area representation of fraction multiplication is explained using an area model for 1/4  2/3. The explanation of the diagram is clearer than anything in the Lessons themselves, but it isn’t pointed out that this explanation generalizes the explanation given for 1/3  6 on the facing page—or that all of the products treated in grade 5 could be placed on such a diagram. It isn’t apparent to teacher or student that the rule for multiplying fractions includes, as a special case, products with one or both factors a whole number. Nor is it clear that the rule for multiplying fractions can explain patterns in decimal products. Cumulatively, the Lessons for grades 4, 5, and 6 present the student with thousands of problems—practically zero of which include both a fraction and a decimal in their formulation. In a grade 5 Algebra Connection, no fractions appear. In grade 5 as a whole, no fractions or decimals get plotted in the coordinate plane. The materials present the properties of multiplication over and over again—in grades 3, 4, 5, and 6—each time as if they were new, and without ever meaningfully generalizing the treatment beyond whole numbers. In grade 6, when consistent with the standards the properties of operations are being presented as algebra per se, the properties are shown entirely with whole numbers. And in the dozens of problems or illustrations that follow, not a single fraction appears (and there is only one problem with decimals). Students aren’t invited to test any properties with fractions or otherwise synthesize their understanding of number. Solving equations in grade 6 isn’t well connected to proportional relationships. For context, note that properties of equality are taught in grade 6, and students at grade 6 apply the properties of equality to solve linear equations. Yet when the context is proportional relationships, the properties of equality are forgotten, and a linear equation like 7/3 = x/15 is never solved by using the properties of equality. Using a ratio table to find the solution is productive at this grade, of course, but so is viewing “proportions” as the equations they are. Limitations in Math Practices One component of the program that foregrounds math practices is the Problem Solving Lessons. These can enrich the way students engage with required content, but they can also import content into the curriculum that isn’t required by the standards. In addition to Problem Solving Lessons, another component of the program that foregrounds math practices is the CC Mathematical Practices tags on individual problems (tags such as Reason or Persevere). Some problems that do reflect math practices don’t have tags, and conversely problems bearing tags aren’t always good instances of math practices—as when Model is attached to a noncontextual problem; or when Reason is attached to a problem about nomenclature; or when Persevere 8 is attached to a perfectly ordinary word problem. The Persevere label may also have the effect of clueing students that a multi-step problem is coming. A teacher-facing note tagged with Critique the Reasoning of Others says to “Ask students what they know about 2/3 and 4/6. [They are equivalent fractions.] Ask So, is her brother correct? [Yes, they are both correct.]” “Yes, they are both correct” is a very low bar for critiquing mathematical reasoning in a grade 5 classroom. Prompts also sometimes ask for explanations in cases where doing so is likely to be sterile. With respect to modeling, the materials at all grades are designed so that teachers and students spend sufficient time working with applications. In grades 3 and above, there are multi-step problems (as called for in the standards), but it does not appear that the program in grades 3 and above systematically combines addition/subtraction situations with multiplication/division situations in multistep problems. Word problem strategy often hinges on key words, which isn’t a good reflection of modeling. (It isn’t key words that tell you to what to do; it’s what’s happening in the situation.) In general the level of modeling and complexity in application problems is sufficient to meet the standards, but the level also sometimes exceeds what is required and sometimes drifts away from the content of the grade entirely. Science connections sometimes present wordy problems without any synergy between math and science (“Coral reefs cover less than 1/500 of the ocean floor, but they contain more than 1/4 of all marine life. Which is a common denominator for 1/500 and 1/4?”). Fluency Gaps The standards’ expectations for fact recall and fluency are that single-digit sums are to be known from memory by end of grade 2, with fluency in related differences expected at the same grade; and that single-digit products are to be known from memory by end of grade 3, with fluency in related quotients expected at the same grade. The program does include a series of Basic-Facts Timed Tests to assess these standards. However, it isn’t clear when to administer these tests or how to keep records on how far individual students in grades 1–3 have progressed toward meeting the fact recall and fluency standards. Basic-Facts Timed Tests are provided in grades 1, 2, 3, and 4. For some or perhaps many students, it is probably sensible to continue practicing single-digit products and related quotients during grade 4, even though the relevant standard is grade 3. However, since the corresponding addition/subtraction standard is in grade 2, it is hard to understand why the Basic-Facts Timed Tests for grades 3 and 4 cover addition/subtraction and multiplication/division equally. In written calculation, the grade 4 materials misalign completely to the capstone standard for multi-digit addition and subtraction, both in messaging in teacher-facing materials and in the student-facing curriculum. The relevant Topic Test in grade 4 does not generate evidence about whether students are fluent in multi-digit addition and subtraction, nor about what their calculation errors might be. More generally, the End-of-Year tests generate little or no evidence about whether capstone standards for multi-digit calculation in any grade have been met. Low support for teachers to conduct student discussions of mathematics 9 The questions suggested for teachers in Instruct in Small Steps are often closed. Instruction is frequently rule-based, and the model of the classroom doesn’t include defined structures for student discourse that could realistically allow students to discuss mathematics guided by the teacher. For example,  After asking students to find the products 3  4, 3  40, and 3  400 using any desired method, the guidance for the share-out is simply, “Have students share how they found the products. Discuss patterns they observed.” Who discusses? How do they discuss? What could the teacher expect to hear in the discussion, and what are the best methods or insights to highlight for the purposes of the Topic? What are the community’s norms for moments like these?  In grade 5, to convert 3/5 to a decimal, students are told to do it by long division. However, previously in the same Lesson, students were also told that 1/10 and 0.1 are the same quantity, and students also rewrote fractions in equivalent form. Therefore what if an idea were to occur to a student that instead of doing long division, one could rewrite 3/5 as an equivalent fraction 6/10 and just observe that 6/10 and 0.6 say the same thing? The program’s instructional design doesn’t take advantage of such plausible “aha” moments, nor does it regularly contrive to produce them.  In one Math Background, the recommendation given to teachers for multiplying mixed numbers reads: "When multiplying mixed numbers, the first step should be to rewrite the mixed numbers as improper fractions. Students should get into the habit of doing this conversion whenever they are multiplying or dividing mixed numbers." This method is efficient, and efficiency is important. But the directive would also foreclose the teacher from making a natural connection between fraction and whole-number problems, as in 25  48 = 20  40 + 20  8 + 5  40 + 5  8 and 3½  2¼ = 3  2 + 3  ¼ + ½  2 + ½  ¼. Meanwhile, an Error Alert says that “...[S]tudents should be careful not to do the conversion to improper fractions when they go back to adding or subtracting mixed numbers. This is a common mistake that even advanced students often make .... Although the addition or subtraction can still be done using the improper fractions, the problem often becomes more complicated than it was intended to be, which can lead to errors.”   Perhaps the advanced students are doing this because it’s correct mathematics. Instead of being encouraging when students do a problem the long way using correct principles, the Error Alert calls it a mistake. This could possibly confuse some students who had thought they understood fractions, and it likely removes one more chance for a mathematical discussion in the classroom. In a Lesson about “Multiplying Two Decimals,” the Objective is that “Students will use the standard algorithm to multiply decimals by decimals.” This is allowed in grade 5, but not required (cf. 5.NBT.B.7 and 6.NS.B.3), and the algorithmic focus decreases the coherence of the Lesson. For example, the distributive property isn’t mentioned in either the student-facing or teacher-facing materials, breaking a natural connection to whole-number multiplication in base ten. For a problem about the distributive property, the teacher-facing note does not signal the algebraic importance of the problem. Instead, the note reads: “Test-Taking Tip: Make a Plan. Remind students that the profit is the sale price minus the cost. How much money was made on the sale of each hat? [$13.35] How many hats were sold? [500] How do you find the total amount of money made? [Multiply 500 by $13.35.]” Test-taking tips can be valuable, and the commentary here could help students get the right answer, which is important. But the teacher isn’t informed that the purpose of the problem is to use structure to (MP.7) to identify the correct expression. (Such problems will regularly appear on tests aligned to the standards.) Nor 10 is the teacher invited to elicit from students a second expression, 500  18.50 – 500  5.15, which models the situation differently (total outlay minus total expenses) and gives the same answer thanks to the distributive property. To be clear, the point of the foregoing isn’t that more methods are always better; indeed in many programs, the trouble is that too many methods are entertained, some of them primitive for the grade level and others doomed never to generalize. (And when teaching an algorithm per se, discussing alternatives is a category error.) The standards are consistent with a range of pedagogies, and the purpose of the foregoing observations isn’t to advance a particular pedagogy under the guise of standards alignment. Rather, the point is to shed light on the level of support the program provides to teachers for the job the program itself asks them to do. Limitations in other supports for teachers Math Background can be clear and helpful, as in this example: “In primary grades, there is a big emphasis on mastering the basic addition facts. This is no small task. ... For many children, it takes hours of practice to reach the point where strategies can be dropped and fact mastery achieved. Other children can do this quickly, and in some cases, may not need to use strategies at all. These children can be introduced to strategies, but should not be required to use them to recall addition facts.” But Math Background can also be incoherent, or incomplete. In grade 1, the Math Background spanning Topics 1 and 2 cumulatively garbles the inverse relationship between addition and subtraction. A grade 4 Math Background about multiplication omits fraction multiplications expected for the first time at this grade; and it omits one of the key developments in grade 4, “times as much” thinking about multiplication, despite its being the focus of a Lesson in the Topic at hand. In a grade 1 Lesson, students discuss how to find the total number of books in three stacks of books displayed by the teacher: the stacks have 2 books, 4 books, and 2 books, respectively. The facilitation note says, “Give children time to discuss possible strategies for adding three numbers. [Add two stacks first, then add the number in the third stack.]” But what should the teacher do if a struggling student prefers to count all 8 books? Teachers are not shown a trajectory from count-all to counting on to property-based strategies like doubles-plus-1 or making ten. ELL supports are frequent in the materials, and they are present at different grain sizes. The suggestions in the Reading Comprehension and Problem Solving notes could also be helpful for ELL students, but they are not positioned as such. As is common in the genre, an intense amount of energy is devoted to differentiating instruction, even though some Intervention/Reteaching resources would likely help most students, and even though some Advanced/Enrichment resources could be fruitful and motivating for more than just the “high kids.” Absent appropriate instructional leadership, differentiation practices could potentially result in the various classrooms within a school enacting somewhat different curricula, or the various schools in a district enacting somewhat different curricula. Other variables to control for might include the fact that if some teachers skip the Extend phase in Develop the Concept: Interactive then this would usually compromise the treatment of the concept; and if time runs short, some teachers might not assign 11 problems in the Problem Solving section, which would pull the treatment of the subject in those classrooms below the standards’ expectations.  There are worthy features in the program under review. One can find well aligned, well made things at various grain-sizes (individual Center Activities, individual Lessons, Individual Topics, and multi-Topic sequences). Deft problems-in-brief as well as worthy longer tasks can also be found. The instructional model opts for efficiency, and efficiency is an important consideration. One should recognize that EnVisionsMATH Common Core 2012 is an extensive set of resources that was published quite soon after the standards were released; given the implied pace of development, the findings presented herein aren’t necessarily surprising. Nevertheless, this program is far from aligned to the standards. B. Pre-Algebra In this section we present the overall finding for pre-Algebra, followed by some discussion relevant to the finding. Additional details are included in Appendix B. Overall finding: Based on the materials reviewed, the curriculum in Pre-Algebra is best described as Far from aligned. A time-capsule of sorts, this program dates from an era before the current state standards were adopted, and it failed the Preliminary Alignment Check. Consistent with the methodology of the rubric, the remaining sections of the rubric were not completed. Unsurprisingly given its vintage, Pre-Algebra has virtually no relationship to the state mathematics standards. For example, the following topics from the state standards for High School all appear in this middle-grades course; such topics cannot but interfere with the study of grade-level mathematics. High School Topic Included in Pre-Algebra Arithmetic and geometric sequences Quadratic, cubic, and exponential functions Polynomials Volume of pyramids, cones, and spheres Permutations and combinations Two-variable inequalities In addition, there are topics present in the course that are not mentioned in the standards at any grade, let alone the middle grades. And the topics in the course that do correspond to the state standards may be found throughout the entire middle-school grade band in the standards—often in a different order within the course than is found in the standards’ grade-level ordering. Finally, some topics from the middle grades standards are not covered in the course. Without even considering such essential questions of alignment as conceptual understanding, Modeling, and other Standards for Mathematical Practice, it is unimaginable that a standards-aligned course could be delivered using this program. 12 C. Algebra 1, Geometry, and Algebra 2 In this section we present the overall finding for Algebra 1, Geometry, and Algebra 2, followed by some general themes. Additional details are included in Appendix C. Overall finding: Based on the materials reviewed, the curriculum in Algebra 1, Algebra 2, and Geometry is best described as Far from aligned or infeasible to modify to reach alignment. Several themes in alignment emerged from the review. Content Misalignments Some topics are covered in the Algebra 2 course but not required by the standards—for example, linear programming and absolute value inequalities. Also in the Algebra 2 course, material on linear functions and linear systems (much of it retrograde for Algebra 2) overlaps quite heavily with the Algebra 1 course. This duplication expands the content of both courses, and it makes both courses’ instructional goals less clear to teachers and students. In the Geometry course, there seems to be no recognition of, or design for, the possibility that students have seen the Pythagorean theorem or the distance formula as early as grade 8, where it appears in the standards. The course-to-course progression shows some evidence of vertical coherence in specific areas. The extent of the material is so great, especially in the Geometry and Algebra 2 courses, that in a single academic year, the material might only be learned and taught superficially. The choice to include matrices, the Law of Sines, the Law of Cosines, permutations and combinations, and expected value—all of which are (+) material (designated as beyond the college- and career-ready line in the standards)— was a choice to make the coverage burden heavier. So was the choice to include topics not required by the standards at all. Finally, note that the (+) material is not accurately indicated as such in the materials. Correlations to the standards omit the (+) symbol when citing content standards like NVM.A.1(+) or G-SRT.D.10(+), which sends teachers an inaccurate signal about the demands of college and career readiness. Limitations in Modeling As Content And Practice In the standards for High School, Modeling is emphasized more than in the standards for grades K–8. This can be seen by the evolution of Modeling from a Standard for Mathematical Practice in grades K–8 to both a practice and a content category in High School. Modeling is on a par with Algebra, Functions, Geometry, and other content categories in the organization of the High School standards. The reason the High School standards have this design is that any set of validly College- and Career-Ready standards must emphasize Modeling and quantitative literacy during the High School years, when students are preparing for their postsecondary mathematical futures—futures that will involve using math in college, work, and life. Studies repeatedly reveal the importance of being literate with mathematics, and this means not only being able to apply the traditional content of high school mathematics, but also being able to apply the powerful skills first learned during the middle grades to solve demanding tasks.ii As Steen (2007) noted, “High schools focus on elementary applications of advanced mathematics whereas 13 most people really make more use of sophisticated applications of elementary mathematics. … Many who master high school mathematics cannot think clearly about percentages or ratios.”iii In fact, the international PISA Mathematics test is entirely devoted to mathematical literacy—and very little of the test draws on any mathematics beyond that first introduced the middle grades. (That doesn’t make the test easy, as Steen observed.) All of which is to say that Modeling deserves the emphasis it receives in College- and Career-Ready standards. Aligning to the Modeling standards doesn’t require that rich Modeling tasks be prevalent in the materials, but it does require that the richest Modeling experiences found in each High School course reflect highly developed practices of mathematical Modeling. Issues emerge here for the materials examined, because the design of the program as a whole places a low ceiling on the depth of Modeling in the program. In the Common Core Performance Tasks, one of which is presented in each Chapter Opener, the design is that students chip away at them throughout the Chapter; and while this helps students to apply what they are learning in the Lessons—the task to some extent structures the Chapter experience—the approach also turns the tasks into highly scaffolded experiences, placing a ceiling on how students engage in both MP.1 and MP.4. The localization of each Common Core Performance Task to a Chapter with a specific content focus also essentially dictates what mathematics to use; indeed, the commentary to the student typically removes any doubt (“you’ll use a system of linear equations to model the three criteria”). Although an On Your Own extension is always provided, the extension problem is of the same kind, and in a busy course one may doubt that On Your Own extensions happen frequently. There are no problems on the End-of-Course Assessments that resemble the performance tasks, either scaffolded or unscaffolded. In one Common Core Performance Task in which linearity is unrealistic given the context, a student engaging actively in the Modeling practice might have chosen a linear model anyway, provided the results of doing so could be useful in the situation. But instead of students choosing the mathematics to use, the task design avoids those Modeling practices by (1) providing an artificial, perfectly linear data set for the problem; and (2) telling the student directly that this perfect linear pattern will continue. Artificiality in the program’s application problems can be extreme. Quantities in tasks that could have been researched online or estimated, thereby increasing students’ quantitative literacy, are instead provided for students or abstracted away. Even in Common Core Performance Tasks, Modeling amounts to little more than the art of solving lengthy versions of schoolbook word problems, with heavy signposting and scaffolding. To conclude this theme, it isn’t that it would have been better to leave Common Core Performance Tasks without scaffolding; doing so would have made almost every one of the tasks too difficult. The design of having students work on the tasks throughout the Chapter probably succeeds in getting more students to the answer, and it could be a sound strategy for bringing students along in their ability to solve substantial applications. However, it remains true of the program design that students do not engage in substantial Modeling practices during any high school course. Problems of Coherence and Concepts There were missed opportunities to make connections and convey particular concepts: 14    The treatment of the distributive property doesn’t take into account, or communicate to students, that they have been using the property ceaselessly ever since elementary school, for example in working with area models of multiplication or when performing calculations such as 3  2¼ = 6 + ¾ or 7  111 = 700 + 70 + 7. Instead, a handful of artificial integer examples are given, missing an opportunity to ground formal algebra in the familiar. A problem thoughtfully relating the distributive property to calculation strategies appears only as the Lesson ends. When students first encounter the term imaginary, they are encouraged to think of its everyday meaning, as follows: “Something is imaginary if it has no factual reality.” Later, teachers are warned about “the common misconception that imaginary numbers do not exist.” Might the misconception be coming from the curriculum itself? In the Geometry course, the diagram provided alongside the definition of congruent angles misses an opportunity to show a case where not all rays have equal lengths—which might have helped teachers to address a common misunderstanding that had been noted only two pages earlier in the Math Background. A possible teaching moment is sacrificed. Big Ideas are sometimes not so big, and Essential Questions are sometimes unhelpfully put. For example, “How are radical expressions represented?” immediately conjures the answer “with radicals.” The way transformations are handled in the Geometry course is, roughly speaking, first to treat SAS and other congruence theorems as postulates for a considerable time, doing some proving on that basis, and then later to circle back and build up the machinery of transformations, using it to (re)define congruence and bolster the earlier theorems which had to that point only been postulated. A possible effect of this approach is to raise questions in everyone’s mind about why transformations are happening at all; absent the overall picture, they could just look like ‘stuff we have to cover.’ Some of the Take Note boxes present Key Concepts. In some cases, it is unclear in what sense these Key Concepts qualify as concepts. Do we consider that “The distance between two points A(x1, y1) and B(x2, y2) is d = Sqrt[(x2 – x1)2 + (y2 – y1)2]” qualifies as a concept? This symbolic formula is something that might be good to commit to memory (probably not), but in any case, on the ordinary usage of the word, concepts are things understood, not things memorized. Perhaps the concept here ought to have been the idea that one can find the distance between two points so long as you know their coordinates. Or perhaps it ought to have been the insight that the distance formula is how the Pythagorean theorem looks in coordinates—so that if you forget the distance formula, you could use a2 + b2 = c2 and a diagram to recover it. This connection between the theorem and the formula might have been the basis of a Lesson in itself, instead of being grist for a problem. The problem-based pedagogy of the program has limitations if it makes ideas (as distinct from results) difficult to concentrate on. Another questionable instance of a concept is the Concept Summary called “Solving Equations.” Perhaps the concept here ought to have been the understanding that an equation asks which values of the variables make the statement true—or perhaps the idea that solving an equation means either finding all of those values or showing that there aren’t any. In any case, the Concept Summary says: “Step 1 Use the Distributive Property to remove any grouping symbols. Use properties of equality to clear decimals and fractions. ...” (There are four more steps after that.) Recipes are not concepts. A legitimate concept about solving equations could have been broadly and durably valuable for students, but the 15 payoff to remembering the recipe in the Concept Summary is meager because it has a limited range of applicability. Every chapter includes many practice problems of various kinds (supplemented by peripherals like the Student Companion). If teachers assign sufficient quantities of practice and also provide individualized feedback and troubleshooting, then it seems likely that students will attain algebraic fluency and procedural skill. What is less clear is whether the students will know why they’re doing things, how the subject holds together, or how to apply the skills autonomously to solve problems. The volume of practice provided causes well-made problems to fade into the woodwork, and heavy scaffolding in what would otherwise be a valuable task can deny students opportunities to synthesize the material in the Lesson. Limitations in Math Practices The embodiment of MP.5 in the Common Core Performance Tasks is often the opposite of using tools strategically (“You’ll find the inverse of a 33 matrix using a calculator”; “You’ll use a graphing calculator or other graphing utility...”). A calculator icon could short-circuit several mathematical practices in a non-routine problem like x + 3 = 3x, for which symbolic solution methods aren’t available to students. The Geometry course stresses precision in the use of terms and notation. The opening portrayal of the deductive system is problematic however, saying “A postulate or axiom is an accepted statement of fact.” Facts are usually thought of, even by high school students, as empirical truths, not agreed-upon starting points in a communal enterprise of logical deduction. Soon afterwards, an inapposite analogy between undefined terms and hard-to-define words in English is offered. Vocabulary checks in the program don’t always clarify terms and delineate their precise scope, which is an important part of disciplinary vocabulary in mathematics. Throughout the program there are tasks flagged Reasoning. Getting the answers to these problems usually requires that some reasoning occur, but it should be said that in relation to MP.3 (constructing and critiquing arguments), the suggested answers can paint an impoverished picture of a mathematical argument. Some problems come nearer to the practices in question, but answers to Reasoning problems don’t always sketch an argument of any sort and do not always avoid the reasoning error of arguing the converse of the proposition requested. Instead of being an opportunity to look for and make use of structure, a problem on an End-of-Year Assessment commands the student to “Use factoring.” Were that directive removed, the problem would engage students in MP.7. Low support for teachers to conduct student discussions of mathematics Teaching supports for classroom discourse in the program usually take the form of suggested questions to ask the class; these guiding questions frequently have closed answers. Even good questions about a problem at hand could be countermanded by a “Step 1, Step 2, Step 3” instruction or other printed 16 guidance. And even when suggested questions are mathematically meaty, there isn’t a robust discoursestructure designed into the program to bring rich classroom conversations reliably into being. “Describe the differences between a linear function and an exponential function” ought to be a meaty discussion among students, but the teacher isn’t given support to host such a conversation (and the idea is positioned as merely vocabulary). Guiding questions might lead the class through the required argument, but given the designed flow of the Lesson, no student need ever speak up to provide that argument in toto (MP.3). When a “Take Note” for the teacher is positioned as “Error Prevention,” a possible learning moment for students is conveyed as students just being wrong. The teacher supports provided can be counterproductive in other ways. For example, the directive to “Remind [the students] that multiplication is repeated addition” treats students as mental third-graders and would render baffling such products as (⅛)(1), Sqrt[2] or 1.1ex that have no apparent interpretation as sums. In the Math Background for Solving Equations, teachers are told about “Solving Proportions” like 12/9 = 8/x—in particular told that in these cases “students have choices,” namely “They can multiply both sides of the equation by the least common multiple (LCM) or use the Cross Products Property.” The advice to use the LCM seems to suggest that students studying this program are carefully sheltered from non-whole-number problems; and neither piece of advice encourages teachers to view “proportions” as the equations they are. For example, what if instead of cross-multiplying, we were to divide both sides of the equation 12/9 = 8/x by 8? The result is 1/6 = 1/x, from which the solution can be read by inspection. Yet another choice, inefficient but mathematically productive, could be to use the graphical method that was outlined on the very page that faces the one with the equation on it. This graphical method was alas carelessly explained, the explanation suggesting incorrectly that a point, rather than a number, solves the equation 2(2x  1) = 2(x  2). Similarly, the note on measuring segments doesn’t distinguish between points and numbers. Fortunately, the relevant Lesson gets this right—but it ought to be right for the teacher too. Thus, the educativeness of the curriculum is mixed. On the plus side are the notes for the teacher about why topics on the Get Ready! assessments were included; the Essential Questions (where coherent); the existence of guiding questions in the margins of the Guided Instruction phase; common student errors flagged in Math Background; and the better examples of other marginalia, such as the “Take Note” that invites the teacher to draw students’ attention to the fact that planes are described with reference to lines, and lines with reference to points. Tasks are not always free of unnecessary language complexity. ELL support is available via the Online Teacher Resource Center, often positioned as vocabulary work, instead of providing ELL students with content-focused language work. Lessons include notes to support teachers of ELL students, but these notes typically appear after the end of the Lesson flow. Systematic use of language routines in the Lessons themselves is not evident. Multiple representations are used, including for ELL students, but the lessons don’t appear systematically to use them to support students who struggle in other specific, predictable ways, such as systematically using technology to solve equations for students who are still developing the symbolic fluency to solve algebraically/formally. 17 The Assignment Guides for the Practice components of the Lessons suggest assigning different problems to students at different levels (Basic, Average, Advanced). Absent appropriate instructional leadership, this could potentially result in the various classrooms within a school enacting somewhat different curricula, or the various schools in a district enacting somewhat different curricula. (It does appear to be a policy of the materials that the Basic selections extend beyond the initial skill practice of “section A.”) There are pre-assessment moments (Get Ready!) and post-assessment moments (Lesson Check, Lesson Quiz, Chapter Test, End-of-Course Assessment). As is common in the genre, an intense amount of energy is devoted to differentiating instruction, even though some “Reteaching” resources would likely help most students, and even though some Enrichment resources, Activities, Games, and Puzzles could be fruitful and motivating for more than just the “high kids.” The Assess & Remediate cycle does not appear to have been augmented or offset by a substantial design effort to address inevitable unfinished learning within the main Lesson flow. In the items for the Entry-Level Assessment for the Geometry course, there is significant algebra coverage. Will the effect of this be to funnel weak Algebra 1 students into a “low” Geometry classroom, or into a lower high school track altogether, when in fact the algebra skills used in geometry might be reinforced there as needed, and practiced in new and potentially more visually engaging contexts? Program designers and those who implement programs might reexamine their assumed ability to diagnose, level, and place students using brief measures (Oakes, 2005). There are worthy features in the program under review. Some weaknesses could be remediable: a district could tune practice, adjust the peak Modeling level by integrating external resources, and make various fine-grained alignment adjustments. But larger-scale content problems, and pervasive weaknesses in math practices and teacher supports such as those described here, are fundamental to the learning experience and are not straightforward to address. 3. Recommendations to Consider in Adopting New Materials Alignment criteria Materials must pass a searching alignment review. Recommended criteria for reviewing materials— identical to those used for the reviews above—are included as Appendices D and E. The alignment criteria have been adapted to the present purpose from the Instructional Materials Evaluation Tool (IMET), which is part of an authoritative alignment toolkit applicable to the Michigan mathematics standards. The alignment criteria isolate essential high-level features of the standards, allowing fundamental misalignments to be detected. Fit to the district Programs that align well to the standards might not all offer the same fit for DPSCD; some factors to consider are in Appendices D and E. Instructional core vs. peripheral components 18 To improve mathematics achievement, the pedagogy of the instructional materials gradually has to work in DPSCD teachers’ hands, which means it has to be robust in itself, well enough supported in the materials, and feasible enough for teachers to implement when provided with instructional leadership, common planning time, and professional learning. While assessments and acting on student data are a necessary part of the enterprise, when evaluating mathematics curricula they are not the factors to optimize for. An ounce of good teaching is worth a pound of differentiation and remediation. Pathways In the middle grades and high school, the question of mathematics curriculum is inseparable from course pathways. The existence of Placement Tests in the EnVision materials raises a question about whether or how these or other assessments are being used to track young children. In all grades, DPSCD should review tracking and acceleration policies to ensure school buildings do not observe tracking practices that may work against DPSCD goals. Offering a Pre-Algebra course in the middle grades is unnecessary because the standards in the middle grades already constitute such a course. Regarding high school pathways, a WestEd studyiv of observed course-taking patterns in California concluded that “For some students, taking algebra 1 while still in middle school may make the most sense. For others, taking it in grade 9, or even grade 10, may make more sense, presuming the student continues to take math courses and develop the requisite foundation for learning algebra 1 concepts and skills.... [D]istrict emphasis should not be on accelerating all students into grade-8 algebra 1, but, instead, should be on ensuring that students are ready for the next level of math, all along the way.” “When students take algebra 1 (that is, in which grade) is less important than whether students are ready to take it.” The study also notes that “Few students who repeat algebra 1 ever reach the level of Proficient or higher” on the state Algebra 1 test. San Francisco Unified School District recently found positive results from ending its policy of Algebra I for all 8th graders, and pushing the main branching point forward to high school.v The Algebra 1 retake rate dropped for every ethnic group, and the number of low grades in middle school decreased. This year’s juniors are the first wave of the new policy, and the number of juniors taking Precalculus this year exceeds the number of seniors who took AP Calculus last year. DPSCD can review the facts of the case in San Francisco Unified, as well as relevant policy research and internal research, and on that basis develop and propose a similar policy for the district. Bridge to postsecondary The State of Michigan requires a fourth course in the final year of high school. DPSCD can identify good fourth-year options for students. To help connect graduates to postsecondary opportunities, DPSCD can also take steps to increase the number of grades 9–11 students who take advantage of Khan Academy’s free practice on the SAT exam, which is the Michigan Department of Education prescribed state high school assessment. DPSCD can also compare course topic coverage and depth to topic coverage and depth on the state assessment. 19 Appendix A: Materials Alignment Criteria as Applied to Existing Instructional Materials (Grades K–6) 20 Mathematics Instructional Materials Alignment Review (Grades K–6) (Based on Key Criteria of the Instructional Materials Evaluation Tool (IMET)) Program Reviewed: EnVisionMATH Common Core 2012 Reviewer instructions: 1. First, carry out the Preliminary Alignment Check in Section I. If the program does not pass the Preliminary Alignment Check, then it is Far From Aligned. 2. If, and only if, the program passes the Preliminary Alignment Check, review the program against the remaining criteria in Sections II–V. For each Section, describe evidence as indicated, and then assign a holistic rating using the rubric provided. 3. Based on the results of step (2), the program will earn a total number of alignment points. The total number of points can be used to identify large differences in degree of alignment between programs that merit consideration having passed the Preliminary Alignment Check. For programs with identical point totals, trends in the evidence recorded in the rubric can help to distinguish qualitatively between greater and lesser alignment. Note: The program under review failed the Preliminary Alignment Check, but Sections II–IV were completed to generate additional information about the program. Note: Citations to the program (for example, “K Lesson 4-9”, “G2 Center Activity 13-5”, etc.) are particular instances of observations; they aren’t necessarily the only instances. Any single item noted in the evidence might or might not break the alignment of the program; holistic ratings take into account this evidence as part of the total picture. 21 Program Reviewed: EnVisionMATH Common Core 2012 Section I. Preliminary Alignment Check Note 1: If the current version of the program has been reviewed by EdReports at www.edreports.org for all grades K–5 and/or for all grades 6–8, and if the program has received at least 12 points for “Gateway 1” in every grade reviewed, then the program can be considered to pass the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 2: If the current version of the program has been reviewed by EdReports at www.edreports.org for any grade, and if the program has received fewer than 8 points for “Gateway 1” in any grade reviewed, then the program can be considered to fail the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 3: If neither Note 1 nor Note 2 applies, then the program must be reviewed against the criteria in the Preliminary Alignment Check (Table I.1). Note 4: All materials passing the Preliminary Alignment Check must continue on to Sections II–V and be reviewed according to the criteria listed in those sections. There are five criteria in the Preliminary Alignment Check (Table I.1). If any criterion isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. First, describe trends in the evidence for these criteria; then, circle Yes/No to indicate whether each criterion is met. Table I.1. Criteria for Preliminary Alignment Check. See IMET pp. 4–11 for guiding questions. Metric 1: Materials reflect the basic architecture of Metric 2AB: Students and teachers using the the Standards by not assessing the topics listed materials as designed devote the large majority of below* before the grade level indicated. time to the Major Work of the grade. Supporting Work also enhances focus and coherence simultaneously by also engaging students in the Major Work of the grade. Grades K–2 Met (circle one): Yes No Grades 3–6 Met (circle one): Yes No Evidence relevant to these criteria Grades K–2 Met (circle one): Yes Grades 3–6 Met (circle one): Yes No No Metric 2CD: Materials follow the grade-by-grade progressions in the Standards. Content from previous or future grades does not unduly interfere with on-grade-level content. Lessons that only include mathematics from previous grades are clearly identified as such to the teacher. Grades K–2 Met (circle one): Yes Grades 3–6 Met (circle one): Yes No No Metric 1 is met in grades K–2. Metric 1 is not met in grades 3–6 (G5 Lesson 14-1, Lesson 14-2, Lesson 14-4, Topic 14 Reteaching, Topic 14 Test; G6 Lesson 11-7, Topic 11 Reteaching, Topic 11 Test). The misrepresentation of the Kindergarten standards in the materials is especially pronounced. Center Activities in Kindergarten include pattern work that isn’t required in the standards (Center Activity 3-7). Classification work in Kindergarten is far more involved than in the standards, taking focus away from the number core (Center Activity 13-5, Topic 13 Test). Measurement work in Kindergarten is far more involved than in 22 Program Reviewed: EnVisionMATH Common Core 2012 the standards, taking focus away from the number core (Topic 12). Extensiveness of geometry in Kindergarten is out of proportion to the standards (Topic 14, Topic 15, Topic 16). The very substantial classification work in Kindergarten is positioned more in support of data representation than in support of the number core (note that classification follows the material on joining addends, so that classification isn’t a means to that end). See also Topic 13. Number at the Kindergarten level is nearly absent in the treatment of geometry (Topics 14–16). The number line is present in Kindergarten (Lesson 4-9), but the number line first appears in the standards in grade 2. Estimation is present in Kindergarten (Lesson 6-2), but estimating isn’t required by the Kindergarten standards. Graphs are present in Kindergarten (Lesson 9-9), but graphs first appear in the standards in grade 2. Kindergarten Measurement verges into material that is grade 3 in the standards (Lesson 12-1). Kindergarten Center Activity 6-6 is pattern extension work not in the standards. Grade 1 work often aligns to K.OA (G1 Lesson 1-3; compare Interactive Math Story for Topic 9, K TE pp. 167G, 167H). The Readiness assessment in grade 1 assesses skills not mentioned in the Kindergarten standards (such as counting back); shows the same misaligned emphasis on sorting, classifying, and graphs as typifies the Kindergarten materials; and most dismaying, fails to assess capstone standards from Kindergarten including K.CC.1, K.CC.2, K.CC.4, K.OA.5, K.CC.6 (for numbers greater than 5), or K.CC.7. In fact, standards code K.CC.7 does not appear in the “Looking Back” section of any grade 1 Skills Trace. This standard is addressed only obliquely in K Topic 4, and an Item Analysis for Diagnosis and Intervention alters the plain meaning of K.CC.7 (K Topic 4 Test, TE p. 89A). The Benchmark Test for Topics 1-4 does not assess the standard. Whether by design or oversight, K.CC.7 appears to have been written out of the curriculum in grade K or grade 1. Comparing fractions using benchmarks is not required by the standards in grade 3 (Lesson 10-3, Topic 10 Test). Grade 3 geometry aligns to the grade 4 standards (Lesson 11-1, Lesson 11-2, Lesson 11-4). Grade 4 Lesson 1-2 miscoded to 4.OA.5. Pattern work in Topic 2 misaligns to the cited (single) standard; for example, the standard refers to a given rule, but Topic 2 concentrates on guessing and writing the rule (Lesson 2-1, Lesson 2-2, Lesson 2-3, Lesson 2-4, Topic 2 Test). Distance-time graphs in grade 5 are not required by the standards (Lesson 14-5 and Topic 14 Test problem 6) and appear before ordered pairs or the coordinate plane (Topic 16). Distance-time graphs in Lesson 16-4, Lesson 16-5 are middle grades material. G5: Lesson 8-1 aligns to grade 6. Lessons 8-5, 8-6 aren’t required by the standards. Lesson 8-8 variables are not required in grade 5. The Visual Learning problem in G5 Lesson 6-6 is a rate problem. Integer arithmetic is not required in the grade 6 standards. (One can understand why a curriculum author designing for the standards as written might treat integer arithmetic in grade 6 anyway, but the discrepancy is noted; and there is a tradeoff because it increases the already large coverage burden for this grade.) Lesson 11-3 aligns to grade 7, and including it also necessitates Lesson 11-2. Lesson 11-4 aligns to grade 8. Lesson 15-1 aligns to grade 7. Lessons 15-3 and 15-5 exceed what is required in grade 6. In Lesson 19-4: mode and range are not required by the standards. “Common Core Review” does not always match the grade level of the standards (G5 Lesson 3-5 TE p. 72A; G4 Lesson 1-6 TE p.20A problem 2). *Topics for Metric 1. (No other topics should be added to this list. Note that other topics in the standards are addressed in criteria 2A–C.) • Probability, including chance, likely outcomes, probability models. (Introduced in the standards in grade 7) 23 Program Reviewed: EnVisionMATH Common Core 2012 • Statistical distributions, including center, variation, clumping, outliers, mean, median, mode, range, quartiles; and statistical association or trends, including two-way tables, bivariate measurement data, scatter plots, trend line, line of best fit, correlation. (Introduced in the standards in grade 6) • Coordinate transformations or formal definition of congruence or similarity. (Introduced in the standards in grade 8) • Symmetry of shapes, including line/reflection symmetry, rotational symmetry. (Introduced in the standards in grade 4) If any criterion in Table I.1 isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. Preliminary Alignment Check Result (circle one): Passed Far from Aligned Section II. Alignment: Concepts, Fluency, Applications Table II.1. Criteria for alignment in concepts, fluency, and applications. See IMET pp. 14–19 for guiding questions. Metric AC-1A. The materials support the Metric AC-1B. The materials are designed so that Metric AC-1C. The materials are designed so that development of students’ conceptual students attain the fluencies and procedural skills teachers and students spend sufficient time understanding of key mathematical concepts, required by the Standards. working with applications, without losing focus on especially where called for in specific content the Major Work of each grade. standards or cluster headings. Evidence relevant to these criteria Counters assigned inconsistent meanings—simultaneously a unit and not a unit (K Topic 1 Opener, “Count the Birds”). Early counting complexified by the problem solving cycle (K Lesson 1-7 bridge). A picture of two worms has a prominent numeral 1 next to it (K Lesson 1-1). In G3 Lesson 1-2, the emphasis is that there are many ways to name numbers, not on the fundamental concept that 1 thousand = 10 hundreds (emphasized after the fact, in the Reteaching for Topic 1). Grade 3 work with three-digit addition builds economically from place-value blocks (Lesson 3-1, Lesson 3-2) to an efficient place-value method that reflects the fluency standard for this grade (Lesson 3-3), including for more than two addends (Lesson 3-4). This sequence is paralleled for subtraction (Lesson 3-6, Lesson 3-7, Lesson 3-8, Lesson 3-9). However, the Chapter 3 test does not assess fluency with three-digit sums and differences, which is the only thing the standards require here. The test as a whole aligns to grade 2 standards. 24 Program Reviewed: EnVisionMATH Common Core 2012 The relevant Topic Test in grade 4 does not generate evidence about whether students are fluent in multi-digit addition and subtraction (Topic 4 Test), or what their calculation errors might be. The grade 4 materials misalign completely to the capstone standard for multi-digit addition and subtraction, both in messaging in teacher-facing materials (Topic 4 Skills Trace, Big Ideas, Essential Understandings, Math Background) and in the student-facing curriculum. Concepts of multiplication and division are prominent in grade 3, and coherent both in themselves and in their sequencing (Lesson 4-3, Lesson 6-1, Lesson 6-6, Lesson 7-1, Lesson 7-3, Lesson 8-1, Lesson 8-7). Applications of both operations are prominent (Topics 4–8). The concepts of area are thinly presented, and area is defined in terms of square units before square units are defined (G3 Lesson 14-1, G3 Lesson 14-2). The area formula for a rectangle neither emerges organically from concepts of multiplication nor becomes retrospectively understood by them (G3 Lesson 14-4). Enduring Understandings associated with the Big Idea titled Properties include order of operations (which have no relevance to Properties). In the example showing how to multiply two decimals (G5 TE pp. 158, 159), one of the steps requires multiplying two decimals. This apparent vicious circle passes without comment in the teacher-facing materials. In decimal grids, the whole (that is, 1) isn’t shown (G4 Lesson 13-4, G4 Lesson 13-8, G4 Topic 13 Test). Only a handful of problems combine fractions and decimals in a single operation (G5 Lesson 9-6 Problem Solving problem 19—but the sample answer in the teacher-facing materials finds a way to keep the fractions and decimals separate (TE p. 233); G5 Lesson 11-6 Problem Solving problem 29—here the advice is to use the distributive property, but this hasn’t been the approach instructionally to this point.) G5 Topic 6 Algebra Connections: no fractions appear. G5 Topic 16, no fractions or decimals get plotted in the coordinate plane. G5 Lesson 6-5, Multiplying a Decimal by a Whole Number is not connected to multiplying a fraction by a whole number (even later in Topic 11), even though multiplying a fraction by a whole number was done in grade 4. G5 Lesson 11-9 (how many 1/4s are in 3?) has no lookback to Lesson 7-3 (how many quarters are in $15.50?). In grade 5, fraction multiplication isn’t one thing, but three (G5 Topic 11 TE p. 273A; also compare TE p. 61A with TE p. 273A, 273B). Properties of multiplication are cast as properties of whole-number multiplication. In G4 TE p. 102A, the Math Background is “Finding products by breaking numbers apart using place value is an application of the Distributive Property. The property for whole numbers a, b, and c is: a(b + c) = ab + ac.” Four years’ worth of properties of multiplication pass by without meaningfully generalizing the treatment beyond whole numbers (G3 Lesson 5-3; G4 Lesson 1-3; G5 Lesson 3-1; G6 Lesson 2-2, Lesson 2-4). G5 Lesson 3-5, “Distributive Property,” includes no numbers at all in fraction or decimal form. Nor does it raise a question for later about whether the property holds for these numbers. Reteaching for G5 Topic 3 only uses whole numbers. In the index for the G5 Student Edition, Lesson 3-5 is the only page reference given in the entry for the distributive property. In the entire G5 Student Edition, it would appear that no problems with fractions were designed to prompt distributive property thinking. A single problem was found involving decimals with the distributive property in the forefront (Lesson 9-7, Problem Solving problem 28). The teacher25 Program Reviewed: EnVisionMATH Common Core 2012 facing note for this problem (TE p. 235) does not signal the algebraic importance of this problem, but instead reads: “Test-Taking Tip: Make a Plan.” In grade 6, the properties (G6 Lesson 2-2) are presented only with whole numbers. After twenty-seven whole number problems or illustrations, we get a single problem with decimals, 33.5 + (20.5 + 21.5) = (33.5 + 20.5) + 21.5. There are no fractions. G6 Lesson 2-4 helpfully surfaces the distributive property as it was long used for mental math—but again with no fractions or decimals. Grade 6 Lesson 6-1 essentially repeats Grade 5 Lesson 11-1, down to the artwork on the pizzas. Error Alert: “After working with multiplication of mixed numbers, students should be careful not to do the conversion to improper fractions when they go back to adding or subtracting mixed numbers. This is a common mistake that even advanced students often make after spending some time multiplying and dividing mixed numbers” (G5 Topic 11 Math Background). When the grade 4 teacher is prompted to ask “Why might it be better to write this as a multiplication problem rather than a repeated addition problem,” the suggested answer doesn’t evoke unit thinking or grade-level “times-as-much” thinking, but instead is only about answer-getting: “You might miscount when writing all the 10s.” (G4 Lesson 5-1 TE pp. 116, 117) A problem with times-as-much thinking is not part of the lesson flow but is instead hidden in the Problem Solving Section (TE p. 117 problem 17). Unit thinking comes to the fore in the next Lesson, but now nearly identical problems 3  10 and 3  50 are handled differently. The role of unit thinking in the Lesson is easy for teachers and students to miss because the subtitle of the Lesson is not about thinking but about rules: “What is the rule when you multiply by multiples of 10 and 100?” (G4 Lesson 5-2) G5 Lesson 6-4, in a “What You Think” box for the problem 0.36  4, students aren’t to think anything about fractions; nor are they to think about multiplication. The next segment of this Lesson, Another Example, considers the problem 0.5  0.3. Here again students don’t think about fractions, or about multiplication; and since repeated addition suddenly fails as an idea about multiplication, students are asked to think now about shading. It is simply announced that “The product is the area where the shading overlaps.” The representation isn’t being used to reflect mathematical thinking; it’s being used as an analog calculator. (In fact, right next to it is a suggestion to multiply using a real calculator, with pictures showing which buttons to press.) In grade 6, students are a year older and yet the very same example (0.36  4) is given the very same treatment (G6 Lesson 3-4). Even the artwork for the problem is identical. G5 Lesson 2-1 Mixed Problem Solving / Science, problem 9. “... If the number of drips from a faucet is 30 per minute, how many drips is this for 10 minutes? Use repeated addition.” In G5 Lesson 6-6, “Multiplying Two Decimals,” the Objective is “Students will use the standard algorithm to multiply decimals by decimals.” This is allowed in grade 5, but not required (cf. 5.NBT.7) and the algorithmic focus compromises the coherence of the Lesson. The distributive property is not mentioned in either the student-facing or teacher-facing materials, breaking the connection to whole-number multiplication in base ten. 26 Program Reviewed: EnVisionMATH Common Core 2012 No connection to properties of operations when solving a linear equation (G6 Lesson 13-1). G5 Lesson 3-1: “Haley said that she would always know her 0 and 1 multiplication facts. Explain why Haley would say this” (Problem Solving problem 26). A grade 3 problem. In grade 6 Lesson 5-4, the fraction concepts are grade 3; in Lesson 7-1, they are grade 4; in Lesson 7-3, they are grade 5, miscoded to 6.NS.1. Lesson 7-4 aligns to the grade 5 standards, as do Lessons 8-1, 8-3, and 8-4. Substantial parts of Lesson 8-5 are copy/pasted from grade 5 Lesson 11-8. G6 Lesson 2-3 is miscoded to 6.EE.3. In Reteach, the only verb visible to students (the only thing they are asked to do) is “Remember.” Rating for Section II, Alignment in Concepts, Fluency, Applications _X_Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section III. Alignment: Standards for Mathematical Practice Metric AC-2B: Tasks and assessments of student learning are designed to provide evidence of students’ proficiency in the Standards for Mathematical Practice. Metric AC-2C: Materials support the Standards’ emphasis on mathematical reasoning. Materials call for students to produce mathematical arguments. Materials don’t teach multiple methods or strategies just for the sake of variety, but instead support the teacher in using that variety to draw mathematical connections between methods for the benefit of student learning. Metric AC-2D: The richest Modeling experiences found in each grade 6–8 reflect significant practices of mathematical Modeling. (Rich Modeling tasks need not be prevalent in the materials to meet this criterion.) 27 Metric AC-2E. Lessons alert the teacher to opportunities for students to discuss important mathematics, and provide sufficient teacher support so that discussions are likely to be successful and build students’ understanding. Program is educative for teachers when combined with professional development in mathematics. Program Reviewed: EnVisionMATH Common Core 2012 Guiding questions Considering the variety of tasks and assessments provided (observation checklists, portfolio recommendations, performance tasks, tests and quizzes), do students have opportunities to demonstrate proficiency with each of the Standards for Mathematical Practice over the course of the year? Does emphasizing the Standards for Mathematical Practice tend to open up room for students to work on content not required by the standards? Are students challenged to make sense of word problems, or are word problems always so scaffolded or modeled that students learn recipes for them? Is Modeling incorrectly portrayed as being about such things as using manipulatives, connecting representations, or showing one’s thinking? Find, or have the publisher provide, the richest Modeling experiences in grades 6, 7, and 8. How prominent are Modeling practices required in these tasks? Are calculators used to avoid showing proficiency with calculation and procedures? Or are students asked to be strategic about technology? Are students pushed to improve the precision of their mathematical statements? Are grades K–5 students supported to look for and express regularity in repeated reasoning about the addition table, the multiplication table, the properties of operations, the relationship between addition and subtraction or multiplication and division, and the place value system? Are grades 6–8 students supported to look for and express regularity in repeated reasoning about proportional relationships and linear functions? Are students merely asked to guess patterns and extend them term by term, or are they asked to draw mathematical conclusions about patterns? Evidence relevant to these criteria Problem Solving Lessons can enrich the way students engage with required content (G4 Lesson 11-8), but they can also import content into the curriculum that isn’t required by the standards (G3, Lesson 1-8, Topic 1 Test, Lesson 6-8, Topic 6 Test, Lesson 9-8, Topic 9 Test, Lesson 11-8 Independent Practice problem 5, G4 Lesson 3-6, Independent Practice problem 6, G4 Topic 3 Test; G6 Lesson 1-7). In addition to using patterns to teach important content such as the multiplication table, “Look for Patterns” also imports functions before they are required (G3 Lesson 14-6 Independent Practice problem 9; G4 Topic 16 Test; G5 Lesson 14-5). Problem Solving Lessons can also involve retrograde content (G5 Lesson 25, which is centered on grade 2 word problems). Problems labeled with “CC Mathematical Practices” are not always the problems that reflect the Standards for Mathematical Practice. For example, a deft problem-in-brief like (7 + ?) + 6 = 7 + 6 involves MP.7, but there is no ‘practice tag’ on it (G3, Lesson 2-1, Independent Practice, problem 14). The “Persevere” labels on word problems might clue students that a multi-step problem is coming (G3, Lesson 1-8, Independent Practice problem 10; G3, Lesson 5-1, Problem Solving problem 40). A Tip that “There is extra information in the problem” short-circuits the part of MP.1 about making sense of problems. Other times, the Persevere labels are inapt (G3 Lesson 8-3, Problem Solving problem 40). G3, Lesson 5-3, Problem Solving problem 40: why a Reason tag? Prompts sometimes ask indiscriminately for explanations even where doing so is likely to be sterile (G4 TE p. 117A Quick Check Student Samples). G5, Lesson 9-5, Problem Solving problem 34 teacher facilitation note on TE p. 231: “Critique the Reasoning of Others. Ask students what they know about 2/3 and 4/6. [They are equivalent fractions.] Ask So, is her brother correct? [Yes, they are both correct.]” 28 Program Reviewed: EnVisionMATH Common Core 2012 Materials at all grades K–6 are designed so that teachers and students spend sufficient time working with applications. The level of modeling and complexity of applications is sufficient to meet the standards, but the level also sometimes exceeds what is required and sometimes drifts away from the content of the grade entirely (G4 Topic 4 Performance Assessment). Word problem strategy often hinges on key words, instead of understanding the operations (G4 Lesson 1-10 Independent Practice problem 7). “Coral reefs cover less than 1/500 of the ocean floor, but they contain more than 1/4 of all marine life. Which is a common denominator for 1/500 and 1/4?” (G5, Lesson 9-6, Problem Solving) The ‘practice tag’ Model can misrepresent MP.4, which is about contextual problems (G3, Lesson 2-1, Problem Solving, problem 9). Math Background incoherence (put G1 Topic 1 Math Background side-by-side with G1 Topic 2 Math Background). Essential Understandings are sometimes a mix of the essential and the inessential (G4 TE p. 121A). G4 Topic 1 Math Background omits fraction multiplications occurring for the first time at this grade, and it omits one of the key developments in grade 4, “times as much” thinking about multiplication, which is the focus of a Lesson in Topic 1 (Lesson 1-10). After asking students to find the products 3  4, 3  40, and 3  400 using any desired method, the guidance for the share-out is simply “Have students share how they found the products. Discuss patterns they observed.” (G4 Lesson 5-2 TE p. 118B) Questions in Instruct in Small Steps are often closed and there is no evident routine structure for discussion, critique, or debate. Guidance does not tell the teacher what to do if a student uses a primitive “count all” strategy for the small numbers in the problem (G1 TE p. 191). In grade 5, to convert 3/5 to a decimal, students are shown (told) to do it by long division (Lesson 1-2). What if we rewrite 3/5 as an equivalent fraction 6/10 and use the main idea of the Lesson, that 6/10 and 0.6 mean the same thing? Moreover in Grade 4, students did learn to rewrite fractions as decimals this way (for example, 3/4 = 3  25/4  25 = 75/100 = 0.75; G4 Topic 13 TE p. 343). Opportunities to develop mathematics are not taken; frequent rule-based approach (G3, Lesson 5-6; G4, Lesson 4-2; G5 Lesson 1-2, Topic 11 Math Background). G1: The Math Background for Topic 5 (TE 161A) is clear and helpful; also the note about the role of the count-all strategy (G1 Topic 1 TE p. 1B). In Lesson 5-8 TE p. 191A, the problem posed is to find the total number of books in three stacks of books: the stacks have 2 books, 4 books, and 2 books, respectively. The facilitation note says, “Give children time to discuss possible strategies for adding three numbers. [Add two stacks first, then add the number in the third stack.]” But what should the teacher do if a struggling student prefers to count all 8 books? Rating for Section III, Alignment in Standards for Mathematical Practice _X_Far from aligned or infeasible to modify to reach alignment (0 pts) 29 Program Reviewed: EnVisionMATH Common Core 2012 ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section IV. Support for All Students Materials include evidence that teachers/ students are reasonably able to complete the core content within a regular school year. Materials include evidence of all students having the opportunity to work with and meet grade-level standards. Support for English Language Learners and other special populations is thoughtful and helps those students meet the same Standards as all other students. Design of lessons attends to the needs of a variety of learners (for example, using multiple representations, deconstructing/reconstructing the language of problems, providing suggestions for addressing common student difficulties). Materials include regular, balanced assessments that measure progress; valid recommendations are provided for how to address results from assessments for students who show lack of mastery as well as for students who demonstrate proficiency. Guiding questions: Are problems posed with carefully considered language? How strong, and how up-to-date, is the research that informs the supports provided to teachers who have students who are English Language Learners? Do the teacher materials or other components describe a detailed approach and framework for the way the materials support English Language Learners? Do the materials make systematic use of a number of productive language routines? Are the particular routines chosen for a given lesson well matched to the mathematical task at hand? Evidence relevant to these criteria Grade 6 in particular is heavily loaded because of pervasive review and repetition, the choice to cover integer arithmetic (G6 Lesson 10-4, Lesson 10-5, Lesson 10-6, Lesson 10-7) and off-grade geometry. There are suggestions for supporting ELL students at some points of Lessons (G4 Lesson 15-4 p. 409A). Artwork in problems is sometimes helpful as scaffolding (G4 Lesson 5-6 Interactive Learning); other times, it is just cartoons. The suggestions in the Reading Comprehension and Problem Solving notes could be helpful for ELL students, but are not positioned as such. Likewise, some Test-Taking Tips could be helpful for ELL students and are not positioned as such (G4 Lesson 5-1 TE p. 117). 30 Program Reviewed: EnVisionMATH Common Core 2012 Some post-Lesson suggestions for Intervention Could have made for a richer Lesson had they happened then (G4 TE p. 117B). Some Enrichment Leveled Homework is off-topic (G4 TE p. 117B). Rating for Section IV, Supports for All Students __Far from aligned for all students or infeasible to modify to alignment for all students (0 pts) _X_Nearing alignment for all students and straightforwardly modifiable to alignment for all students (1 pt) ___Aligned for all students and straightforwardly modifiable to better alignment for all students (2 pts) ___Richly aligned for all students, perhaps after minor modification (3 pts) Section V. Fit to Your District Program is easy to learn and implement given your resources, personnel, and history to allow all students to meet grade-level standards. Evidence relevant to these criteria Program fits or can fit into your existing school and community culture. Rating for Section V, Fit to your District ___Not suited ___Could work ___Well suited Total Score (Sections II–IV): 1 pt 31 Program is more affordable relative to others that are equally effective and appropriate to your circumstance. Appendix B: Materials Alignment Criteria as Applied to Existing Instructional Materials (Pre-Algebra) 32 Mathematics Instructional Materials Alignment Review (Pre-Algebra) (Based on Key Criteria of the Instructional Materials Evaluation Tool (IMET)) Program Reviewed: Holt Pre-Algebra (2008) Reviewer instructions: 1. First, carry out the Preliminary Alignment Check in Section I. If the program does not pass the Preliminary Alignment Check, then it is Far From Aligned. 2. If, and only if, the program passes the Preliminary Alignment Check, review the program against the remaining criteria in Sections II–V. For each Section, describe evidence as indicated, and then assign a holistic rating using the rubric provided. 3. Based on the results of step (2), the program will earn a total number of alignment points. The total number of points can be used to identify large differences in degree of alignment between programs that merit consideration having passed the Preliminary Alignment Check. For programs with identical point totals, trends in the evidence recorded in the rubric can help to distinguish qualitatively between greater and lesser alignment. Note: Citations to the program (for example, “Lesson 5-7”, “Hands-On Lab 12A”, etc.) are particular instances of observations; they aren’t necessarily the only instances. Any single item noted in the evidence might or might not break the alignment of the program; holistic ratings take into account this evidence as part of the total picture. 33 Program Reviewed: Holt Pre-Algebra (2008) Section I. Preliminary Alignment Check Note 1: If the current version of the program has been reviewed by EdReports at www.edreports.org for all grades K–5 and/or for all grades 6–8, and if the program has received at least 12 points for “Gateway 1” in every grade reviewed, then the program can be considered to pass the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 2: If the current version of the program has been reviewed by EdReports at www.edreports.org for any grade, and if the program has received fewer than 8 points for “Gateway 1” in any grade reviewed, then the program can be considered to fail the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 3: If neither Note 1 nor Note 2 applies, then the program must be reviewed against the criteria in the Preliminary Alignment Check (Table I.1). Note 4: All materials passing the Preliminary Alignment Check must continue on to Sections II–V and be reviewed according to the criteria listed in those sections. There are five criteria in the Preliminary Alignment Check (Table I.1). If any criterion isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. First, describe trends in the evidence for these criteria; then, circle Yes/No to indicate whether each criterion is met. Table I.1. Criteria for Preliminary Alignment Check. See IMET pp. 4–11 for guiding questions. Metric 1: Materials reflect the basic architecture of Metric 2AB: Students and teachers using the the Standards by not assessing the topics listed materials as designed devote the large majority of below* before the grade level indicated. time to the Major Work of the grade. Supporting Work also enhances focus and coherence simultaneously by also engaging students in the Major Work of the grade. No rating Met (circle one): Yes No Met (circle one): Yes No Evidence relevant to these criteria Metric 2CD: Materials follow the grade-by-grade progressions in the Standards. Content from previous or future grades does not unduly interfere with on-grade-level content. Lessons that only include mathematics from previous grades are clearly identified as such to the teacher. Met (circle one): Yes No Metric 1 is not evaluated, because the grade level of the Pre-Algebra course is to some extent indeterminate: it might be considered grade 7 in the sense that the K–6 program reviewed ends at grade 6, or it might be considered grade 8 in the sense that it is an alternative to Algebra 1 in grade 8. The lack of a rating for Metric 1 does not affect the review, because considering the other metrics, the materials would not pass the Preliminary Alignment Check regardless of the rating for Metric 1. There is virtually zero correspondence between the topics in this course and the topics at any given grade level in the adopted state standards. For example,  The following topics from the course appear in state standards for High School (this list is not exhaustive): 34 Program Reviewed: Holt Pre-Algebra (2008) High School Topic Arithmetic and geometric sequences Quadratic, cubic, and exponential functions Polynomials Volume of pyramids, cones, and spheres Permutations and combinations Two-variable inequalities Where in the course Lesson 12-1, Lesson 12-2, Hands-On Lab 12A, Lesson 12-3, Chapter 12 Mid-Chapter Quiz, Technology Lab Generate Arithmetic and Geometric Sequences, Chapter 12 Test Lesson 12-3, Lesson 12-4, Lesson 12-6, Lesson 12-7, Technology Lab 12B, Problem Solving on Location: Alabama, Math-ables Squared Away, Chapter 12 Test, Chapter 12 Performance Assessment Chapter 13 (all) Lesson 6-7, Lesson 6-10, Chapter 6 Test Lesson 9-6, Technology Lab Permutations and Combinations, Chapter 9 Test Lesson 11-6, Problem Solving on Location: Missouri, Technology Lab Graph Inequalities in Two Variables, Chapter 11 Test  Topics are present in the course but not mentioned in the standards for any grade, let alone the middle grades (Lesson 5-9; all of Chapter 14).  The topics that do correspond to the middle grades in state standards may be found throughout the entire middle-school grade band in the standards (Chapter 1 includes material from grades 5, 6, 7, and 8 in the standards; Chapter 5 includes material from grades 7, 8, and High School).  Some topics from the middle grades standards are not covered (distance between points in the coordinate plane, patterns of association in bivariate categorical data, properties of rigid motions). Review material is not identified as such. The teacher-facing objective of Lesson 3-5 is, “Students add and subtract fractions with unlike denominators,” which a grade 5 expectation in the standards; the Guided Instruction reads, “In this lesson, students learn to add and subtract fractions with unlike denominators...,” and there is no suggestion that students will have worked with these problems by middle school. The situation is similar for measuring angles and other content from Chapter 5. 35 Program Reviewed: Holt Pre-Algebra (2008) *Topics for Metric 1. (No other topics should be added to this list. Note that other topics in the standards are addressed in criteria 2A–C.) • Probability, including chance, likely outcomes, probability models. (Introduced in the standards in grade 7) • Statistical distributions, including center, variation, clumping, outliers, mean, median, mode, range, quartiles; and statistical association or trends, including two-way tables, bivariate measurement data, scatter plots, trend line, line of best fit, correlation. (Introduced in the standards in grade 6) • Coordinate transformations or formal definition of congruence or similarity. (Introduced in the standards in grade 8) • Symmetry of shapes, including line/reflection symmetry, rotational symmetry. (Introduced in the standards in grade 4) If any criterion in Table I.1 isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. Preliminary Alignment Check Result (circle one): Passed Far from Aligned Section II. Alignment: Concepts, Fluency, Applications Table II.1. Criteria for alignment in concepts, fluency, and applications. See IMET pp. 14–19 for guiding questions. Metric AC-1A. The materials support the Metric AC-1B. The materials are designed so that Metric AC-1C. The materials are designed so that development of students’ conceptual students attain the fluencies and procedural skills teachers and students spend sufficient time understanding of key mathematical concepts, required by the Standards. working with applications, without losing focus on especially where called for in specific content the Major Work of each grade. standards or cluster headings. Evidence relevant to these criteria Rating for Section II, Alignment in Concepts, Fluency, Applications ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) 36 Program Reviewed: Holt Pre-Algebra (2008) Section III. Alignment: Standards for Mathematical Practice Metric AC-2B: Tasks and assessments of student learning are designed to provide evidence of students’ proficiency in the Standards for Mathematical Practice. Metric AC-2C: Materials support the Standards’ emphasis on mathematical reasoning. Materials call for students to produce mathematical arguments. Materials don’t teach multiple methods or strategies just for the sake of variety, but instead support the teacher in using that variety to draw mathematical connections between methods for the benefit of student learning. Metric AC-2D: The richest Modeling experiences found in each grade 6–8 reflect significant practices of mathematical Modeling. (Rich Modeling tasks need not be prevalent in the materials to meet this criterion.) Metric AC-2E. Lessons alert the teacher to opportunities for students to discuss important mathematics, and provide sufficient teacher support so that discussions are likely to be successful and build students’ understanding. Program is educative for teachers when combined with professional development in mathematics. Guiding questions Considering the variety of tasks and assessments provided (observation checklists, portfolio recommendations, performance tasks, tests and quizzes), do students have opportunities to demonstrate proficiency with each of the Standards for Mathematical Practice over the course of the year? Does emphasizing the Standards for Mathematical Practice tend to open up room for students to work on content not required by the standards? Are students challenged to make sense of word problems, or are word problems always so scaffolded or modeled that students learn recipes for them? Is Modeling incorrectly portrayed as being about such things as using manipulatives, connecting representations, or showing one’s thinking? Find, or have the publisher provide, the richest Modeling experiences in grades 6, 7, and 8. How prominent are Modeling practices required in these tasks? Are calculators used to avoid showing proficiency with calculation and procedures? Or are students asked to be strategic about technology? Are students pushed to improve the precision of their mathematical statements? Are grades K–5 students supported to look for and express regularity in repeated reasoning about the addition table, the multiplication table, the properties of operations, the relationship between addition and subtraction or multiplication and division, and the place value system? Are grades 6–8 students supported to look for and express regularity in repeated reasoning about proportional relationships and linear functions? Are students merely asked to guess patterns and extend them term by term, or are they asked to draw mathematical conclusions about patterns? Evidence relevant to these criteria Rating for Section III, Alignment in Standards for Mathematical Practice 37 Program Reviewed: Holt Pre-Algebra (2008) ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section IV. Support for All Students Materials include evidence that teachers/ students are reasonably able to complete the core content within a regular school year. Materials include evidence of all students having the opportunity to work with and meet grade-level standards. Support for English Language Learners and other special populations is thoughtful and helps those students meet the same Standards as all other students. Design of lessons attends to the needs of a variety of learners (for example, using multiple representations, deconstructing/reconstructing the language of problems, providing suggestions for addressing common student difficulties). Materials include regular, balanced assessments that measure progress; valid recommendations are provided for how to address results from assessments for students who show lack of mastery as well as for students who demonstrate proficiency. Guiding questions: Are problems posed with carefully considered language? How strong, and how up-to-date, is the research that informs the supports provided to teachers who have students who are English Language Learners? Do the teacher materials or other components describe a detailed approach and framework for the way the materials support English Language Learners? Do the materials make systematic use of a number of productive language routines? Are the particular routines chosen for a given lesson well matched to the mathematical task at hand? Evidence relevant to these criteria Rating for Section IV, Supports for All Students __Far from aligned for all students or infeasible to modify to alignment for all students (0 pts) ___Nearing alignment for all students and straightforwardly modifiable to alignment for all students (1 pt) ___Aligned for all students and straightforwardly modifiable to better alignment for all students (2 pts) 38 Program Reviewed: Holt Pre-Algebra (2008) ___Richly aligned for all students, perhaps after minor modification (3 pts) Section V. Fit to Your District Program is easy to learn and implement given your resources, personnel, and history to allow all students to meet grade-level standards. Evidence relevant to these criteria Program fits or can fit into your existing school and community culture. Rating for Section V, Fit to your District ___Not suited ___Could work ___Well suited Total Score (Sections II–IV): _NA 39 Program is more affordable relative to others that are equally effective and appropriate to your circumstance. Appendix C: Materials Alignment Criteria as Applied to Existing Instructional Materials (Algebra 1, Geometry, and Algebra 2) 40 Mathematics Instructional Materials Alignment Review (High School) (Based on Key Criteria of the Instructional Materials Evaluation Tool (IMET)) Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Reviewer instructions: 4. First, carry out the Preliminary Alignment Check in Section I. If the program does not pass the Preliminary Alignment Check, then it is Far From Aligned. 5. If, and only if, the program passes the Preliminary Alignment Check, review the program against the remaining criteria in Sections II–V. For each Section, describe evidence as indicated, and then assign a holistic rating using the rubric provided. 6. Based on the results of step (2), the program will earn a total number of alignment points. The total number of points can be used to identify large differences in degree of alignment between programs that merit consideration having passed the Preliminary Alignment Check. Note: The program under review failed the Preliminary Alignment Check, but Sections II–IV were completed to generate additional information about the program. Note: Citations to the program (for example, “A2 TE p. 642”, etc.) are particular instances of observations; they aren’t necessarily the only instances. Any single item noted in the evidence might or might not break the alignment of the program; holistic ratings take into account this evidence as part of the total picture. 41 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Section I. Preliminary Alignment Check Note 1: If the current version of the program has been reviewed by EdReports at www.edreports.org, and if the program has received at least 15 points for “Gateway 1,” then the program can be considered to pass the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 2: If the current version of the program has been reviewed by EdReports at www.edreports.org, and if the program has received fewer than 10 points for “Gateway 1,” then the program can be considered to fail the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 3: If neither Note 1 nor Note 2 applies, then the program must be reviewed against the criteria in the Preliminary Alignment Check (Table I.1). Note 4: All materials passing the Preliminary Alignment Check must continue on to Sections II–V and be reviewed according to the criteria listed in those sections. There are five criteria in the Preliminary Alignment Check (Table I.1). If any criterion isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. First, describe trends in the evidence for these criteria; then, circle Yes/No to indicate whether each criterion is met. Table I.1. Criteria for Preliminary Alignment Check. See IMET pp. 5–8 for guiding questions. Metric 1: In any single course, students spend a majority of their time on Metric 2: Materials base courses material widely applicable to, and prerequisite for, a range of college on the content specified in the majors, postsecondary programs and careers. Student work in Geometry Standards. involves significant work with applications/Modeling and problems that use algebra skills. There are problems at a level of sophistication appropriate to high school (beyond mere review of middle school topics) that involve the application of knowledge and skills from grades 6–8. Met (circle one): Yes No Evidence relevant to these criteria Met (circle one): Yes No Metric 3: The progression of content from course to course is coherent. Lessons that only include mathematics from previous grades or courses are clearly identified as such to the teacher. Met (circle one): Yes No The design of the program as a whole places a low ceiling on the depth of Modeling in the program (see Section III). Even in the A1 course, where the middle grades are recent history, the program lacks an evident design to include high-school-level application problems requiring only knowledge and skills from grades 6–8; and such problems appear to be sparse in the materials. In the G course, algebraic applications have a weak presence on the End-of-Course Assessment, and functional thinking about geometric quantities (G TE p. 628 problem 43) rarely appears in the course. While algebra is frequently present in the G course, sometimes it takes the form of numerous problems without any Modeling component (G TE pp. 372, 373, 398): in such cases, algebra is being used as a tool for geometry, as opposed to algebra and geometry being used together as a tool for Modeling. 42 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Some topics are covered in the A2 course that aren’t required by the standards—for example, linear programming (Lesson 3-4) and absolute value inequalities (Lesson 1-6). In the A2 course, material on linear functions and linear systems (much of it retrograde for an A2 course) overlaps quite heavily with the A1 course; compare the two courses’ Chapter titles, Lesson titles, and specific problems (A1 TE p. 301 Problem 1 and A2 TE p. 69 Problem 2). In the G course, there seems to be no recognition of, or design for, the possibility that students have seen the Pythagorean theorem or the distance formula as early as grade 8 (G TE p. 52). The placement of trigonometric ratios in the G course and trigonometric functions in the A2 course, as well as the episode of review of quadratic equations in the G course, indicate minimal levels of vertical coherence in these areas, but the tasks in the G course do little to reinforce students’ understanding or skill with quadratic equations or functions. If any criterion in Table I.1 isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. Preliminary Alignment Check Result (circle one): Passed Far from Aligned Section II. Alignment: Concepts, Fluency, Applications Table II.1. Criteria for alignment in concepts, fluency, and applications. See IMET pp. 11–16 for guiding questions. Metric AC-1A. The materials support the Metric AC-1B. The materials are designed so that Metric AC-1C. The materials are designed so that development of students’ conceptual students attain the fluency and procedural skills teachers and students spend sufficient time understanding of key mathematical concepts, required by the Standards. working with applications, without losing focus on especially where called for in specific content material widely applicable to, and prerequisite for, standards or cluster headings. a range of college majors, postsecondary programs and careers. Evidence relevant to these criteria The treatment of the distributive property (A1 TE p. 46) doesn’t take into account, or communicate to students, that they have been using the property ceaselessly ever since elementary school, for example in working with area models of multiplication or when performing calculations such as 3  2¼ = 6 + ¾, or 7  111 = 700 + 70 + 7. Instead, a handful of artificial integer examples are given, missing an opportunity to ground formal algebra in the familiar. A problem thoughtfully relating the distributive property to calculation strategies (499  5) appears only as the Lesson ends (A1 TE p. 49 problem 7). 43 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Big Ideas are sometimes not so big (G TE p. 543A), and Essential Questions are sometimes unhelpfully put. For example, “How are radical expressions represented?” (A1 TE p. 612) immediately conjures the answer “with radicals.” Sometimes, essential concepts are contained in the Concept Bytes; it might not be clear to teachers when skipping a Concept Byte would defeat the corresponding Lesson (G TE p. 506). When students first encounter the term imaginary, they are encouraged to think of its everyday meaning, as follows: “Something is imaginary if it has no factual reality” (A2 TE p. 191). Later, teachers are warned about “the common misconception that imaginary numbers do not exist.” Might the misconception be coming from the curriculum itself? In the G course, the diagram provided alongside the definition of congruent angles (G TE p. 27) misses an opportunity to show a case where not all rays have equal lengths—which might have helped teachers to address a common misunderstanding that had been noted only two pages earlier in the Math Background (G TE p. 27). A possible teaching moment is sacrificed. In the A1 course, instead of students talking about a set of data as symmetrically distributed, uniformly distributed, or skewed, students instead classify histograms as symmetric histograms, uniform histograms, or skewed histograms (A1 TE p. 733). This typology of graphics divorces the discussion from the data and the context. Some of the Take Note boxes present Key Concepts. In some cases, it is unclear in what sense these Key Concepts qualify as concepts. Do we consider that “The distance between two points A(x1, y1) and B(x2, y2) is d = Sqrt[(x2 – x1)2 + (y2 – y1)2]” qualifies as a concept? (G TE p. 42) Another questionable instance of a concept is the Concept Summary called “Solving Equations” (A1 TE p. 105). The volume of practice provided causes well-made problems to fade into the woodwork (A1 TE p. 607 problem 24; A1 TE p. 50 problem 21; G TE p. 8 problem 24; G TE p. 877 problem 10; A2 TE p. 600 problem 39), and heavy scaffolding in what would otherwise be a valuable task can deny students opportunities to synthesize the material in the Lesson (A2 TE p. 643 problem 42). The materials include many application problems; in the main these are not evasive of material widely applicable to, and prerequisite for, a range of college majors, postsecondary programs and careers. Rating for Section II, Alignment in Concepts, Fluency, Applications 44 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core X Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section III. Alignment: Standards for Mathematical Practice Metric AC-2B: Tasks and assessments of student learning are designed to provide evidence of students’ proficiency in the Standards for Mathematical Practice. Metric AC-2C: Materials support the Standards’ emphasis on mathematical reasoning. Materials call for students to produce mathematical arguments. Materials don’t teach multiple methods or strategies just for the sake of variety, but instead support the teacher in using that variety to draw mathematical connections between methods for the benefit of student learning. Metric AC-2D: The richest Modeling experiences found in each course reflect highly developed practices of mathematical Modeling. (Rich Modeling tasks need not be prevalent in the materials to meet this criterion.) Metric AC-2E. Lessons alert the teacher to opportunities for students to discuss important mathematics, and provide sufficient teacher support so that discussions are likely to be successful and build students’ understanding. Program is educative for teachers when combined with professional development in mathematics. Guiding questions Considering the variety of tasks and assessments provided (observation checklists, portfolio recommendations, performance tasks, tests and quizzes), do students have opportunities to demonstrate proficiency with each of the Standards for Mathematical Practice over the course of the year? Does emphasizing the Standards for Mathematical Practice tend to open up room for students to work on content not required by the standards? Are students challenged to make sense of word problems, or are word problems always so scaffolded or modeled that students learn recipes for them? Find, or have the publisher provide, the richest Modeling experiences in each course. How prominent are Modeling practices required in these tasks? Do students use graphing calculators, spreadsheets, etc. only on command, or are students asked to be strategic about technology? Are students pushed to improve the precision of their mathematical statements? Do students perform algebraic manipulations only on command, or do students look for structure and rewrite expressions for a purpose? Are students supported to look for and express regularity in repeated reasoning when working with functions? Evidence relevant to these criteria 45 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core The design of the Common Core Performance Tasks, one of which is presented in each Chapter Opener, is that students chip away at them throughout the Chapter. While this helps students to apply what they are learning in the Lessons—the task to some extent structures the Chapter experience—the approach also turns the tasks into highly scaffolded experiences, placing a ceiling on how students engage in both MP.1 and MP.4. The localization of each Common Core Performance Task to a Chapter with a specific topic focus also essentially dictates what mathematics to use; indeed, the commentary to the student typically removes any doubt (A2 TE p. 133: “you’ll use a system of linear equations to model the three criteria”). Although an “On Your Own” extension is always provided, the extension problem is of the same kind; and in a busy course, one may doubt that “On Your Own” extensions happen frequently. In one Common Core Performance Task (A1 TE p. 233) in which linearity is unrealistic given the context, a student engaging actively in the Modeling practice might have chosen a linear model anyway, provided the results of doing so could be useful in the situation. But instead of students choosing the mathematics to use, the task design avoids those Modeling practices by (1) providing an artificial, perfectly linear data set for the problem; and (2) telling the student directly that this perfect linear pattern will continue. Artificiality in the program’s application problems can be extreme (G TE p. 883 problem 64). Quantities in tasks that could have been researched online or estimated, thereby increasing students’ quantitative literacy, are instead provided for students (A2 TE p. 133) or abstracted away (G TE 761, which asks only for area and not, say, cost). Even in Common Core Performance Tasks, Modeling amounts to little more than the art of solving lengthy versions of schoolbook word problems, with heavy signposting and scaffolding. The embodiment of MP.5 in the Common Core Performance Tasks is often the opposite of using tools strategically (A2 TE p. 763: “You’ll find the inverse of a 33 matrix using a calculator”; A1 TE p. 545: “You’ll use a graphing calculator or other graphing utility...”). Note also the calculator icon that short-circuits several mathematical practices in a non-routine problem like x + 3 = 3x, for which symbolic solution methods aren’t available to students (A1 TE p. 457 problems 30, 31). There are no problems on the End-of-Course Assessments that resemble the performance tasks, either scaffolded or unscaffolded. (Among the closest cases would be A2 TE p. 969 problem 54.) The G course stresses precision in the use of terms and notation. The opening portrayal of the deductive system (G TE p. 13) is problematic however, saying “A postulate or axiom is an accepted statement of fact.” Facts are usually thought of as empirical truths, not agreed-upon starting points in a communal enterprise of logical deduction. Soon afterwards (G TE p. 18), an inapposite analogy between undefined terms and hard-todefine words in English is offered. Throughout the program there are tasks flagged Reasoning. Getting the answers to these problems usually requires that some reasoning occur, but it should be said that in relation to MP.3 (constructing and critiquing arguments), the suggested answers can paint an impoverished picture of a 46 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core mathematical argument. For example, on the question (A1 TE p. 791 problem 19) of whether the formula P(A or B) = P(A) + P(B) – P(A and B) would give the correct answer for mutually exclusive events, the answer provided to the teacher (“Yes; for mutually exclusive events P(A and B) = 0”) leaves the reader to connect the dots: since P(A and B) = 0, the general formula becomes P(A or B) = P(A) + P(B), which is the formula for mutually exclusive events. MP.3 isn’t about happening upon the key insight that breaks the back of a problem; it’s about fashioning arguments. Some problems come nearer to the practices in question (A2 TE p. 642 problem 6). Answers to Reasoning problems don’t always sketch an argument of any sort (G TE p. 39 problem 41) and do not always avoid the reasoning error of arguing the converse of the proposition requested (G TE p. 515 problem 23). Instead of being an opportunity to look for and make use of structure, a problem on an End-of-Year Assessment commands the student to “Use factoring” (TE A1 p. 797). Were that directive removed, the problem would engage students in MP.7. Vocabulary checks (G TE p. 455) don’t always clarify terms and delineate their precise scope, which is an important part of disciplinary vocabulary in mathematics. Teaching supports for classroom discourse in the program usually take the form of suggested questions to ask the class; these guiding questions frequently have closed answers. Even good questions about a problem at hand could be countermanded by a “Step 1, Step 2, Step 3” instruction or other printed guidance. And even when suggested questions are mathematically meaty, there isn’t a robust discourse-structure designed into the program to bring rich classroom conversations reliably into being. “Describe the differences between a linear function and an exponential function” (A1 TE p. 456) ought to be a meaty discussion among students, but the teacher isn’t given support to host such a conversation (and the idea is positioned as merely vocabulary). In a problem cited above (G TE p. 406), the guiding questions lead the class through the required argument, but given the designed flow of the Lesson, no student need ever speak up to provide that argument in toto (MP.3). When a “Take Note” for the teacher is positioned as “Error Prevention” (G TE p. 14), a possible learning moment for students is conveyed as students just being wrong. The teacher supports provided can be counterproductive in other ways. For example, the directive to “Remind [the students] that multiplication is repeated addition” (A2 TE p. 5) treats students as mental third-graders and would render baffling such products as (⅛)(1), Sqrt[2] or 1.1ex that have no apparent interpretation as sums. In the Math Background for Solving Equations (A1 TE pp. 79A, 79B), teachers are told about “Solving Proportions” like 12/9 = 8/x , in particular told that in these cases “students have choices,” namely “They can multiply both sides of the equation by the least common multiple (LCM) or use the Cross Products Property.” The advice to use the LCM seems to suggest that students studying this program are carefully sheltered from non-whole-number problems; and neither piece of advice encourages teachers to view “proportions” as the equations they are. For example, what if instead of cross-multiplying, we were to divide both sides of the equation 12/9 = 8/x by 8? The result is 1/6 = 1/x, from which the solution can be read by inspection. Yet another choice, inefficient but mathematically productive, could be to use the graphical method that was outlined on the very page that faces the one with the equation on it. This graphical method was alas carelessly explained, the explanation suggesting incorrectly that a point, rather than a number, solves the equation 2(2x  1) = 2(x  2). Similarly, the note on measuring segments (G TE p. 3B) doesn’t distinguish between points and numbers. Fortunately, the relevant Lesson gets this right (G TE p. 20)— but it ought to be right for the teacher too. 47 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Thus, the educativeness of the curriculum is mixed. On the plus side are the notes for the teacher about why topics on the Get Ready! assessments were included; the Essential Questions (where coherent); the existence of guiding questions in the margins of the Guided Instruction phase; common student errors flagged in Math Background; and the better examples of other marginalia, such as the “Take Note” (G TE p. 53) that invites the teacher to draw students’ attention to the fact that planes are described with reference to lines, and lines with reference to points. Some evidence already provided in Section II also indicates weaknesses in the educativeness of the program. Rating for Section III, Alignment in Standards for Mathematical Practice X Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section IV. Support for All Students Materials include evidence that teachers/ students are reasonably able to complete the core content within a regular school year. Materials include evidence of all students having the opportunity to work with and meet grade-level standards. Support for English Language Learners and other special populations is thoughtful and helps those students meet the same Standards as all other students. Design of lessons attends to the needs of a variety of learners (for example, using multiple representations, deconstructing/reconstructing the language of problems, providing suggestions for addressing common student difficulties). Materials include regular, balanced assessments that measure progress; valid recommendations are provided for how to address results from assessments for students who show lack of mastery as well as for students who demonstrate proficiency. Guiding questions: Are problems posed with carefully considered language? How strong, and how up-to-date, is the research that informs the supports provided to teachers who have students who are English Language Learners? Do the teacher materials or other components describe a detailed approach and framework for the way the materials support English Language Learners? Do the materials make systematic use of a number of productive language routines? Are the particular routines chosen for a given lesson well matched to the mathematical task at hand? Evidence relevant to these criteria 48 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core The extent of the material is so great, especially in the G and A2 courses, that in a single academic year, the material might only be learned and taught superficially. The choice to include matrices (A2 Chapter 12), the Law of Sines (G Lesson 8-5), the Law of Cosines (G Lesson 8-6), permutations and combinations (G Lesson 13-3), and expected value (G Lesson 13-3, Lesson 13-7)—all of which are (+) material in the standards—was a choice to make the coverage burden heavier. So was the choice to include topics not required by the standards at all (see Section I). Finally, note that the (+) material (designated as beyond the college- and career-ready line in the standards) is not accurately indicated as such in the materials. Correlations to the standards omit the (+) symbol when citing content standards like N-VM.A.1(+) or G-SRT.D.10(+) (A2 TE p. 762; G TE p. 527), which sends teachers an inaccurate signal about the demands of college and career readiness. Tasks are not always free of unnecessary language complexity (A2 TE p. 133). ELL support is available via the Online Teacher Resource Center, often positioned as vocabulary work (A1 TE p. 100A; A2 TE p. 306A), instead of providing ELL students with content-focused language work. Lessons include notes to support teachers of ELL students, but these notes typically appear after the end of the Lesson flow. Systematic use of language routines in the Lessons themselves is not evident. Multiple representations are used, including for ELL students (G TE T21), but the lessons don’t appear systematically to use them to support students who struggle in other specific, predictable ways, such as systematically using technology to solve equations for students who are still developing the symbolic fluency to solve algebraically/formally. The Assignment Guides for the Practice components of the Lessons suggest assigning different problems to students at different levels (Basic, Average, Advanced). Absent appropriate instructional leadership, this could potentially result in the various classrooms within a school enacting somewhat different curricula, or the various schools in a district enacting somewhat different curricula. (It does appear to be a policy of the materials that the Basic selections extend beyond the initial skill practice of “section A.”) There are pre-assessment moments (Get Ready!) and post-assessment moments (Lesson Check, Lesson Quiz, Chapter Test, End-of-Course Assessment). As is common in the genre, an intense amount of energy is devoted to differentiating instruction, even though some “Reteaching” resources would likely help most students, and even though some Enrichment resources, Activities, Games, and Puzzles could be fruitful and motivating for more than just the “high kids.” The Assess & Remediate cycle does not appear to have been augmented or offset by a substantial design effort to address inevitable unfinished learning within the main Lesson flow. In the items for the Entry-Level Assessment for the Geometry course (G TE T69), there is significant algebra coverage. Will the effect of this be to funnel weak Algebra 1 students into a “low” Geometry classroom, or into a lower high school track altogether, when in fact the algebra skills used in geometry might be reinforced there as needed, and practiced in new and potentially more visually engaging contexts? Program designers and those who implement programs might reexamine their assumed ability to diagnose, level, and place students using brief measures (Oakes, 2005). 49 Program Reviewed: Pearson Algebra 1 Common Core / Geometry Common Core / Algebra 2 Common Core Rating for Section IV, Supports for All Students X Far from aligned for all students or infeasible to modify to alignment for all students (0 pts) ___Nearing alignment for all students and straightforwardly modifiable to alignment for all students (1 pt) ___Aligned for all students and straightforwardly modifiable to better alignment for all students (2 pts) ___Richly aligned for all students, perhaps after minor modification (3 pts) Section V. Fit to Your District Program is easy to learn and implement given your resources, personnel, and history to allow all students to meet grade-level standards. Evidence relevant to these criteria Program fits or can fit into your existing school and community culture. Rating for Section V, Fit to your District ___Not suited ___Could work ___Well suited Total Score (Sections II–IV): 0 pts 50 Program is more affordable relative to others that are equally effective and appropriate to your circumstance. Appendix D: Blank Materials Alignment Criteria for Detroit Public School Community District Use in Procurement (Grades K–8) These criteria are adapted from the Instructional Materials Evaluation Tool (IMET),vi which is part of an authoritative alignment toolkit applicable to the Michigan mathematics standards. 51 Mathematics Instructional Materials Alignment Review (Grades K–8) (Based on Key Criteria of the Instructional Materials Evaluation Tool (IMET)) Program Reviewed: Reviewer instructions: 1. First, carry out the Preliminary Alignment Check in Section I. If the program does not pass the Preliminary Alignment Check, then it is Far From Aligned. 2. If, and only if, the program passes the Preliminary Alignment Check, review the program against the remaining criteria in Sections II–V. For each Section, describe evidence as indicated, and then assign a holistic rating using the rubric provided. 3. Based on the results of step (2), the program will earn a total number of alignment points. The total number of points can be used to identify large differences in degree of alignment between programs that merit consideration having passed the Preliminary Alignment Check. For programs with identical point totals, trends in the evidence recorded in the rubric can help to distinguish qualitatively between greater and lesser alignment. 52 Program Reviewed: Section I. Preliminary Alignment Check Note 1: If the current version of the program has been reviewed by EdReports at www.edreports.org for all grades K–5 and/or for all grades 6–8, and if the program has received at least 12 points for “Gateway 1” in every grade reviewed, then the program can be considered to pass the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 2: If the current version of the program has been reviewed by EdReports at www.edreports.org for any grade, and if the program has received fewer than 8 points for “Gateway 1” in any grade reviewed, then the program can be considered to fail the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 3: If neither Note 1 nor Note 2 applies, then the program must be reviewed against the criteria in the Preliminary Alignment Check (Table I.1). Note 4: All materials passing the Preliminary Alignment Check must continue on to Sections II–V and be reviewed according to the criteria listed in those sections. There are five criteria in the Preliminary Alignment Check (Table I.1). If any criterion isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. First, describe trends in the evidence for these criteria; then, circle Yes/No to indicate whether each criterion is met. Table I.1. Criteria for Preliminary Alignment Check. See IMET pp. 4–11 for guiding questions. Metric 1: Materials reflect the basic architecture of Metric 2AB: Students and teachers using the the Standards by not assessing the topics listed materials as designed devote the large majority of below* before the grade level indicated. time to the Major Work of the grade. Supporting Work also enhances focus and coherence simultaneously by also engaging students in the Major Work of the grade. Met (circle one): Yes No Evidence relevant to these criteria Met (circle one): Yes No Metric 2C: Materials follow the grade-by-grade progressions in the Standards. Content from previous or future grades does not unduly interfere with on-grade-level content. Lessons that only include mathematics from previous grades are clearly identified as such to the teacher. Met (circle one): Yes No *Topics for Metric 1. (No other topics should be added to this list. Note that other topics in the standards are addressed in criteria 2A–C.) • Probability, including chance, likely outcomes, probability models. (Introduced in the standards in grade 7) • Statistical distributions, including center, variation, clumping, outliers, mean, median, mode, range, quartiles; and statistical association or trends, including two-way tables, bivariate measurement data, scatter plots, trend line, line of best fit, correlation. (Introduced in the standards in grade 6) • Coordinate transformations or formal definition of congruence or similarity. (Introduced in the standards in grade 8) • Symmetry of shapes, including line/reflection symmetry, rotational symmetry. (Introduced in the standards in grade 4) 53 Program Reviewed: If any criterion in Table I.1 isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. Preliminary Alignment Check Result (circle one): Passed Far from Aligned Section II. Alignment: Concepts, Fluency, Applications Table II.1. Criteria for alignment in concepts, fluency, and applications. See IMET pp. 14–19 for guiding questions. Metric AC-1A. The materials support the Metric AC-1B. The materials are designed so that Metric AC-1C. The materials are designed so that development of students’ conceptual students attain the fluencies and procedural skills teachers and students spend sufficient time understanding of key mathematical concepts, required by the Standards. working with applications, without losing focus on especially where called for in specific content the Major Work of each grade. standards or cluster headings. Evidence relevant to these criteria Rating for Section II, Alignment in Concepts, Fluency, Applications ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section III. Alignment: Standards for Mathematical Practice Metric AC-2B: Tasks and assessments of student learning are designed to provide evidence of students’ Metric AC-2C: Materials support the Standards’ emphasis on mathematical reasoning. Materials call for students to produce Metric AC-2D: The richest Modeling experiences found in each grade 6–8 reflect significant practices of mathematical Modeling. (Rich 54 Metric AC-2E. Lessons alert the teacher to opportunities for students to discuss important mathematics, and provide sufficient teacher Program Reviewed: proficiency in the Standards for Mathematical Practice. mathematical arguments. Materials don’t teach multiple methods or strategies just for the sake of variety, but instead support the teacher in using that variety to draw mathematical connections between methods for the benefit of student learning. Modeling tasks need not be prevalent in the materials to meet this criterion.) support so that discussions are likely to be successful and build students’ understanding. Program is educative for teachers when combined with professional development in mathematics. Guiding questions Considering the variety of tasks and assessments provided (observation checklists, portfolio recommendations, performance tasks, tests and quizzes), do students have opportunities to demonstrate proficiency with each of the Standards for Mathematical Practice over the course of the year? Does emphasizing the Standards for Mathematical Practice tend to open up room for students to work on content not required by the standards? Are students challenged to make sense of word problems, or are word problems always so scaffolded or modeled that students learn recipes for them? Is Modeling incorrectly portrayed as being about such things as using manipulatives, connecting representations, or showing one’s thinking? Find, or have the publisher provide, the richest Modeling experiences in grades 6, 7, and 8. How prominent are Modeling practices required in these tasks? Are calculators used to avoid showing proficiency with calculation and procedures? Or are students asked to be strategic about technology? Are students pushed to improve the precision of their mathematical statements? Are grades K–5 students supported to look for and express regularity in repeated reasoning about the addition table, the multiplication table, the properties of operations, the relationship between addition and subtraction or multiplication and division, and the place value system? Are grades 6–8 students supported to look for and express regularity in repeated reasoning about proportional relationships and linear functions? Are students merely asked to guess patterns and extend them term by term, or are they asked to draw mathematical conclusions about patterns? Evidence relevant to these criteria Rating for Section III, Alignment in Standards for Mathematical Practice ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) 55 Program Reviewed: Section IV. Support for All Students Materials include evidence that teachers/ students are reasonably able to complete the core content within a regular school year. Materials include evidence of all students having the opportunity to work with and meet grade-level standards. Support for English Language Learners and other special populations is thoughtful and helps those students meet the same Standards as all other students. Design of lessons attends to the needs of a variety of learners (for example, using multiple representations, deconstructing/reconstructing the language of problems, providing suggestions for addressing common student difficulties). Materials include regular, balanced assessments that measure progress; valid recommendations are provided for how to address results from assessments for students who show lack of mastery as well as for students who demonstrate proficiency. Guiding questions: Are problems posed with carefully considered language? How strong, and how up-to-date, is the research that informs the supports provided to teachers who have students who are English Language Learners? Do the teacher materials or other components describe a detailed approach and framework for the way the materials support English Language Learners? Do the materials make systematic use of a number of productive language routines? Are the particular routines chosen for a given lesson well matched to the mathematical task at hand? Evidence relevant to these criteria Rating for Section IV, Supports for All Students __Far from aligned for all students or infeasible to modify to alignment for all students (0 pts) ___Nearing alignment for all students and straightforwardly modifiable to alignment for all students (1 pt) ___Aligned for all students and straightforwardly modifiable to better alignment for all students (2 pts) ___Richly aligned for all students, perhaps after minor modification (3 pts) Section V. Fit to Your District 56 Program Reviewed: Program is easy to learn and implement given your resources, personnel, and history to allow all students to meet grade-level standards. Evidence relevant to these criteria Program fits or can fit into your existing school and community culture. Rating for Section V, Fit to your District ___Not suited ___Could work ___Well suited Total Score (Sections II–IV): _____ 57 Program is more affordable relative to others that are equally effective and appropriate to your circumstance. Appendix E: Blank Materials Alignment Criteria for Detroit Public School Community District Use in Procurement (High School) These criteria are adapted from the Instructional Materials Evaluation Tool (IMET),vii which is part of an authoritative alignment toolkit applicable to the Michigan mathematics standards. 58 Mathematics Instructional Materials Alignment Review (High School) (Based on Key Criteria of the Instructional Materials Evaluation Tool (IMET)) Program Reviewed: Reviewer instructions: 1. First, carry out the Preliminary Alignment Check in Section I. If the program does not pass the Preliminary Alignment Check, then it is Far From Aligned. 2. If, and only if, the program passes the Preliminary Alignment Check, review the program against the remaining criteria in Sections II–V. For each Section, describe evidence as indicated, and then assign a holistic rating using the rubric provided. 3. Based on the results of step (2), the program will earn a total number of alignment points. The total number of points can be used to identify large differences in degree of alignment between programs that merit consideration having passed the Preliminary Alignment Check. 59 Program Reviewed: Section I. Preliminary Alignment Check Note 1: If the current version of the program has been reviewed by EdReports at www.edreports.org, and if the program has received at least 15 points for “Gateway 1,” then the program can be considered to pass the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 2: If the current version of the program has been reviewed by EdReports at www.edreports.org, and if the program has received fewer than 10 points for “Gateway 1,” then the program can be considered to fail the Preliminary Alignment Check. Indicate here if that is the case: ______ Note 3: If neither Note 1 nor Note 2 applies, then the program must be reviewed against the criteria in the Preliminary Alignment Check (Table I.1). Note 4: All materials passing the Preliminary Alignment Check must continue on to Sections II–V and be reviewed according to the criteria listed in those sections. There are five criteria in the Preliminary Alignment Check (Table I.1). If any criterion isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. First, describe trends in the evidence for these criteria; then, circle Yes/No to indicate whether each criterion is met. Table I.1. Criteria for Preliminary Alignment Check. See IMET pp. 5–8 for guiding questions. Metric 1: In any single course, students spend a majority of their time on Metric 2: Materials base courses material widely applicable to, and prerequisite for, a range of college on the content specified in the majors, postsecondary programs and careers. Student work in Geometry Standards. involves significant work with applications/Modeling and problems that use algebra skills. There are problems at a level of sophistication appropriate to high school (beyond mere review of middle school topics) that involve the application of knowledge and skills from grades 6–8. Met (circle one): Yes No Evidence relevant to these criteria Met (circle one): Yes No Metric 3: The progression of content from course to course is coherent. Lessons that only include mathematics from previous grades or courses are clearly identified as such to the teacher. Met (circle one): Yes If any criterion in Table I.1 isn’t met, then the Preliminary Alignment Check fails and the materials are Far From Aligned. Preliminary Alignment Check Result (circle one): 60 Passed Far from Aligned No Program Reviewed: Section II. Alignment: Concepts, Fluency, Applications Table II.1. Criteria for alignment in concepts, fluency, and applications. See IMET pp. 11–16 for guiding questions. Metric AC-1A. The materials support the Metric AC-1B. The materials are designed so that Metric AC-1C. The materials are designed so that development of students’ conceptual students attain the fluency and procedural skills teachers and students spend sufficient time understanding of key mathematical concepts, required by the Standards. working with applications, without losing focus on especially where called for in specific content material widely applicable to, and prerequisite for, standards or cluster headings. a range of college majors, postsecondary programs and careers. Evidence relevant to these criteria Rating for Section II, Alignment in Concepts, Fluency, Applications ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section III. Alignment: Standards for Mathematical Practice Metric AC-2B: Tasks and assessments of student learning are designed to provide evidence of students’ proficiency in the Standards for Mathematical Practice. Metric AC-2C: Materials support the Standards’ emphasis on mathematical reasoning. Materials call for students to produce mathematical arguments. Materials don’t teach multiple methods or strategies just for the sake of variety, but instead support the teacher in using that variety to draw mathematical connections between Metric AC-2D: The richest Modeling experiences found in each course reflect highly developed practices of mathematical Modeling. (Rich Modeling tasks need not be prevalent in the materials to meet this criterion.) 61 Metric AC-2E. Lessons alert the teacher to opportunities for students to discuss important mathematics, and provide sufficient teacher support so that discussions are likely to be successful and build students’ understanding. Program is educative for teachers when combined with professional development in mathematics. Program Reviewed: methods for the benefit of student learning. Guiding questions Considering the variety of tasks and assessments provided (observation checklists, portfolio recommendations, performance tasks, tests and quizzes), do students have opportunities to demonstrate proficiency with each of the Standards for Mathematical Practice over the course of the year? Does emphasizing the Standards for Mathematical Practice tend to open up room for students to work on content not required by the standards? Are students challenged to make sense of word problems, or are word problems always so scaffolded or modeled that students learn recipes for them? Find, or have the publisher provide, the richest Modeling experiences in each course. How prominent are Modeling practices required in these tasks? Do students use graphing calculators, spreadsheets, etc. only on command, or are students asked to be strategic about technology? Are students pushed to improve the precision of their mathematical statements? Do students perform algebraic manipulations only on command, or do students look for structure and rewrite expressions for a purpose? Are students supported to look for and express regularity in repeated reasoning when working with functions? Evidence relevant to these criteria Rating for Section III, Alignment in Standards for Mathematical Practice ___Far from aligned or infeasible to modify to reach alignment (0 pts) ___Nearing alignment and straightforwardly modifiable to alignment (1 pt) ___Aligned and straightforwardly modifiable to better alignment (2 pts) ___Richly aligned, perhaps after minor modification (3 pts) Section IV. Support for All Students Materials include evidence that teachers/ students are reasonably able to complete the core content within a regular school year. Materials include evidence of all students having the opportunity to work with and meet grade-level standards. Support for English Language Learners and other special populations is thoughtful and helps those students meet the same Design of lessons attends to the needs of a variety of learners (for example, using multiple representations, deconstructing/reconstructing the language of problems, providing 62 Materials include regular, balanced assessments that measure progress; valid recommendations are provided for how to address results from assessments for students who show lack of mastery as well as for students who demonstrate Program Reviewed: Standards as all other students. suggestions for addressing common student difficulties). proficiency. Guiding questions: Are problems posed with carefully considered language? How strong, and how up-to-date, is the research that informs the supports provided to teachers who have students who are English Language Learners? Do the teacher materials or other components describe a detailed approach and framework for the way the materials support English Language Learners? Do the materials make systematic use of a number of productive language routines? Are the particular routines chosen for a given lesson well matched to the mathematical task at hand? Evidence relevant to these criteria Rating for Section IV, Supports for All Students ___Far from aligned for all students or infeasible to modify to alignment for all students (0 pts) ___Nearing alignment for all students and straightforwardly modifiable to alignment for all students (1 pt) ___Aligned for all students and straightforwardly modifiable to better alignment for all students (2 pts) ___Richly aligned for all students, perhaps after minor modification (3 pts) Section V. Fit to Your District Program is easy to learn and implement given your resources, personnel, and history to allow all students to meet grade-level standards. Evidence relevant to these criteria Program fits or can fit into your existing school and community culture. 63 Program is more affordable relative to others that are equally effective and appropriate to your circumstance. Program Reviewed: Rating for Section V, Fit to your District ___Not suited ___Could work ___Well suited Total Score (Sections II–IV): ______ 64 i Instructional Materials Evaluation Tool (IMET). https://achievethecore.org/page/1946/instructional-materials-evaluation-tool National Center on Education and the Economy (NCEE) (2013), “What Does It Really Mean to Be College and Work Ready? The Mathematics Required of FirstYear Community College Students,” NCEE: Washington, D.C. www.ncee.org/wp-content/uploads/2013/05/NCEE_MathReport_May20131.pdf iii Quoted in Zimba (2013), “PISA and the U.S. Common Core State Standards for Mathematics,” Chapter 4 in Lessons from PISA 2012 for the United States, OECD Publishing, 2013. iv Finkelstein et al., “College Bound in Middle School & High School? How Math Course Sequences Matter.” https://www.wested.org/wpcontent/files_mf/139931976631921CFTL_MathPatterns_Main_Report.pdf v “Bold Algebra Policy in San Francisco Pays Off.” SERP Institute newsletter, October 2017. http://serpinstitute.org/oct2017_serpress.html vi Instructional Materials Evaluation Tool (IMET). https://achievethecore.org/page/1946/instructional-materials-evaluation-tool vii Instructional Materials Evaluation Tool (IMET). https://achievethecore.org/page/1946/instructional-materials-evaluation-tool ii 65