Implementation of Multiple Measures Assessment

Historical resources for colleges who applied multiple measures assessment, including resources for getting started; decision rules and analysis code; guides for implementing multiple measures, non-cognitive variables; self-reported data; creating a research plan; and data submission resources.

MMAP IMPLEMENTATION

Improving Placement Accuracy in California Community Colleges Using Multiple Measures of High School Achievement, April 2019
Peer reviewed article published in Community College Review discussing the results of the Multiple Measures Assessment project.
MMAP Implementation Survey, Full Report, Spring 2018
Report sharing all responses to all questions in the survey, including open-ended responses that may be helpful to colleges who would like to learn from others.
Phase III Project Summary, 2017
Summary providing accomplishments in the past three phases and steps to take moving forward.
Research Brief - Validating Placement Systems Comprising Test and Multiple Measure Information, 2017
Research brief describing how to validate placement decisions using several objective key metrics including throughput, throughput rate, predictive validity, and disproportionate impact.
Data Upload to Cal-Pass Plus for Cohort Match Guide, October 2015
Guide walking pilot colleges through the process of uploading and downloading a cohort to Cal-PASS Plus; includes the steps needed to complete the Student Cohort File upload including the variables to provide to Cal-PASS to make a match, where to go to upload the file, how to retrieve the file after it has been matched, and the data elements sent back to you.
Data Agreement to Access CalPass Plus Data, 2015
Form required for participation; please complete and return this data-sharing agreement to access CalPass Plus data.
Phase I MMAP White Paper, September 2014
White paper providing a background on multiple measures research and plans for moving forward with the project.
MMAP Prospective File Walkthrough
Guide showing colleges how to upload their Student Cohort File to Cal-PASS Plus in order to receive back the Prospective Data File; data upload includes items required for Cal-PASS to match your students to their high school transcript data, and provide the assessment level based on the decision tree output.
Prospective File Data Element Dictionary
Spreadsheet sharing variables and definitions for the data that will be returned to a college after you upload your Student Cohort File; file includes the recommended assessment levels based on the decision tree output, as well as high school transcript information.

CATESOL CONFERENCE

CATESOL 2018 Annual Conference: Research Update on ESL and AB 705 from the RP Group, December 2018
CATESOL packet sharing Cypress College ESL Courses to Fulfill Requirements for GE C2, CSU GE Breadth C2, and IGETC 3B at 2018 CATESOL Conference.

TOOLS

Maximizing High School Transcript Information for MMAP, October 2017 (PPT)
PowerPoint presenting tools to help encourage local participation in CalPASS Plus, and alternative approaches when student high school information is not available, including local transcript review and self-reported transcript information.
Developing a Multiple Measures Research Plan, April 2015
PowerPoint used in webinar providing examples of research plan templates and timelines for planning implementation of multiple measures strategies. Search for “MMAP Webinar: Developing an MM Research Plan.”

WEBINARSS

Cohort Files Upload Process to Cal-PASS Plus, March, 2017
PowerPoint used in webinar walking pilot colleges through the process of uploading their pilot data to Cal-PASS Plus to be included in the statewide analysis to validate the rule sets. Search for "MMAP - Returning Your Pilot Data to CalPASS Plus."
Multiple Measures Assessment Project: Math & English Models from Phase II Multiple Measures Pilot College Convening - North Cañada College
PowerPoint used in webinar providing examples from three pilot colleges on how they have approached multiple measures assessment. Search for "Multiple Measures Assessment Project."
Putting it All Together: Finalizing Your Local Implementation Plan, May 2015
PowerPoint used in webinar providing information for pilot colleges on finalizing local implementation plans. Search for “MMAP: Completion of Local Implementation Plan.”
How to Use R: Decision Trees and Much More! April 2015
PowerPoint used in webinar demonstrating how to use the open source statistical software R to create decision trees. Watch the archived webinar, and search for “How to Use R: Decision Trees and Much More!”

DECISION RULES AND ANALYSIS CODE

CREATING COLLEGE LEVEL DECISION RULES

Using R for Creating Predictive Models, June 2016
Instructions sharing how R Code can be used to create the decision rules for your college using the Retrospective file from Cal-PASS Plus.
Retrospective File Data Element Dictionary
Spreadsheet sharing variables and definitions of the data used in the Retrospective Analysis to create the decision rules for your college; files will assist you in replicating the statewide models using your local data.

PHASE II RULE SETS

Multiple Measures High School Variables Model Summary – Phase II, February 2017
Summary providing decision rules for placing students in English, Math, Reading, and ESL (Top Level L1) using high school transcript-based placement.
Multiple Measures – ESL Decision Trees with Output, November 2016
Rules instructing how to read decision tree output used for the creation of ESL high school transcript-based placement.
Mathematics Placement Models for the Multiple Measures Assessment Project – Phase II, November 2016
Document sharing the decision tree output used for the creation of math high school transcript-based placement.
English Placement Models for the Multiple Measures Assessment Project - Phase 2, November 2016
Document sharing decision tree output used for the creation of English high school transcript-based placement.
Multiple Measures - Reading Decision Trees Output, May 2016
Rules instructing how to read decision tree output used for the creation of reading high school transcript-based placement.

SELF-REPORTED DATA AND NON-COGNITIVE VARIABLES

The Role of Non-cognitive Variables in Identifying Community College Students in Need of Targeted Supports, February, 2020
A scholarly article highlighting the trivial practical utility of non-cognitive variables as a placement measure.
Self-Reported Data Questions: High School Transcript Information Section, October 2018
Instructions showing how to gather self-reported high school transcript information that is now available as an add-on to the CCCApply application. Colleges who would like to add the questions may contact John Hadad.
Non Cognitive Variables Implementation Guide, December 2016
Guide walking pilot colleges through the steps to implement a NCV scale with the multiple measures models.
Recommended Questionnaires: Non-cognitive Variables Criteria Scales, December 2015
Chart displaying criteria scales and the constructs they measure.
Questions in each questionnaire, April 2015
Menu sharing social-psychology/noncognitive scales for MMAP pilot testing in 2015.

RESEARCH PLAN

Multiple Measures Assessment Project Pilot College - Developing a Research Plan, 2015
Guide providing examples on how to develop a research plan to assess the impact of multiple measures in conjunction with your current placement process.
MMAP: Pilot College - Developing a Research Plan, 2015
Sample research plan directing pilot colleges how to develop a local research plan.
Long Beach City College Promise Pathways Evaluation Matrix, September 2012
LBCC Evaluation matrix evaluating their Promise Pathways Initiative.
MMAP Research Plan Template
Sample template assessing the impacts of MMAP for pilot colleges to use to develop a local research template.

ESL RESOURCES

Noncredit English Language Learners' (ELL) Transitions to Credit Courses: Summary from a Mixed-Methods Analysis, May 2019
Report providing the results of a mixed-methods analysis of transitions from adult education programs or noncredit ESL programs to credit ESL programs. Also includes an analysis of CASAS data.
AB 705 Implementation Survey Results - ESL Focused Responses, January 2019
Report providing the results of the AB 705 Implementation Survey administered in fall 2018 and includes all questions and responses regarding ESL courses and students.
CATESOL 2018 Annual Conference: Research Update on ESL and AB 705 from the RP Group, December 2018
CATESOL packet sharing Cypress College ESL Courses to Fulfill Requirements for GE C2, CSU GE Breadth C2, and IGETC 3B at 2018 CATESOL Conference.
Education Level Workshop: Community College Research Update on ESL and AB 705 from the RP Group, December 2018
PowerPoint presenting research update on ESL and AB 705 from the RP Group. 2018 MMAP presentation delivered at 2018 CATESOL Conference.
Education Level Workshop: Community College Research Update on ESL and AB 705 from the RP Group, December 2018
PowerPoint presenting research update on ESL and AB705 from the RP Group. 2018 MMAP presentation delivered at 2018 CATESOL Conference.
Additional Multiple Measure Question Ideas, July 2018
Tool sharing sample questions created by colleges who add them to their assessment process which serve as an additional multiple measure; these are only examples and any college would need to determine if the questions are correlated with student...
ESL Multiple Measures Questions - Translated into 10 Languages, March 2018
The multiple measures survey used by many colleges for their ESL populations, originally created by Irvine Valley College, was translated into 10 different languages by Mission College, including Arharic, Vietnamese, Japanese, Russian, Turkish,...
Research Brief - English as a Second Language Multiple Measures Development, June 2017
Brief informing the CAI Steering Committee about challenges and strategies with respect to analyzing ESL course sequences in order to create a new multiple measures to be used in course placement.

MMAP POWERPOINT PRESENTATIONS

2017 MMAP POWERPOINT PRESENTATIONS

How Machine Learning and Multiple Measures are Reshaping College Placement, November 2017
PowerPoint sharing how machine learning and multiple measures are reshaping college placement at 2017 CAIR Conference.
Multiple Measures Assessment Project: From Pilot to Paradigm, October 2017
PowerPoint sharing how the Multiple Measures Assessment Project went from pilot program to a paradigm at 2017 Strengthening Student Success Conference.
Multiple Measures Assessment Project: English as a Second Language, October 2017
PowerPoint offering how to apply multiple measures to ESL students at 2017 CATESOL Conference.
Automating the MMAP Process - Examples from Pilot Colleges, September 2017
PowerPoint sharing examples from three pilot colleges who have automated parts or all of their MMAP process to help streamline the student placement process and improvements they have made over time.
Implementing and Improving Your MMAP Process - Examples from Pilot Colleges, September 2017
PowerPoint providing participants with examples and strategies from three different approaches at pilot colleges to help inform local implementation.
Multiple Measures Assessment Project: The Opportunity and Adversity of Increasing Placement Accuracy, April 2017
PowerPoint sharing the opportunity and adversity of increasing placement accuracy at 2017 RP Conference.
Research and Evaluation Considerations, March 2017
PowerPoint offering pilot colleges who would like assistance developing a comprehensive research plan to track the impact of new multiple measures models on student placement, success, and sequence completion.
MMAP Implementation: An overview of the process for new pilot colleges, March 2017
PowerPoint walking new pilot colleges or colleges who have not yet placed a cohort using the models through some of the key steps to consider.
Multiple Measures Assessment Project (MMAP), 2017
PowerPoint explaining the Multiple Measures Assessment Project (MMAP) at 2017 CAP Conference.

2016 MMAP POWERPOINT PRESENTATIONS

Multiple Measures Assessment Project: The Opportunity and Adversity of Increasing Placement Accuracy, October 2016
PowerPoint covering the opportunity and adversity of increasing placement accuracy at 2016 Strengthening Student Success Conference.
Multiple Measures Assessment Project (MMAP) Pilot College Examples Webinar, October 2016
PowerPoint showcasing the experiences of three pilot colleges who implemented MMAP: Norco College, Riverside City College, and MiraCosta College.
From Start to Finish: An overview of the implementation process for new pilot colleges, September 2016
PowerPoint providing an overview of MMAP including: creating a team, data matching, analyzing enrollment impacts, communication and outreach; designed for new pilot colleges or those with new members to their team.
From Start to Finish: An Overview of the Implementation Process for New Pilot Colleges, May 2016
PowerPoint walking pilot colleges through the process of implementing the multiple measures; designed for new pilot colleges.
Multiple Measures Assessment Project - Pilot College Examples, March 2016
PowerPoint highlighting experiences of three pilot colleges that have implemented multiple measures: Irvine Valley College, Canada College and Rio Hondo College.
Reading and ESL Rule Sets for MMAP Pilot Colleges, February 2016
PowerPoint walking pilot colleges through the rule sets for reading and ESL courses. NOTE: the reading rule sets shared in this webinar have been updated. See “Reading Decision Trees Output (May 2016)” in the “Decision Rules and Analysis Code”...
Multiple Measures Assessment Project, 2016
PowerPoint explaining the Multiple Measures Assessment Project at the 2016 RP Conference.
Battle of the Conjunctions: Disjunctive vs Compensatory Course Placement, 2016
PowerPoint addressing disjunctive versus compensatory course placement at 2016 CAIR Conference.

2015 MMAP POWERPOINT PRESENTATIONS

Using Multiple Measures to Enhance Student Placement, November 2015
PowerPoint sharing how using multiple measures can enhance student placement at 2015 CAIR Conference.

The Research and Planning Group for California Community Colleges (The RP Group) strengthens the abilities of California Community Colleges to gather, analyze, and act on information in order to strengthen student success.

Useful Links

For information about the services or resources we provide, or to get more information about projects we have supported, email info@rpgroup.org.
369-B Third Street #397
San Rafael, CA 94901
510.527.8500

NEED MORE HELP?

Can't find what you're looking for? Send an email to communications@rpgroup.org.