Program Evaluation

Program Evaluation
Program Evaluation can take components of any or multiple of the designs discussed earlier this semester into one giant research project.

ACCOUNTABILITY
Assessment
Program Evaluation
Outcomes Assessment
Evidence
Formative
Summative

Both Program and Outcome Accountability are important
• You could have a program that does everything it says it’s going to do, but it doesn’t produce any outcomes
– D.A.R.E. is an example of this problem
• Or you could have a program that doesn’t do what it says it’s going to do, but there are still changes in outcomes
– So then how do you know what the changes in outcomes are due to –the program, or some extraneous variable?
Structures of the Program –how is it organized, what infrastructures are in place to support it?
Domains, Standards/Goals and Competencies of the Program
Process of the Program

PROGRAM ELEMENTS
• STRUCTURE
– Definition of the program
• "the program is a developmental educational program...”
• “This program is an integral part...”
• The program includes...”– Rationale for the program
• Importance of the program
• Reasons why students need what the program offers
– Assumptions of the program
• Identify and briefly describe the premise upon which a program rests
– Ethical and legal practice– Roles

PROGRAM STRUCTURAL ELEMENTS
Resources
–Human
–Financial
–Political
• Program Components
–Training
–Student Interventions
–Support for System

DOMAINS
Domains identify conceptual aspects of focus or priorities for the program.
The identification of domains:
1. Provides a description of areas of importance
2. Provides a structure to organize goals and competencies
3. All assessments should be related to the identified domains of the program

STANDARDS/GOALS
Standards define specific desired student results to be achieved by a certain time,
i.e. by the time the student leaves school or by the time that the student finishes the grade level.
They are stated in terms of knowledge, attitudes, skills that have been identified as necessary to the education of the student.
They are written in global terms for each domain.

COMPETENCIES and INDICATORS
Competencies are descriptions of student behaviors (knowledge, skills, attitudes) that are linked to stated goals for their education. They are based on the assumption that the behaviors can be transferred from a learning situation to a real-life situation.

Competencies are:
1. organized into goal content areas
2. arranged developmentally
3. measurable

Components of Program Evaluation
• Process vs. Outcome Evaluation
– Formative = evaluation of process
– Summative = evaluation of final product

• Stakeholders --anyone with a vested interest in the program
– Designers, evaluators, teachers, students, etc.

Program Evaluation: The Assessment Loop
• Components of Assessment
– Measurement
– Evaluation

• What kinds of evidence can be used in Program Evaluation assessments?
– Portfolios, performances, outside judges, observations, local tests, standardized tests, self- assessments, interviews, focus groups, student work (In qualitative research you want to have at least 3 different kinds of evidence to triangulate your results.)
– Nature of evidence depends on nature of question

SCHOOL PROGRAM EVALUATION
Program Evaluation is a cyclical process, in which the outcome of each cycle informs the content and objectives of the following cycle.
Note that both Process (e.g., Action Plan) and Product (e.g. Assessments) are important components of every cycle

Steps in Each Cycle of Program Evaluation:
•Identify Program Goals
•Conduct Needs Assessment
•Set Program Priorities
•Identify Target Groups
•Write Objectives for Activities
•Evaluate the Degree to Which Objectives Were Met
•Re-Set Program Priorities with the Established Program Goals