Keystone Stars Internal Assessment Process Guide 0 The

1679652563
Keystone stars internal assessment process guide 0 the

File Name: Keystone-STARS-Internal-Assessment-Process-Guide.pdf

File Size: 1.60 MB

File Type: Application/pdf

Last Modified: 2 years

Status: Available

Last checked: 8 days ago!

This Document Has Been Certified by a Professional

100% customizable

Language: English

We recommend downloading this file onto your computer

Summary

KEYSTONE STARS Internal Assessment
Process Guide
0
August 1, 2020
KEYSTONE STARS Internal Assessment Process Guide
Contents
About Keystone STARS..........................................................................................................................................3
Program Quality Assessment in Keystone STARS .................................................................................................3
Preparing for the quality journey .........................................................................................................................5
Develop: Creating the Internal Assessment Process Plan ....................................................................................5
Create the Team and “Plan to Plan” .................................................................................................................5
Selecting an Appropriate POI ............................................................................................................................6
Preparing to Conduct Internal Observations ....................................................................................................7
Documenting IAP Planning ...............................................................................................................................7
Implement: Completing the internal assessment process ...................................................................................7
Evaluate: Measuring progress toward CQI goals ..................................................................................................9
Appendix A: Keystone STARS Internal Assessment Process Initial Cycle Planning Document – Template ........11
Appendix B ..........................................................................................................................................................14
Appendix C ..........................................................................................................................................................15
Program Observation Instruments for Classroom Practice ............................................................................15
Program Observation Instruments for Leadership and Management ...........................................................19
Appendix D: Internal Assessment Process & Program Observation Instrument Resources ..............................20
www.pakeys.org .............................................................................................................................................20
2
KEYSTONE STARS Internal Assessment Process Guide
About Keystone STARS
Keystone STARS is a program of Pennsylvania’s Office of Child Development and Early Learning
(OCDEL) and is the state’s Quality Rating Improvement System (QRIS). Keystone STARS has four
primary goals:
1. To improve the quality of early childhood education (ECE);
2. To support ECE providers in meeting their quality improvement goals;
3. To recognize programs for continuous quality improvement and meeting higher
quality standards; and
4. To provide families a way to choose a quality ECE program

OCDEL’s Keystone STARS Program supports all early childhood education programs to improve
program quality. We encourage programs to think about the Keystone STARS Performance
Standards as a tool to support continuous quality improvement (CQI) that impacts services to the
children and families in your care

Program Quality Assessment in Keystone STARS
Program Quality Assessment (PQA) refers to the process of observing classroom practices and/or
classroom environments, and program leadership and management practices. Objective observation
helps early childhood and school- age professionals review their program’s practices to see what they
are doing well and identify areas for growth. PQA feedback is used to inform CQI goals

The COVID-19 pandemic necessitated changes to the role of PQA within Keystone STARS in the 20/21
fiscal year. The specific changes include the temporary suspension of formal external PQA visits for STAR
3 and 4 programs. Changes to temporarily suspend the requirement for an external Program Quality
Assessment (PQA) using ERS or CLASS in the 2020-21 STARS Performance Standards are related to best
practices in the mitigation of the spread of COVID-19. Future Keystone STARS designations may
require an external PQA to move up or maintain a STAR 3 or 4 designation

Programs in Keystone STARS are required to complete internal assessment at STAR 2. As programs work
toward STAR 3 and 4, they are required to create an Internal Assessment Process (IAP) plan that
outlines the key elements in successfully observing program practices and using feedback to inform
goals for classroom and leadership/management practice growth. Programs are not required to achieve
reliability or certification on the Program Observation Instrument (POI) selected and the information
gathered through internal assessment may differ from the information collected through reliable
assessment

Partnership with the program’s Quality Coach and a Pennsylvania Key Program Quality Assessor is
integrated into the Internal Assessment Process to support meaningful use of internal assessment for
CQI goal setting

Internal assessment and moving to web-based PQA support reflects the high priority of reducing
exposure to COVID-19 until a safe return to onsite work becomes possible. However, internal
assessment is an ongoing process that can be used at any time to enhance your program’s commitment
to growth and improvement. Developing a process for your internal assessments will help you prepare
for a meaningful experience that builds on your program’s philosophy, priorities, and culture

3
KEYSTONE STARS Internal Assessment Process Guide
The Keystone STARS Internal Assessment Process (IAP) uses evidence-based program observation
instruments (POI) to objectively measure the quality of program leadership and management practices,
observed classroom practices, environments, and experiences for children. Internal assessment for
continuous quality improvement (CQI) often happens in the classroom while children are present –
some instruments consider practices and systems outside the classroom - and is most meaningful when
the observer is familiar with the instrument being used. Results from internal assessment do not confer
a definitive or permanent status of high or low program quality; they are a guide to support goal setting
within a program-driven plan for growth

A well-developed internal assessment plan created collaboratively by program leadership and teachers
and with the support of Keystone STARS partners is an essential tool in guiding the development of CQI
across all areas of the program. The data-collection and goal-setting steps of the CQI process guide early
learning and school-age programs toward achieving and maintaining high quality environments and
experiences. CQI is an ongoing process that includes:
• evaluating the program’s current environment and/or practices;
• reviewing objective sources of evidence that indicate program quality;
• planning measurable and specific achievable goals for growth and implementing strategies that
move learning programs toward best practices

The Internal Assessment Process (IAP) Guide has been developed to support programs in selecting a
focus for Continuous Quality Improvement (CQI) and planning how to use the information collected
through internal assessment. The IAP Guide is aligned with the required Keystone STARS Internal
Assessment Process Initial Cycle Planning Document and highlights each of the areas necessary for IAP
planning. The IAP Guide supports the effective use of POI and the ongoing CQI cycle of developing,
implementing, and evaluating

4
KEYSTONE STARS Internal Assessment Process Guide
Preparing for the quality journey
Preparation for internal assessment for CQI begins with building the program’s Internal Assessment (IA)
Team and identifying the focus for internal assessment and CQI. The program’s IA Team will be a
sounding-board for ideas, lend support to implementing the Internal Assessment Process (IAP), and
enrich CQI planning. The team should reflect the program’s culture for shared leadership and build on
the unique skills and interests of staff. The IA Team will often be led by a program administrator
(director, assistant director, education coordinator) and might include:
• An age-group Master Teacher,
• A classroom representative from each classroom that will be assessed,
• A teacher with an interest in the focus of your CQI plan, or
• A teacher with experience using POI feedback to set goals for improvement

The IA Team, the program’s Quality Coach, and an OCDEL-approved PA Key Program Quality Assessor
(Assessor) will review the POI choices, discuss what the program wants to learn about classroom and/or
program practice, and the program’s priorities for growth. The POI choices for Keystone STARS (see
Appendix C) include evidence-based, reliable tools used to measure early and school-age care program
quality. The information collected with a POI can be used to inform decision-making on all or part of a
program’s services and support the ongoing CQI cycle of developing, implementing, and evaluating

 If the program has a STARS goal of meeting Keystone STARS quality indicator 3.4.5, the
program’s IA Team will partner with their Quality Coach and an Assessor to plan and document
the Internal Assessment Process. The Quality Coach-Assessor-Program team can also come
together for next steps on applying the information collected to CQI planning and goal-setting

 If the program has a Keystone STARS goal that includes the optional Keystone STARS quality
indicator 3.4.6, the program’s IA Team will partner with an Assessor to identify an additional
Program Observation Instrument (POI) and plan and implement the Internal Assessment Process
for that additional assessment. After completing the internal assessment(s) planned at EC 3.4.5
and the additional internal assessment cycle for 3.4.6, the program will work closely with an
Assessor reliable on the POIs used to apply the information to CQI planning and goal setting

Develop: Creating the Internal Assessment Process Plan
The program’s IA Team will decide if they will use the Internal Assessment Process (IAP) Planning
Document Template (Appendix A of this guide) or create their own IAP Planning Document. The
provided IAP template outlines the basic elements that programs will use to guide their decision-
making, classroom observations, feedback conversations, and CQI goal setting. If the program’s IA Team
designs their own IAP Planning Document it must address, at a minimum, all the information outlined in
the IAP template. Assistance in creating and implementing the Internal Assessment Process Plan is
available from the Early Learning Resource Center and the PA Key Program Quality Assessment team

The following are elements for the IA Team to discuss during the development of the program’s process:
Create the Team and “Plan to Plan”
• Internal Assessment (IA) Team members are selected and identified on the IAP Planning Document

• IA Team sets realistic timelines for completing IAP planning and for completing internal assessments

5
KEYSTONE STARS Internal Assessment Process Guide
o Don’t underestimate the time needed for the planning process. The IA Team’s timeline
should allow for thorough exploration of the POI options

o Include time, if necessary, to purchase POI materials and complete professional development

o Build in time for those schedule “hiccups” that could cause a delay

• The IA Team discusses the goals the program already has for growth and improvement - the POI
used for internal assessment should support the program’s known goals and priorities

Selecting an Appropriate POI
• Programs may opt to use the POI used at EC 2.1 for the quality improvement work outlined in STARS
quality indicator 3.4.5. If the program feels the instrument used for EC 2.1 no longer supports their
CQI goals, the IA Team’s conversations with their Quality Coach and Assessor will be a starting point
for selecting a POI

• The “Thoughtful Questions for Quality Coaches & Providers” found in the STARS Standards
can help facilitate the IA Team’s discussions about POIs and the focus of the CQI plan. (see
Appendix B of this guide for STARS quality indicators EC 3.4.5 and EC 3.4.6)
• Refer to Appendix C of the Resources Section of this Guide for more information on POI
options for EC 3.4.5 and EC 3.4.6

• Pay attention to the age recommendations for each POI, not all instruments can be used
across all age groups; however, a suite of instruments may include age-specific scales

• A program serving infants, toddlers, preschoolers, and school-age children
identifies teaching interactions as their focus. The program’s IA Team selects
the CLASS which includes instruments for each of those age groups

• A program serving infants, toddlers, preschoolers, and school-age children
would like to learn more about their current practices to support
preschoolers with special needs. The program’s IA Team selects the ICP for
their preschool internal assessment and ITERS-R and SACERS-U for the other
age groups

• When a classroom-based POI is selected for the IAP, the IAP Team will determine which classrooms
will be observed during the IA cycle being planned:
• The IA Team considers each classroom’s strengths and the teacher’s or teaching team’s goals
for growth to determine the POI used to inform the CQI plan and prepare for discussions
with the program’s Quality Coach and an Assessor

• Programs are encouraged to annually assess all classrooms using the internal assessment process;
however, each internal assessment cycle does not need to include all classrooms

• The IA Team creates an annual plan that identifies when each classroom will have the opportunity to
participate in the assessment and CQI planning process. STARS quality indicator 3.4.5 relates to the
program’s initial cycle of the annual Internal Assessment Process (IAP)

• On the IAP Planning Document, the program notes the POI to be used in each classroom and the
overall goal behind selecting that instrument

• REMINDER: When the selected POI is classroom-based and age-specific (i.e. ICP, SPQA), a POI
for the other ages served must also be selected

If the selected POI does not provide information for setting CQI goals related to the program’s health and safety
practices, the IA Team considers and integrates plans for ongoing improvement in Health & Safety practices
based upon their Child Care Facility COVID-19 Health and Safety Plan in the Internal Assessment process

6
6
KEYSTONE STARS Internal Assessment Process Guide
Preparing to Conduct Internal Observations
• In partnership with the program’s Quality Coach and an Assessor, the IA Team discusses how the
staff member(s) completing the internal assessment will learn to effectively use the POI selected

Programs are not expected to achieve reliability in using the selected POI. Internal Assessment is an
opportunity to learn about objective assessment, viewing current program practices, environments,
and policies through the lens of best practice, and using the information gathered for authentic CQI
work

• The assessment materials for some POI are not available for public purchase and can only be
accessed by a trained reliable assessor. If the program’s CQI focus aligns with the focus of a
POI that is not publicly available, alternate approaches include:
 Virtual assessment completed by a reliable Pennsylvania Key Assessor
 Internal assessment supports as defined and approved by the POI author
• Consider the time needed for learning about the POI when setting the projected timelines
for planning and implementation

• The program’s IA Team creates a plan for the teacher/teaching team in the observed
classroom to learn about the POI being used and how the instrument connects to prioritized
CQI goals

• The IA Team determines how and when dedicated time for the internal assessments to be conducted
will be set aside
• Each POI will have its own guidance for time sampling, documentation of practices, and
gathering information. If the instrument’s guidance is not practical or within the program’s
current capacity, the IA Team consults with their Quality Coach and an Assessor about
alternate approaches to objectively and effectively gather valid information

Documenting IAP Planning
• The IA Team documents the process developed on the IAP Planning Document (Appendix A of this
guide) or can elect to create their own written process if it includes at minimum the elements
included in that document

• Prior to implementation of the IAP, the IA Team reviews projected timelines and adjusts as
appropriate

• The assessor will document the program – coach – assessor planning conversation(s) and share the
written IAP Consultation Summary with the program

Implement: Completing the internal assessment process
There are many elements to consider as the IA Team implements the completed plan and moves on to
the task of completing the assessments

• Being prepared, objective, and accurate during the Internal Assessment Process is essential

Disorganized or inaccurate assessment feedback can misdirect goal setting and quality
improvement efforts

• Assures that the POI selected will be used with validity

o The program has all materials necessary to use the POI selected. Utilize Appendix C
and Appendix D of this Guide to find sources for purchasing POI manuals and
supporting materials and accessing Professional Development

7
KEYSTONE STARS Internal Assessment Process Guide
o The Assessor can support the IA Team in verifying the documentation necessary to
accurately assess and guide the organization of internal assessment notes

o The IA Team member(s) implementing the internal assessment will thoroughly
review the POI being used, print any scoresheet(s) needed, and check the publisher
or author’s website and the Program Quality Assessment (PQA) pages of the
Pennsylvania Key website for additional guidance on effective use of the POI
selected (For example, the PA Position Statements for the Environment Rating
Scales), updates or clarifications from the POI authors. (See Appendix D of this guide)
• Whenever possible, all program staff will engage in PQAS approved and/or POI author
approved professional development for the POI used prior to conducting the assessments

o Professional development (PD) includes all education, technical assistance, and
coaching activities that support an individual’s or program’s continuous quality
improvement. When bringing teachers together for traditional training is
challenging, the IA Team talks with the Quality Coach and Assessor about
appropriate alternatives

o Some POI specific PD is available through the PD Registry; however, PD for several
POI options is only available through external sources. Talk with your Quality Coach
and Assessor about the resources available to your IA Team

• Review of the time dedicated for the completion of internal assessments
o Adjustments may need to be made to support accurate gathering of information on
current practices. Each POI will have its own guidance for time sampling,
documentation of practices, and gathering information. IA Team decision-making on
any needed adjustments to the established IA Plan will support the results in
accurately informing goals

o REMINDER: It is critical to base internal assessment findings on what is observed or,
when allowed by the POI, documented. Do not complete the POI using historical
knowledge about practices, systems, or classroom environments

• The IA Team member completing the internal assessment will use their observation notes to
create feedback to be shared with each classroom observed and the IA Team

o POI scoring and observation notes should reflect only what was observed or
documented

o The internal assessment is not a test to be passed, it’s an opportunity to learn more
and plan for growth in ways that align with the program’s culture and priorities

o If there are concerns about a program’s IA Team and/or the observed teachers
being overwhelmed by a POI score, the Quality Coach and Assessor can consult with
the Internal Assessor about options for keeping any numerical score separate from
the feedback shared

o If the program’s STARS goal includes optional STARS quality indicator 3.4.6, the
program’s IA Team will schedule a post-observation meeting with an Assessor to
discuss the assessment outcome

• The IA Team will make note of the internal assessment results that inform goals and action
steps for ongoing improvement in health & safety practices as required in STARS quality
indicator 3.4.5

8
KEYSTONE STARS Internal Assessment Process Guide
• Once the data-collection and/or scoring is complete and the information from the internal
assessment has been shared with the teaching staff that were observed, the IA Team will
begin planning for next steps by answering the Next Steps questions on the IAP Planning
Document

• The Next Steps section of the IAP Plan act as a starting point for building the program’s CQI
plan and establishing measurable achievable goals that connect to priorities for quality
growth

o CQI Planning templates can be found in the STARS Program Manual and on the PA
Keys website (see the Resources section of this guide) to help the program organize
improvement goals and plan for measuring progress toward those goals

Evaluate: Measuring progress toward CQI goals
Internal assessment provides insight into program practices, celebrates a program’s strengths, ensures
that providers are well prepared to meet the needs of children and families, demonstrates a
commitment to best practice and CQI, and increases positive outcomes for all children. The information
gathered through internal assessment guides CQI goal setting so programs can allocate resources
according to their priorities for improvement. Objective feedback from thoughtful internal assessments
set the foundation for the on-going cycle of CQI

The program’s IA Team and partners can build on the work done during the IAP planning and
implementation through follow-up observations to measure progress toward data-informed goal(s) for
improvement. The program’s Quality Coach and Assessor are available to support refinement of CQI
goals based on measured progress and facilitate conversations about additional internal assessment
planning

9
KEYSTONE STARS Internal Assessment Process Guide
Appendix: IAP Planning Template and
Resources
Appendix A: Keystone STARS Internal Assessment Process Initial Cycle
Planning Document – Template
Facility Name
MPI
Team for Internal Assessment Process Creation
Team Member Role: Name:
Program’s Process Lead
(internal observer)
ELRC Quality Coach
Pennsylvania Key
Program Quality
Assessor
Other Colleagues
Projected completion date for Internal Assessment Process development:
Projected date for completion of initial round of internal assessments (classroom observations) using
the established process:
11
Complete the chart for each classroom being assessed during this internal assessment cycle (at least one classroom in
each age group served should be observed)

Classroom Teaching team Age group POI Selected How does this POI support our CQI goals?
(i.e. classroom environment, intentional
teaching, inclusive practices)
Note: Programs are encouraged to assess all classrooms annually using their Internal Assessment
process. Internal assessment should be completed on a typical day for the classroom being
observed. Avoid days when special events are planned and try to avoid days when the regular
teaching team is not present. Observe during the most active part of the day when most children
are present

Develop: Creating the Internal Assessment Process Plan The program’s IA Team will decide if they will use the Internal Assessment Process (IAP) Planning Document Template (Appendix …

Download Now

Documemt Updated

Popular Download

Frequently Asked Questions

What is the star 34 internal assessment process iap?

In place of an external PQA, during FY 2020, STAR 3 and 4 programs will work with their Quality Coach and a member of the Pennsylvania Key PQA Team to develop an Internal Assessment Process (IAP). This IAP will help to guide the program’s Continuous Quality Improvement (CQI) plan and activities.

What are the keystone stars performance standards?

Since the inception of the Keystone STARS program in 2002, the Performance Standards have been the bedrock of the program. The standards are guided by the principle that quality early learning programs are the foundation for children's success, and investment in the early care and education of Pennsylvania's children is an investment in the future.

What is a keystone stars designation visit?

The Keystone STARS designation process requires programs to have a full, onsite designation visit every three years. During this onsite designation visit, ELRC Quality Coaches or Designators meet with program leadership, tour the program, and speak with members of the teaching staff.

What programs does keystone stars support?

A vital part of Pennsylvania's early learning system, Keystone STARS works with programs, such as Head Start, school-aged programs, and programs certified by the Department of Human Services.