Discussion: Exploring Program Evaluation Settings

Dr. Jones wants you and your program evaluation committee to look at other examples of program evaluation efforts to learn as a Professional Learning Community. Case studies are intended to foster rich discussions about evaluation practices, touching on the real issues that arise when conducting an evaluation project (Spaulding, 2014, p. xix).To prepareReview this week’s Learning Resources, focusing specifically on the following resources:The “Expansion of a High School Science Program” and “Evaluation of a Proven Practice for Reading Achievement”” chapters in the Spaulding textProgram Evaluation Standards StatementsProgram Evaluation Model 9-Step ProcessProgram Evaluation Matrix TemplatePost the following to report to Dr. Jones about examples of program evaluation efforts you reviewed and learned from:My topic is:Providing teachers that teach English Language Learners more professional development courses. Compare how data was collected in the “Expansion of High School Science Program” and “Evaluation Proven Practice for Reading Achievement” chapters to your own data collection plans.Identify some of the challenges evaluators face when collecting data and information from stakeholders and relate these challenges to your own educational setting.Support your conclusions with two or more references to this week’s Learning Resources and/or scholarly sources within the last 5 years beyond the course material.References:Spaulding, D. T. (2014). Program evaluation in practice: Core concepts and examples for discussion and analysis (2nd ed.). San Francisco, CA: Jossey-Bass.Chapter 8, “Expansion of a High School Science Program” (pp. 121–128)Chapter 9, “Evaluation of a Proven Practice for Reading Achievement” (pp. 129–135http://cloud-collaboration.kahootz.com/how-to-crea…http://www.jcsee.org/program-evaluation-standards-…


Don't use plagiarized sources. Get Your Custom Essay on
Discussion: Exploring Program Evaluation Settings
Just from $13/Page
Order Essay




Unformatted Attachment Preview

SLDS Best Practices Brief
Stakeholder Communication: Tips from the States
States agree: Effective communication with stakeholders (districts, program offices,
postsecondary education leaders, other state agencies, legislators, etc.) throughout system
design, development, and deployment is vital to the ultimate success of a statewide
longitudinal data system (SLDS). A group of state staff experienced with stakeholder
communications efforts and the related challenges convened to share strategies, best
practices, and lessons learned.
The SLDS Communications Mantra: Connect. Listen. Respond. Sustain.
Identify and reach out to a range of key stakeholders early on.
Be inclusive, but in a focused way and at appropriate times.
Create realistic expectations in all communications.
Form or leverage groups to lead the outreach effort.
Identify key individuals as “ambassadors” and “point people.”
Create and follow a clearly-defined, carefully-conceived outreach plan.
Acknowledge differences among stakeholder groups and tailor your style.
View communications, in part, as a change management activity.
Structure meetings around very specific questions or products.
Listen and be responsive to stakeholder input.
Implement some early wins to gain and sustain stakeholder support.
Give stakeholders credit for help with system design and enhancement.
Do Not:
Do not engage in large-scale, open forums.
Do not ignore the political environment.
Brief 1
May 2011
This product of the Institute
of Education Sciences (IES)
SLDS Grant Program was
developed with the help of
knowledgeable staff from state
education agencies and partner
organizations. The information
presented does not necessarily
represent the opinions of the IES
SLDS Grant Program. We thank
the following people for their
valuable contributions:
Nancy Copa
Naru Nayak
Florida Department of Education
Deb Holdren
Georgia Department of Education
Warren Danforth
Troy Wheeler
Idaho Department of Education
Charles McGrew
Kentucky P-20 Data Collaborative
Mickey Garrison
Josh Klein
Doug Kosty
Mike Rebar
Oregon Department of Education
Robin Taylor
SLDS Program, State Support Team
Do not get “techy” with most stakeholder audiences.
Do not assume that everyone is in sync on system goals.
Do not fail to follow through on communications promises.
Do not NOT communicate.
SLDS Best Practices Brief: Stakeholder Communication
For more information on the IES
SLDS Grant Program, additional
Best Practices Briefs, or for support
with system development, please visit
According to states, the following practices will support
effective communication with stakeholders. (Note: These
suggestions do not necessarily represent the views of the
IES SLDS Grant Program).
Identify and reach out to a range of key
stakeholders early on.
Ideally, SLDS components should be designed from the
start with input from all of the intended user groups at
both the state and local levels. Identify these key stakeholder
groups early on to inform them about the project and begin
to establish interest in and support for the system. Maintain
close communication with these stakeholders throughout
the design, development and deployment phases through
direct engagements to collect input. Hold meetings,
convene focus groups1, and conduct surveys to gain input
from the field on initial design, ongoing development,
and deliverables such as reports and tools. Hold frequent
meetings both within the state education agency (SEA),
with legislative staff, and with districts. Convene meetings
with local education agencies (LEA) at least quarterly to
discuss any changes to the data system that will affect local
staff directly, as well as developments occurring at the state
level that may not have a direct impact. These methods can
be very effective ways of engaging stakeholders, building
a sense of collaboration between state and local staff, and
collecting information about user needs. Use other forms
of communication such as emails, webinars, and a website
to share ongoing status updates (announcements, events,
grant information, contacts, etc.).
Be inclusive, but in a focused way and at
appropriate times.
Input from a full range of stakeholder groups will promote
the ultimate utility and success of the SLDS, but too many
voices can stifle progress and steer the project off track.
Strike a balance by engaging with a manageable number of
representatives from key groups and convening meetings
on a schedule that matches project needs.
Create realistic expectations in all
Be very clear and realistic when defining the purpose of
the project and who it can affect. Be careful not to talk
about the project in terms of benefits to teachers and
students, and positive effects in the classroom unless it is
known for certain that the project can achieve these goals.
Too many SLDS projects have promised great things for
teachers when, in fact, the state ultimately fails to provide
the timely data that teachers need to affect the classroom.
Communicate in such a way as to “under promise and over
Form or leverage groups to lead the outreach
Task a key group (or groups) with coordination of the
communications effort, utilizing existing groups whenever
possible. Some successful models used by the states include:
• Executive group or steering committee including
leaders from across the state to develop a
communications plan, and coordinate outreach and
information gathering efforts.
• Stakeholder advisory groups comprised of members
from a range of stakeholder groups to gather
feedback from and speak for their respective groups.
Which particular groups are represented should be
determined by the type of initiative being addressed.
For example, do you need an IT or end-user perspective
(or both)? A P-20W (preK through workforce) group
of this sort may include school board representatives,
representatives from other state agencies (e.g.,
labor, corrections, public assistance), policymakers,
researchers, business leaders, teachers, parents, etc
• Association groups to engage stakeholders in-person
and remotely to teach stakeholders about the system,
promote the available reports and data portals, and
gather user feedback.
Identify key individuals as “ambassadors” and
“point people.”
Identify individuals to serve as champions and point people
on the SLDS or particular system features. For instance,
representatives from regions around the state (one state
calls them “ambassadors”) communicate with the SEA
on a regular basis and convey messages to and from local
stakeholders. It is also beneficial to identify ambassadors
from various peer groups. For instance, a highly regarded
superintendent is more likely to have influence on other
superintendents than would a school principal (and vice
versa). This approach can help to 1) collect feedback from
SLDS Best Practices Brief
the “ground” to shape state plans, and 2) keep the field
informed about SLDS developments.
Create and follow a clearly-defined,
carefully-conceived outreach plan.
Early on, develop a communications plan2 that clearly
defines key information about the planned outreach
(e.g., audiences that should be kept informed of
SLDS development news, authors or leaders of each
communication activity, methods and formats of
communication to be used, key messages of each
communication, desired outcome of the communication,
and the dates and frequency of communications. Also,
determine when you will simply communicate about the
project (one directional) and when you will actively solicit
feedback (two directional). Be sure to identify an owner
for the plan.
Acknowledge differences among stakeholder
groups and tailor your style.
Acknowledge that while some stakeholders will be receptive
to SLDS efforts, others may not. This latter group may
feel threatened by the impending changes. The potential
need to learn new skills, alter the way they do business,
or the perception of a loss of control of the data are all
common sources of anxiety and resistance from staff. Try
to anticipate which stakeholder groups are more likely to
fit into each of these two groups (receptive or resistant)
and tailor the messaging accordingly to avoid alienating
any of your stakeholders. It is also helpful to identify and
address the distinct perspectives and interests of each
group. For instance, a teacher will be interested in an
SLDS for different reasons than a district superintendent,
so communications should be tailored to appeal to those
distinct interests. Also, be sure to avoid technical or
education jargon when communicating with particular
groups who may not be fluent in those languages.
for the SLDS, alert users to changes in skills that may be
required for use of the data system, and suggest resources
that may be available. The range of skills that individuals
may need to effectively use an SLDS will vary from few to
many. The less intimidating the system is to stakeholders,
the more likely they will be to accept and utilize the
system. Therefore the more effectively the information is
communicated to stakeholders, the less opportunity there
will be for unrealistic expectations, misunderstandings, and
Structure meetings around very specific
questions or products.
When meeting with stakeholders to assess needs, structure
discussions around specific questions the agency wants to
answer. These questions should be those identified by the
field as areas of need (as determined by surveys or focus
groups). Give the group specific products to respond to and
keep discussions focused, rather than starting with a blank
slate. Strike a balance between presenting requirements
that are very detailed and prescriptive and those that are
too loosely defined. When engaging stakeholder groups,
recognize that neither extreme will be ideal, so set the stage
by acknowledging that reality. Even if the group decides to
depart from your starting point, this approach will help to
focus the effort and generate better quality feedback.
Listen and be responsive to stakeholder input.
As user satisfaction will ultimately determine SLDS success,
be sure to listen to stakeholder input. But don’t stop
there. Make sure stakeholders know you have heard them
by responding to and acting on what they have told you
(within reason). Also be sure to communicate the revisions
or changes that have been made based on stakeholders’
feedback. Not only will this improve the system, it will also
strengthen engagement and buy-in, and will help to foster a
collaborative environment.3
View communications, in part, as a change
management activity.
Implement some early wins to gain and sustain
stakeholder support.
Creating and implementing an SLDS involves change
for many stakeholders. Good leaders use stakeholder
communications as an important way to manage the
change process. Beyond the function of information
gathering, your communications serve to set expectations
Identify the deliverables that can be most easily completed
and tackle them for some quick wins. Provide legislators,
educators and other stakeholders with not only data, but
answers to their questions. Show return on investment (e.g.,
short- and long-term student earnings after graduation, total
SLDS Best Practices Brief
number of students leaving the state after graduation, time
it takes students to find jobs after graduation, high school
feedback reports on college success, etc.). Delivering this
valuable information will show stakeholders the true value
of the SLDS.
Give stakeholders credit for help with system
design and enhancement.
Be sure to give credit where it is due. Acknowledge
stakeholders for their input into system design and
development. System success should not be attributed
solely to state education agency leadership or IT staff.
Do not get “techy” with most stakeholder
When communicating or meeting with a non-technical
group, focus on the end user perspective of the system—
the questions they want the system to answer—rather than
the infrastructure that will make it possible to get those
answers. Save the technical discussions for staff who will
actually need to understand that side of the system. For
instance, legislators will not likely be the people logging
into and using the system, so save the discussion of system
access and use for their staff.
Do Not:
Do not assume that everyone is in sync on
system goals.
Based on their experiences, states warn against the
following common communications mistakes. (Note:
These suggestions do not necessarily represent the views
of the IES SLDS Grant Program).
Regularly check in with stakeholders to make sure everyone
is on the same page when it comes to system goals and
desired outcomes. Recognize when those aspirations
evolve along the way and make sure stakeholders are aware
of those changes.
Do not engage in large-scale, open forums.
Do not open the floor to a large group of stakeholders
with vague guidelines for discussion. The more productive
approach may be to engage with leaders of key stakeholder
groups (superintendents associations, technical associations,
school boards, principals, etc.) for focused meetings. Too
many voices can slow progress, so if you do engage a big
group, structure the meeting around a specific set of guiding
questions or products and keep the discussion focused.
Do not ignore the political environment.
Political connections and support can be very beneficial
when it comes to getting funding for system start-up and
sustainability. Engage legislative and executive staff early
on to discuss system planning; ensure that it is designed to
meet their needs and that they feel ownership of the system.
Make sure legislators understand how the system will meet
their information needs and that they are convinced that it
actually will. Another very successful strategy for winning
political support has been to encourage district staff to
serve as advocates and reach out to local legislators (through
communications, presentations, committee hearings, etc.)
to demonstrate the utility of the state system at the local
Do not fail to follow through on
communications promises.
If you do establish an expectation among stakeholders for
continued communication, make sure you follow through
on your communications promises. Not following through
is as bad as not communicating and may be even worse.
This will foster a lack of respect for the work and create an
atmosphere of distrust. Even if the communicator doesn’t
have anything new to report, it is still beneficial to stay in
touch with stakeholders.
Do not NOT communicate.
Lack of communication will imperil an SLDS effort.
1A detailed description of an approach used by Florida to gather and
prioritize user needs is available at
2A sample Communications Plan template is available at
3For a report on how the Oregon DATA Project implemented a
comprehensive strategy for stakeholder input, which ultimately decided
the direction of the project, see the report at

Findings from the field

SLDS Best Practices Brief
Janet E. Wall, EdD, CDFI
President, Sage Solutions
What Is Evaluation?
An evaluation is a purposeful, systematic, and careful collection and analysis of information used
for the purpose of documenting the effectiveness and impact of programs, establishing
accountability and identifying areas needing change and improvement.
Summative Evaluation
There are many types of evaluation, depending on the purpose, timing, and procedures used. A
summative evaluation, sometimes called outcome evaluation, is conducted for the purpose of
documenting the results of a program. Specific goals of a program are identified and the degree
of accomplishment of those goals is documented. The results of a summative evaluation might
point to changes that should be made in a program in order to improve it in subsequent
implementations. The results of summative evaluations can specify program status and
conditions for accountability purposes. The results can also be used as a needs assessment for the
subsequent planning of changes in a program or of the introduction of new programs and
interventions. The following table presents some questions that might be addressed by a
summative evaluation.
When Conducted
After a program has
been implemented
and completed
Examples of Summative Evaluation Questions
What did the program accomplish?
Did the program reach its goals and objectives?
What impact did the program have on its recipients?
What were the outcomes?
Who benefited from the program?
How much was the benefit?
Was the benefit greater with this program as compared with another
Did all types of students or clients benefit from the program?
What were the positive outcomes?
What were the negative outcomes?
What should be improved/changed in the program?
Does the benefit of the program warrant the cost?
Formative Evaluation
A formative evaluation, also known as process or implementation evaluation, is performed to
examine various aspects of an ongoing program in order to make changes/improvements as the
program is being implemented. This type of evaluation attempts to document exactly what is
transpiring in a program. Data are collected and analyzed at a time when program changes can be
made to ensure that the quality of the program implementation is maintained throughout. For
example, if a career development program has been introduced in a school district, it is important
to know to what extent the program has actually been implemented as designed. The following
table suggests some questions that might be addressed in a formative evaluation.
When Conducted
While the program is
ongoing – perhaps
several times
Examples of Formative Evaluation Questions
Is the program being implemented as it was designed?
Do the students or clients understand the program’s concepts?
What are the misconceptions about the program?
Are all program implementers implementing the program in the same
Is the program being implemented on schedule?
Is there sufficient time to implement all aspects of the program?
What aspects of the program do not seem to be working as well as you
Do program implementers need additional training on the program?
Are there any negative outcomes surfacing?
Who Should Conduct or Lead the Evaluation Process?
In some instances an evaluation is designed to answer multiple questions at multiple locations
with multiple types of individuals using sophisticated statistical techniques. In such cases it is
best to bring in a professionally trained individual or company to assist with the evaluation.
However, evaluation does not need to be complex, highly statistical and in need of specialized
experts to be useful and worthwhile. The best evaluations are conducted by those who know and
care about the program and its effects on students and clients — that means you. In an age of
increasing accountability, you need to show the worth of what you are doing and the impact that
programs are having on your students and clients.
Evaluation Process—Overview
Effective program evaluation is a carefully planned and systematic approach to documenting the
nature and results of program implementation. The evaluation process described below is
designed to give you good information on your program and what it is doing for students, clients,
the community and society. The process will help you think through the evaluation in a thorough
manner before you begin to implement a pro …
Purchase answer to see full

Calculate your paper price
Pages (550 words)
Approximate price: -

Why Work with Us

Top Quality and Well-Researched Papers

We always make sure that writers follow all your instructions precisely. You can choose your academic level: high school, college/university or professional, and we will assign a writer who has a respective degree.

Professional and Experienced Academic Writers

We have a team of professional writers with experience in academic and business writing. Many are native speakers and able to perform any task for which you need help.

Free Unlimited Revisions

If you think we missed something, send your order for a free revision. You have 10 days to submit the order for review after you have received the final document. You can do this yourself after logging into your personal account or by contacting our support.

Prompt Delivery and 100% Money-Back-Guarantee

All papers are always delivered on time. In case we need more time to master your paper, we may contact you regarding the deadline extension. In case you cannot provide us with more time, a 100% refund is guaranteed.

Original & Confidential

We use several writing tools checks to ensure that all documents you receive are free from plagiarism. Our editors carefully review all quotations in the text. We also promise maximum confidentiality in all of our services.

24/7 Customer Support

Our support agents are available 24 hours a day 7 days a week and committed to providing you with the best customer experience. Get in touch whenever you need any assistance.

Try it now!

Calculate the price of your order

Total price:

How it works?

Follow these simple steps to get your paper done

Place your order

Fill in the order form and provide all details of your assignment.

Proceed with the payment

Choose the payment system that suits you most.

Receive the final file

Once your paper is ready, we will email it to you.

Our Services

No need to work on your paper at night. Sleep tight, we will cover your back. We offer all kinds of writing services.


Essay Writing Service

No matter what kind of academic paper you need and how urgent you need it, you are welcome to choose your academic level and the type of your paper at an affordable price. We take care of all your paper needs and give a 24/7 customer care support system.


Admission Essays & Business Writing Help

An admission essay is an essay or other written statement by a candidate, often a potential student enrolling in a college, university, or graduate school. You can be rest assurred that through our service we will write the best admission essay for you.


Editing Support

Our academic writers and editors make the necessary changes to your paper so that it is polished. We also format your document by correctly quoting the sources and creating reference lists in the formats APA, Harvard, MLA, Chicago / Turabian.


Revision Support

If you think your paper could be improved, you can request a review. In this case, your paper will be checked by the writer or assigned to an editor. You can use this option as many times as you see fit. This is free because we want you to be completely satisfied with the service offered.

Order your essay today and save 15% with the discount code DISCOUNT15