I wanted a graphical representation of the standards that should be used in guiding this evaluation so that it could be used as a tool for communicating the standards guiding quality throughout the evaluation process. Using a video resource from Michael Quinn Patton and a standards communicated on the Evaluation Ontario website, I created a mental model of the standards guiding the evaluation. Then, I used the resources listed at the end to guide me through the process of considering all of the necessary dimensions of standards of practice so that I could reflect and consider how my program will meet these standards.
Each of the building blocks of a solid evaluation structure for program evaluation have been considered are outlined in detail below. Adjustments can be made during the evaluation process, based on process use knowledge gained.
Quality
Evaluation Planning
Standards
of Practice and Guiding Principles for the Evaluation
UTILITY
1.
Evaluator
Credibility: This evaluation will be
overseen by those trained and knowledgeable in evaluation practices.
2.
Attention
Stakeholders: This evaluation will
involve stakeholders and primary users from the beginning to follow up stage of
the evaluation by inviting stakeholders to participate in open discussion forum
and formal feedback mechanisms like surveys and brainstorming sessions.
3.
Negotiated
Purposes: This evaluation will
identify and reassess needs of, and with stakeholders by checking in with
stakeholders during interactions and allowing for feedback through our
website.
4.
Explicit
Values: This evaluation will be
transparent in it’s core values., as identified on website, during service and
in meetings.
5.
Relevant
Information: This evaluation will
serve the needs identified by the stakeholders by inquiring to the needs and
checking if the methods used are meeting the needs.
6.
Meaningful
Process and Products: This
evaluation will encourage ongoing understanding of the program and process of
evaluation by engaging with staff and
primary users to uncover meaning.
7.
Timely
and Appropriate Communicating and Reporting: Evaluations ensure timely and appropriate
reporting by using email alerts to drive primary users to report release on the
website. Social media will also point
toward use.
8.
Concern
for Consequences and Influence: Report
findings will be presented as evidence is gathered, without altering or
omitting content for any other reason beyond legal requirements (personal
privacy, for example).
FEASIBILITY STANDARDS
1.
Project
Management: This evaluation will use
Trello project management software to ensure tasks are assigned and completed
as desgined.
2.
Practical
Procedures: This evaluation will
take into consideration what is practical from te primary user’s point of
view. For example, student availability
for meetings would indicate times for meetings, interviews and brainstorming
sessions.
3.
Contextual
Viability: This evaluation will
engage the assistance of the College Diversity Coordinator as an added
safeguard to ensure cultural needs and political interests are in appropriate
balance.
4.
Resource
Use: This evaluation will engage the
College sustainability committee to ensure efficient use of resources and
minimization of energy use.
PROPRIETARY STANDARDS
1.
Responsive
and Inclusive Orientation: This
evaluation will not limit participation with any primary user or stakeholder
who wishes to appropriately participate.
2.
Formal
Agreement: This evaluation will
endeavour to meet the cultural needs and expectations of clients and
stakeholders by putting into place confidential feedback mechanisms on the
program evaluation website.
3.
Human
Rights and Respect: This evaluation
will designed in a respectful manner that protects human rights. Feedback regarding concerns will be
encouraged.
4.
Clarity
and Fairness: This evaluation will
use plain language and will seek advice from Literacy partners to ensure the
wording is inclusive, fair and clear.
5.
Transparency
and Disclosure: This evaluation will
ensure that findings are not withheld from any stakeholder, client or primary
user, unless required by law.
6.
Conflicts
of Interests: This evaluation will
be open to conversations and clarifications of conflicts of interest or
potential for conflicts of interest.
7.
Fiscal
Responsibilities: This evaluation
will minimize costs in compliance with College Finance Policy.
ACCURACY STANDARDS
1.
Justified
Conclusions and Decisions: This evaluation
will adequately justify conclusions and decisions in order to ensure cultural
and contextual sensitivity.
2.
Valid
Information: This evaluation will
ensure validity by serving the purpose for the evaluation as defined in the
consultation phase.
3.
Reliable
Information: This evaluation will
use standard assessment and input practices to ensure that the information is
dependable.
4.
Explicit
Program and Context Descriptions:
This evaluation will use documentation that is clear and concise, but
with enough detail to communicate the full scope.
5.
Information
Management: This evaluation will
adhere to Ministry policy with the physical and virtual safety of data within
the evaluation.
6.
Sound
Design and Analyses: This evaluation
will be designed in such a manner that the questions are asked with
consistency, the consultation remains open and the data is complied
appropriately for analysis
7.
Explicit
Evaluation Reasoning: This
evaluation will compile findings of themes, cross tabulations and numerical
findings in such a way that it is most clearly communicated, including graphic
representation, whenever possible.
8.
Communicating
and Reporting: This evaluation will
endeavour to report finding without bias or misrepresentions.
EVALUATOR ETHICS
2.
Integrity:
The evaluator will have demonstrated
personal integrity in relationships with stakeholders, as evidenced to
commitment to the evaluation and it’s process. The evaluator will have no conflict of
interest and will be sensitive to the cultural and social environment in which
the evaluation takes place. The evaluator will seek truth and enlightenment in
the evaluation process and will not misuse process or findings data in any way. The evaluator will report the facts and the
context in which the facts were gathered.
3.
Accountability: The evaluator will take ownership over
his/her performance and results. The
evaluator will be accountable for being socially and culturally responsible,
drawing on the expertise of college departments when needed for guiding principles. Evaluators will be accountable for spending
and managing budgets appropriately.
Evaluators will be sensitive to the use of time as a resource and
allocate it appropriately.
EVALUATION
QUALITY
1.
Timely: This evaluation will ensure quality standards
by releasing weekly findings to the evaluation website and not withholding
knowledge gathered from process or findings.
Trello software will prompt the action for updating website and for
updating social media updates.
2.
Relevant: This evaluation will be committed to being
relevant in process and findings by investing in early consultation with
stakeholders, then being flexible in the process evaluation to change course,
in order to keep the evaluation relevant to primary users and stakeholders. Feedback mechanisms of open dialogue,
welcomed feedback and technology (Survey Monkey) will support the dialogue
about evaluation relevance.
MEANINGFUL USE
1.
Findings
Use: This evaluation
will make finding
transparent and will explain the context and process used for the findings in a
clear and concise manner. The evaluation
will guard against misuse of findings, mischievous use, misevaluation or non
use by making the expectations known during the initial consultation process,
ensuring that primary users meet commitments of process use findings and by
follow-up up after the evaluation to ensure meaningful use.
2.
Process
Use: This evaluation will fully
endeavour to build relationships with primary users in order offer a reflective
view of the program, the organization, communication styles, structure and
values during the evaluation process.
The evaluation will guide against misuse by ensuring that the process
does not suppress information, allow mischievous use, misevaluate or in any way
condone non-use of process findings.
RESOURCES:
Evaluating Ethics, politics, standards and guiding principles. Better Evaluation [Website]. Retrieved from:
http://www.betterevaluation.org/en/resource/guide/IPDET_ethicsguide_mod14
Fitzpatrick, Worthen & Sanders. (2007) Evaluation ethics, politics and guiding principles. International program for development evaluation and training (703-733). Retrieved from:
http://www.betterevaluation.org/en/resource/guide/IPDET_ethicsguide_mod14
Shulha, L. & Cousins, B.
(1997). Evaluation use: theory, research and practice since
1986. Evaluation Practice. Volume 18 (3). (195-208).
Standards and Guidelines. Evaluation Ontario [Website]. Retrieved from: http://www.evaluationontarioca/membership/standards-guidelines/
No comments:
Post a Comment