5 Evaluating Open Educational Resources
“How to Find and Evaluate OER”
By Abbey Elder. Licensed under CC BY 4.0. Transcripts.
Learning Objectives
By the end of this chapter, one will be able to:
- Understand Perspectives on Evaluating and Defining “Quality” as it Relates to Course Materials
- Utilize Relevant Checklists/Rubrics for Evaluating Open Educational Resources
Introduction
The previous section focused on organizing a search and finding relevant Open Educational Resources (OER). This section will focus on evaluating OER and aligning these resources with course objectives. These skills help to narrow resources listed down to content that works best for instructors and their course. When evaluating OER, we encourage the reader to consider how they define and measure quality.
Learning Objective Check-In
Evaluation
Suggestions for widely-adopted rubrics and checklists for evaluation include:
- OER Review Standards and Approval Rubric: United States Edition
(Developed with OPEN FL) - Achieve OER Rubrics
- Peralta Online Equity Rubric
- Affordable Learning Georgia (PDF)
Please take a minute to review each one. Choose one that works best (see Appendix B in OER Review Standards and Approval Rubric: United States Edition for more options). If a department already uses a customized rubric for evaluating course material or textbooks for adoption, use that tool. Evaluating OER shouldn’t be much different from evaluating other course material under consideration for adoption.
Rights and Permissions
Using content without copyrights (public domain works), obtaining copyright permission (through written requests, CC or open licenses, or terms and agreements), or using works under a copyright exemption (Teach Act or Fair Use) are key components of the OER adoption process. Reviewing rights and permissions is a good first step in evaluating selected content. Instructors are advised to document the rights information for each content source and ensure that the planned use is permitted.
Content Validity
It’s important to review the who, what, when, where, and why of OER content just as one would with any other source of information. This analysis should also include reading available reviews or ratings. Many OER have these publicly available (for example, the Open Textbook Library). Consider how effective the OER will be in meeting teaching and learning goals. Colleagues may already be using this resource and be willing to give honest feedback on the OER. Ensure that the content is accurate and appropriate for higher education. Select content that has no obvious errors, conflicts of interest, or ethical concerns.
Accessibility (ADA and Beyond!)
The first step with accessibility is to confirm that students will have no barriers or restrictions to the content (apart from a student login to the learning management system provided by your institution). Next we check that educational content is available to all students at the same time. This means that any accommodations (navigable interface, structured layout, alt-text, closed captions, etc.) are already in place or can be made available before using the content in the course. Another key component to accessibility is language. Content should be available in the language needed for your learning environment. It should be written in a manner that is consistent, clear, and comprehensive to students. Instructors are encouraged to include opportunities to assess learning without a graded penalty before having students complete graded tasks. Because students have a variety of learning and assessment styles, allow options for completing activities in multiple formats where appropriate. When possible, provide print and multiple file format options to students, allowing them to learn in the medium of preference.
Supplementary Resources
OER that come with supplemental resources are instructor and student favorites and are more widely available each day. This content should also be reviewed for permissions, content validity, and accessibility. Please note that the OER and supplementary resources may be on different platforms. Other instructors who are using OER in the associated course may be an excellent source for finding and evaluating this type of content.
Cost and Impact
While OER is available as “free + permissions” there is a cost to implement OER in a course, which can vary depending on the content chosen and how it can be implemented into the environment. First determine the time it would take to adjust course content, resources, or structure to align with the OER. Then think about the number of resources that may be used from the department or institution. This calculation may include support time, tools or product purchases, and cost of hosting new services. Consider how long the OER will be sustainable for a course. These factors help determine the cost of implementation. Another important consideration is cost avoidance. Calculate this impact for a course by using this formula:
[Existing or Comparable Commercial Textbook Cost:____
X
Average Student Enrollment Each Term/Semester:____]
–
Cost for Students, if any
(for example, required support/platforms/content):_____
=
Total Cost Avoidance:___________
Instructor Voice
“Adopting Open Educational Resources promotes a favorable environment of accessibility, equity, and innovation in education, enriching learning experiences while breaking down barriers to knowledge for all.”
Douglas Kines, Professor of Biological Sciences
Florida State College at Jacksonville
Conclusion
Thoughtful evaluation of OER is the capstone of the search and retrieval process. Luckily, there are tools like checklists and rubrics to help appraise the OER found in order to make an informed choice for adoption. The next chapter will discuss the mechanics of incorporating selected OER into a course.
