Evaluation of E-Learning Courses
The following matrix was designed by Badrul Khan, an expert on e-learning and educational technology, to evaluate online courses.
If the question does not apply to the course being evaluated, print Not Applicable (N/A).
Pedagogical Dimension
· Q1. Does the course provide clear instructional goals (or are clear learning outcomes specified?)
· Q2. If appropriate, are objectives for each section of the course clearly specified?
· Q3. What is the instructor’s role?
· More facilitator than didactic
· More didactic than facilitative
· A combination of both
· Q4. If the course design allows the instructor to serve as facilitator, how/ where does facilitation occur?
· mailing list
· online discussion forum
· audio conference
· video conference
· Q5. Does the course provide clear directions of what learners should do at every stage of the course?
· Q6. Does the course provide a sense of continuity for the learners (i.e., each unit of the lesson builds on the previous unit where appropriate)?
· Q7. Does this course utilize a matrix of media to create a rich environment for active learning?
· Q8. Does the course exploit the flexibility of the hypertext/hypermedia environment of the Web?
Technological Dimension
· Q9. Does the course have personnel who can assist learners to set up for starting the course? (Or does the course have orientation programs that provide technical training prior to a course?)
· Q10. Is the cost of required hardware, software, and Internet service a deterrent to taking this course?
· Q11. Do students receive any guidance on how to set up hardware equipment for desktop video conferencing (if needed for the course)?
· Q12. Are the hardware requirements for the course clearly stated?
· Q13. Are the software requirements for the course clearly stated?
· Q14. Do students receive any guidance on how to do the following:
· send e-mail attachments
· install required software,
· scan a picture
· print within Web page frames
· create online presentation using presentation software
· other
Interface Design Dimension
· Q15. Do Web pages look good (i.e., effectively designed) in a variety of Web browsers and devices-in text-based browsers, all recent versions of Internet Explorer and Netscape, and so on?
· Q16. Does the course use a standard font type so that text appears the same in different computer platforms and browsers?
· Q17. Does the course follow “one idea per paragraph” rule?
· Q18. Is the text chunked and presented in a way that enables scanning and comprehension?
· Q19. Does the course provide structural aids or a site map to guide the learner’s navigation?
· Q20. To avoid bandwidth bottlenecks, does the course ask students to download large audio, video, and graphic files to their hard drives before the instructional events?
· Q21.
How quickly can users find answers to the most frequently asked questions on the course site?
· Q22. Does the course use easy to understand terminology?
Evaluation Dimension
· Q23. How is the learner’s assessment of various parts of the course administered? Check all that apply:
· multiple choice
· true/false
· fill-in-the blanks
· essay questions
· papers
· projects
· assignments
· proctored tests
· portfolio development
· case studies
· lab report
· journal
· other
· Q24. Does the assessment provide students with the opportunity to demonstrate what they have learned in the course?
· Q25. Does the course have a system to accept students’ online evaluations of the following?
· Content
· Instructor
· Learning environment
· Learning resources
· Course design
· Technical support
· Other
· Q26. Does the course provide an instant feedback button on most screens/pages to receive learners’ feedback for improvement of the course?
Management Dimension
· Q27. What content exists and what content must be created? Of existing content, what content requires “reprint” permission?
· Q28. Does the course acquire permission to use copyrighted information and materials?
· Q29. Does the course provide students with designated and secure (e.g., password protected) online spaces to store their notes and resources?
· Q30. Does the course have a system of keeping track of student submissions, and online quizzes?
· Q31. Does the course notify students about any changes in due dates or other course relevant matters (e.g., server down) via the following means?
· announcement page
· alert boxes
· running footer added to a page
· phone call
· Q32. Does the course provide back-up materials or alternative activities for students (i.e., what students will do) if any of the following is either not operating properly or unavoidable during a scheduled lesson period?
· Access to the courseware
· Discussion forum
· chat room
· e-mail and mailing list
· books
· online resources
· library materials
· study guide
· instructor
· tutor
· technical support
Resource Dimension
· Q33. Does the course provide guidance on how to organize it for online learning?
· Q34. Does the course provide information or ideas about how many hours (approximately) per week students are expected to spend on course assignments?
· Q35. Does the course provide toll-free telephone numbers for online support services?
· Q36. Does the course provide clear guidelines to the learners on what support can and cannot be expected from a help line?
· Q37. Does the course provide examples of previous students’ work on the Web? If yes, select all that apply and circle whether searchable and browsable:
· projects (searchable/browsable)
· papers (searchable/browsable)
· text dialogue from discussion forums (searchable/browsable)
· text dialogue from online conferencing exchanges (searchable/browsable)
· Q38. Does the institution’s library have library resources online? If yes, do students have access to its databases via the Internet or other networks?
· Q39. Does the host institution’s library have a system of getting books and other materials for students via interlibrary loan?
· Q40. Does the library fax or email any documents to students?
Ethical Dimension
· Q41. To improve cross-cultural verbal communication and avoid misunderstanding, does the course try to reduce or avoid the use of jargon, idioms, ambiguous or cute humor, and acronyms?
· Q42. To improve visual communication, is the course sensitive to the use of navigational icons or images? For example, a pointing hand icon to indicate direction would violate a cultural taboo in certain African cultures by representing a dismembered body part (Reeves & Reeves, 1997).
· Q43. Is the course offered to a geographically diverse population? If yes, is the course sensitive about students from different time zones (e.g., synchronous communications are scheduled at reasonable times for all time zones represented)?
· Q44. Is the course designed to have tolerance for learners who adapt to individualized distributed learning environment more slowly than others?
· Q45. Does the course allow students to remain anonymous during online discussions?
· Q46. Does the course provide any guidance to learners on how to behave and post messages in online discussions so that their postings do not hurt others’ feelings?
· Q47. If a student fails to follow the etiquette of the course, how does the instructor work with students to promote compliance?
· Q48. Does the course provide privacy guidelines on online postings?
· Q49. Does the course provide policies regarding fraudulent activities in course-related testing, assignments, and projects?
· Q50. Does the course get students’ permission to post any of the following on the Web?
· Students’ photographs
· Students’ projects
Institutional Dimension
· Q51. Do faculty receive training on how to moderate and/or maintain a listserv?
· Q52. Does technical and other staff receive training on how to communicate with remote learners in difficult situations?
· Q53. Does the course limit the number of students per faculty member?
· Q54. Does the course provide academic quality such as one would expect in a traditional course?
· Q55. Does the site provide any information regarding whether the course is transferable to other accredited institutions?
· Q56. Is the course offered at a lower fee than on-campus courses? It is not uncommon for students to expect lower fees for online courses.
· Q57. Would student information submitted online to the registrar’s office be kept secure and confidential to the extent possible?
· Q58. Are instructor/tutor and technical staff available during online orientation?
· Q59. Can students purchase packages of course-related supplemental reading materials online from the campus bookstore?
· Q60. Can students apply for financial aid/loans online?