A Model for Evaluating Quality of Web-Based Courses

Rahim Ashkeboussi
Ph.D., Associate Professor of Finance
Frostburg State University
E-mail: ashkeboussi@frostburg.edu

Introduction

The rise of online course offerings by colleges around the world has created both excitement and skepticism among students, educators, administrators, employers, and government officials for different reasons. Students who otherwise would not be able to attend college, due to time, location, course scheduling, or cost constraints, welcome Distance Education initiatives. {Note 1} Administrators and government officials are hopeful that Distance Education is capable of reaching a larger audience at a substantially lower cost per student. While employers are increasingly utilizing Distance Education capabilities for the purpose of their employees' training, they are worried about the educational quality of the graduates from Distance Education programs.

Estimates, based on the results of the most recent survey released by the U.S. Department of Education on the growth of Distance Education, indicate that enrollment in for-credit Distance Education courses grew to 2.9 million in the 2000-2001 academic year from 1.3 million in 1997-1998 {Note 2}.

Naturally, such dizzying growth in online course offerings breeds controversy. Among the hotly debated issues concerning the quality of learning are whether:

1. the extent of learning that normally takes place in a traditional classroom could be replicated online; and
2. virtual classrooms allow the same amount of vigor and interactivity as their traditional counterparts.

Broad (1999) maintains that "the rise of online distance education has presented a major challenge to traditional regional accrediting bodies as well as state regulators and has raised doubts about the ability of our existing quality assurance system to function effectively."

As yet, a quality-assurance mechanism for Web-based instruction is not institutionalized. Traditional courses often go online without any specialized appraisal as to whether such courses provide the same or better learning opportunities as their face-to-face counterparts.

The American Association of University Professors (AAUP), on the issue of Quality Control of the Curriculum, states that:

Distance-education courses (or modifications thereto) shall comply with all of the standard practices, procedures, and criteria which have been established for traditional in the classroom courses including but not limited to, faculty involvement at the level of course development and approval, selection of qualified faculty to teach the course, pedagogical determinations about appropriate class size, and oversight of all final course offerings by the appropriate faculty committee to ensure conformity with previously established traditions of course quality and relevance to programs.{Note 3}

In 1999, the AAUP adopted the Statement on Distance Education, which asserts the fundamental principle that:

As with all other curricular matters, the faculty should have primary responsibility for determining the policies and practices of the institution in regard to distance education. The rules governing distance education and its technologies should be approved by vote of the faculty concerned or of a representative faculty body, officially adopted by the appropriate authority, and published and distributed to all concerned.

AAUP clearly contends that virtual classes must meet a high burden of proof that they truly constitute higher education, and it emphasizes the role of faculty in quality control.

Weiss (1999) states many faculty view online courses as the "high-tech version of the old correspondence courses of notorious diploma mills," in part because those courses have a "disregard for depth of inquiry," as evidenced by their brevity and weak access to learning resources and support services. Generally, the faculty, students, and even the accrediting agencies are inexperienced in terms of what makes an online course or program a quality course or program. Meanwhile, concerned online educators are developing evaluation criteria and methods for assessing online courses. This is evident by the broad array of "evaluation" papers at recent conferences and continuous discussion on different Distance Education listservs, such as DEOS, and IFETS-DISCUSSION. According to Mason (1996), there is a real need to devise evaluation methodologies to assess the value of this new teaching medium.

Accrediting agencies are also concerned about quality in Distance Education. The Council for Higher Education Accreditation (CHEA), in its 1998 report, addressing the issue of quality, made several proposals for improving the reliability and validity of distance learning. The report notes that, in distance learning programs, there is "a greater tendency for the assessment process to be led by the administration instead of faculty, with greater use of outside consultants and assessment 'experts' in lieu of internally generated peer reviews." One example of an outside evaluator is the American Council on Education (ACE), which conducts third-party reviews of distance learning courses to establish college-level equivalency. ACE is offering an annual workshop (Evaluating Distance Learning Courses for College Credit Recommendations) to teach the techniques of evaluating distance learning courses. The 1998 CHEA report concludes that quality assurance appears to be less process-driven and more oriented to "bottom line." To assure quality, the report proposes several conditions {Note 4}:

1. Establish performance measures with evidence for reliability and validity for distance learning
2. Require providers to substantiate evidence of contact between faculty and students
3. Require evidence of effective instructional techniques
4. Promote systematic efforts for selecting and training faculty
5. Assure the availability of learning resources
6. Promote ongoing monitoring and enhancement of the technology infrastructure of institutions
7. Focus attention on the development of courseware and the availability of information
8. Examine alternatives to the traditional accreditation process.

In 2000, CHEA devoted its entire annual conference to addressing the issue of quality assurance, stating that, "Assuring quality in distance learning and the emerging internationalizing of quality assurance are two of the most significant challenges facing CHEA, the accrediting community, and all of higher education."

To address the concerns raised by CHEA and AAUP, the goal of this paper is to propose and outline a systematic process for evaluation of online courses.

Evaluation Paradigms and Literature Review

Barnett (1992) elaborates on three approaches to assessment of quality: objectivist, relativist, and developmental. In relation to Web-based courses, the objectivist approach measures outcomes (the students' grades or dropout rate) against pre-set standards, and judges the success or failure of the course. The problem with this approach is that such standards could be arbitrary and don't provide any information about the qualitative aspects of the learning. The relativist approach measures the outcomes of Web-based education against traditional education based on such attributes as achievement tests, students' attitudes, or satisfaction. Members of a team with a common interest, experimenting with Web-based courses, use the developmental approach to address the weaknesses of such courses and provide solutions for those shortcomings. This is similar to formative evaluation coined by Scriven (1967), which provides for a dynamic and ongoing assessment of a constantly changing environment. This is the environment in which the Web-based courses are being developed and taught. The proposed model for quality assurance, in this study, is based on the developmental approach.

Most of the evaluation models for online courses have been solely confined to the formation of categories/criteria and principles to guide the development of quality online courses. These tasks were often accomplished by either an interested faculty group within a college or university, or a team of faculty experts working on behalf of institutions (such as WebCT) serving the educational entities. {Note 5}

Two major collaborative programs, Innovations in Distance Education (IDE) project, launched in 1995 and Web Initiative in Teaching (WIT) launched in 1998, addressed the issue of quality and were based on the faculty initiative. One outcome of IDE was the development of "guiding principles" as a result of collaboration among the faculty at Penn State, Lincoln, and Cheyney universities. These principles are to provide a philosophical foundation for the design and development of educational programming at a distance. {Note 6}

WIT (as will be described in more shortly) was similarly a faculty initiative program sponsored by the University System of Maryland for development and delivery of quality online courses. Although both IDE and WIT proposed a set of categories/criteria to be met by quality online courses, neither made any attempt in developing a systematic model for assessment of quality for such courses. This study will undertake this task, as described below.

Objectives of this Study

The proposed guidelines by CHEA and the AAUP on Distance Education is a good starting point for formulating strategies and procedures for a systematic approach to quality assurance of Web-based courses. This study recommends a model of "Web-based Course" assessment based on the foregoing framework articulated by CHEA and AAUP. The model in this study, which I call peer review system (PRS), will consider, in detail, the implementation and inclusion of a set of research-based criteria in addressing the issue of quality in Web-based learning. As Distance Education requires change in the role of instructor and students, {Note 7} development and understanding of a new instructional pedagogy for an effective online discourse is crucial. This is especially significant considering that there is skepticism among faculty, administrators, and the public regarding the rigor and quality of online courses. Faculty need training in Web-based pedagogy, mentoring in learner-centered teaching methods, collegial support during the course development process, and recognition for their efforts if they are to develop high-quality Web-based material that are scaleable and adaptable on a wide scale. Thus, the model in this study requires sustained guidance and feedback from peers during the course development process as a means of quality assurance. The goals of a peer review system are to sanction and validate the Web-based courses, and thereby promote wide-scale applicability and long-term sustainability of the accepted criteria, and computer-mediated Distance Education. Widespread adoption of quality online courses by institutions and recognition of these courses by accrediting agencies will help ensure that students are getting consistent educational value for their money and that credits and credentials across institutions are comparable. A peer review system, such as the one proposed in this study, can:

1. validate the quality of web-based learning programs
2. facilitate the transfer of credits among institutions
3. accelerate the acceptance of this form of scholarly endeavor for promotion and tenure; and
4. help to ease the barriers to faculty engagement in developing and teaching online courses.

Current State of Web-based Course Development

Currently, some campuses that are seriously pursing the offerings of Web-based courses/programs initially organize faculty workshops and training sessions to assist faculty on some aspects of online course development (technology, and/or design), while overlooking the pedagogical aspects of Web-based learning. {Note 8} Next, as the interest of faculty in Web-based courses increase, the establishment of a Center {Note 9} to accommodate the technical needs of the faculty while creating and teaching such courses will follow (Donovan 1998).

As an alternative, others might outsource the development of online courses and the training of faculty to teach those courses to an outside firm without the direct involvement of the faculty in the process. Institutions that don't follow a systematic transformation of their traditional programs to Web-based delivery simply leave the individual faculty in charge of developing and offering an online version of their face-to face courses. Obviously, under this scenario there is no orderly institutional support for the design, delivery, and assessment of online courses. Additionally, a proper infrastructure for flexible, scalable, and sustainable online courses/programs will not evolve.

Individual faculty working in relative isolation (Bickerstaff, 1992) developing stand-alone Web-based courses, either voluntarily or at the request of administrators, are in no position to satisfy the conditions addressed in AAUP and CHEA report. Often the faculty designing Web-based courses have little experience and guidance, minimal technical and financial support, and virtually no outside advisement or evaluation. For the most part, faculty engaged in this innovative work use classroom experience and common sense as guides for enhancing the quality of their technology-based material.

Background

The PRS as proposed in this study is based on the lessons learned as a result of my participation in the WIT program sponsored by the Institute for Distance Education of University System of Maryland (USM). Fourteen teams from the 13 USM institutions participated in the WIT program to develop and teach Web-based courses on their respective campuses. WIT participants learned that fundamental issues regarding course design and pedagogy must be addressed in all courses regardless of their content. What makes a quality course may not be easily codified, nor must every course be designed in exactly the same way to assure quality; nonetheless, course attributes such as navigability, disaster recovery plans, technological requirements, and methods of assessing student outcomes, among others, must be carefully considered and accounted for in the design and development of every Web-based course.

The WIT program provided two weeks of intensive training in the summer of 1998, followed by monthly daylong workshops for further training and interaction among the teams as each team was developing their courses over the following academic year. The purpose of the initial training was to bring the participants up to speed on Web-based pedagogy, and instructional design, as they were developing and teaching their courses. The monthly workshops engaged the participating teams in additional learning of the topics covered during the initial training and also served as a forum for exchange of useful comments and suggestions about the courses being developed by different teams. These monthly meetings resulted in a variety of suggestions ranging from relatively mundane but essential observations (such as which choices of backgrounds or fonts make pages difficult to read) to sophisticated guidance on an alternative course infrastructure, a presentation strategy better suited to enhancing student interest and motivation, or techniques for refining assessments or improving interactivity between students or between students and faculty. Most of the training was aimed at the design of Web-based courses to pedagogically accommodate an effective online teaching. The technology related issues were to be handled by a technology savvy member in each team.

The completed courses were piloted in the fall of 1999. An informal peer review of these courses, by outside reviewers, was conducted, without reference to any specific protocol or model of evaluation. This study proposes a formal system (model) to guide the peer review of Web-based courses. The developed categories and criteria discussed in this model are partially based on the collaborative discussion and feedback of WIT teams and a review of literature on effective computer mediated communication.

Approach

The current (i.e., blind) peer review process for academic and literary work has passed the test of time and proved to be a useful system for disseminating researched based hypothesis and accurate and reliable information. In effect, this mechanism is a quality assurance process before the information is dispersed to the public at large. Since a Web-based course is communicated through a different medium it engages instructional technologies that are not present in a face-to-face course. Furthermore, to effectively communicate with off-campus students, its pedagogy must be different from a face-to-face course. In other words, it must be a learner-centered and highly interactive learning environment by design. It should facilitate different learning styles and heavily rely on Web-based instructional technologies. Thus the appropriateness of the employed instructional technologies and pedagogical techniques as utilized in Web-based courses should similarly be peer reviewed and evaluated before its release to the public.

The PRS begins with collaboration among faculty members, across a university system or unrelated campuses, to formulate a faculty-driven process to improve the quality and accountability of web-based instruction. The partnership should be formed by the initiatives of faculty with the support of the provosts of participating colleges and universities based on clear incentives for the faculty interested in developing and teaching online courses. Incentives could be in the form of release time, monetary grants, or credit towards professional (scholarly) or teaching activities. Through the resources provided by the participating institutions, faculty across different campuses will work to refine, test, validate, and disseminate a set of criteria for evaluating web-based courses. Upon the approval of the quality assurance criteria by the faculty body, the sponsoring institutions should adopt these criteria for assessing the quality of Web-based courses. Additional benefits of such collaboration are avoiding duplicating efforts and saving of resources for the participating institutions.

Four Phases

The PRS is composed of a set of empirically validated criteria that can be used to identify poor, acceptable, and outstanding web-based courses. These criteria must accommodate rapidly evolving practice and must not be a rigidly defined set of standards. Ultimately the goal of the PRS is to provide a summative evaluation of the design, delivery, and assessment techniques of web-based courses. This summative evaluation will then inform and educate faculty about the criteria and provide additional advice and guidance to faculty who are actively involved in developing and delivering web-based courses. To this end, the following four phases are considered appropriate.

Phase 1

The first phase of the PRS will involve the development of a set of "rough" variables based on the learned experience of the participating faculty teams throughout the training and developing/teaching of their online course. These variables will then be refined, categorized, and compared against those in the existing literature on Web-based course evaluation. This process could immediately validate a part of "rough" variables. In other words, those variables supported by literature are considered relevant.

WIT participants, in groups of 6 to 7 members, during two brainstorming sessions collectively produced 86 variables for evaluating their Web-based courses. As each group generated these proposed variables independently, they were first refined to avoid duplication, and then organized under the following 11 criteria:

1. Syllabus
2. Learner-centered approach, design (Alonso)
3. Instructional design/learning strategies
4. Interactivity (Alonso)
5. Use of Media
6. Use of WWW
7. Internal consistency and clarity
8. Assessment
9. User interface, graphic design, formatting
10. Navigation
11. Access, technical robustness and student support

Current literature (see Ragan, Nielson (1997), Boshier (1997), Graham (2001)), has addressed a mix of major categories and criteria to be considered when evaluating quality in Web-based courses. The developed variables by the WIT participants (WIT variables) were consistent with those proposed in the literature. This study classifies the WIT variables under 3 major categories: design, delivery, and assessment. The three categories are depicted below in bold print, with the relevant criteria of each category underlined and the variables in italics.

•   Design
  •   Pedagogy
    •   Course Content/objectives (Khan, 1997)
    •   Engagement (keep the learners interest)
    •   Learner centered
    •   Assignments (clarity about the scope & form)
    •   Instructor's feedback, mentoring, motivation, and facilitation (Overmyer, 1998)
  •   Use of media in support of learning and learning styles (Khan, 1997)
    •   Use of audio and video clips
    •   Use of simulation
    •   Conferencing (synchronous, asynchronous)
  •   Interactivity
    •   Learner/learners
    •   Learners/moderator
    •   Learner/course material-assignments
    •   Learners/resources (library; advising; academic support services)
    •   Sense of belonging to a learning community
  •   Navigation
    •   Avoid long scrolling pages
    •   Avoid constantly running animation on pages
    •   Avoid complex URL
    •   Create a link to home page from every page
    •   Create links among similar and related concepts (ideas)
    •   Avoid non-standard link colors
    •   Weed out outdated information
    •   Include a search feature
    •   Be concerned about the bandwidth
  •   Use of WWW resources (consistent with the course objectives)
    •   Relevance
    •   Suitability of the material (Khan, 1997, p.375)
    •   Sustainability
  •   Web dependent vs. software dependent contents (Khan, 1997, p.375)
•   Delivery
  •   Experience of the moderator with teaching online (Khan, 1997, p.375)
  •   Support system
    •   Academic
    •   Administrative
    •   Technical
  •   Contingency strategies (alternative communication modes)
•   Assessment
•   Students
  •   Instrument (online/monitored quiz or exams, posted or emailed assignments)
  •   Grading techniques (learner or moderator scored, software scored; other arrangements)
  •   Authenticity of student's work

The emphasis on "design" is to alert the instructor/designer that a Web-based course cannot be effective when one simply transfers the existing course material from a face-to-face course to a computer mediated course (Parson 1997, and Walsh 1997) and ignores the Web capabilities and the use of other instructional technologies in the process. Boling and Sousa (1993) stress that "if people cannot use what is being delivered to them, or if they will not use it because it fails to support them in crucial ways, the promise of technology is subverted before it can begin to be fulfilled." A poor interface design can prevent students from learning. Thorp (1998) states, "Technology should not drive pedagogy." Thus the evaluation Web-based courses should carefully pay attention to the usage and manipulation of employed technologies and how they contribute to the soundness of the Web-based course pedagogy. Kearney (1997) contends that online learning requires more active and self-directed learners, which can be a major change for those who tend to be passive about their education. Burge (1994) found that students had specific expectations of their online peers in the areas of timely participation and feedback throughout an online discourse. He asserts that instructors should encourage creativity, reflective thinking, and self-directed learning through a well-structured online class. Thus, the pedagogy of online learning should accommodate a learner-centered model of learning. Beyond the issue of interactivity, Broad (1999) suggests that institutions and programs that deliver online learning opportunities should ensure "their online students have access to an appropriate level of academic and student support services." Overmyer (1999) has moderated a discussion board and reported that the contributors envisioned the intelligent pedagogical agent's (teacher's) role in an overall pedagogical strategy for Distance Education should be tutoring, mentoring, motivating, and facilitating. These issues are addressed above under the pedagogy criteria.

Jones and Okey (1995) indicate the importance of incorporation of screen and interface designs in the developing of a computer-based learning medium. These are important aspects as they are related to the ease of navigation and students interactivity with the course contents. Squires and Preece (1996) argue that in the design of a Web-based course the designer should provide an attractive and easy to use interface. Additionally, "usability by learners while performing their tasks" (Dillon and Zhu, 1997) is an integral component of an efficient course design. The variables identified under the navigation criteria above are mostly those identified by Nielson (1996) and adopted by others. Wagner (1994) believes that "...interaction functions as an attribute of effective instruction, while interactivity functions as an attribute of contemporary instructional systems, particularly those that use telecommunications technologies." Interactivity has been a controversial subject in Distance Education, as the opponents of distance learning has often questioned the quality of online course and its capability of providing an appropriate platform for interaction among the students and the instructor. Thus the need for research on interactivity in Distance Education literature is frequently mentioned (McNabb 1994; Sherry 1996). Educators believe that interactivity is a vital element in the educational process. As noted by Wagner (1997, p. 19) "distance learning practitioners --- particularly instructors and program administrators --- seem to view interactivity as the defining attribute of contemporary distance learning experience." Simonson (1995, p. 12) maintains that educators must strive "...to make the experience of the distance learner as complete, satisfying, and acceptable as that of the local learner." Cronje (1999) taught a graduate Distance Education class and observed that students created their own support system by e-mailing classmates for help, to work on their cooperative work projects. Thus, cooperative work projects provided a framework for students to develop deeper relationships with others. These online discussion groups operated successfully during the course and students reported positive benefits. This is a good example of how carefully designed interactive assignments can create a sense of community for students in distance.

Learners at a distance may not have access to a wide variety of support systems that are readily available to resident learners. These support systems are of three basic types, academic (library, advising, tutorial, bookstore), technical (software, hardware), and administrative (registration, job placement). The variables developed are shown above under the Support system criteria. Attention should also be given to integration of contingency plans for possible "disasters" (i.e. what to do if the server goes down on the day a project is due) into the course design.

Phase 2

To evaluate the newly created/existing courses being taught by the participants, the second phase of the development of the PRS will involve the design of a survey instrument (questionnaire) that addresses all the refined variables identified in phase one and their relevance to the quality of the course. This survey will be administered to both students taking the course and outside course monitors. Outside monitors for each course are two or three outside faculty, assembled by each instructor, who periodically observe the management of the online course, and use the questionnaire to record their assessment of the different aspects of the course.

Phase 3

Phase three will focus on analyzing and comparing the data generated from the two distinctive groups. The goal of this analysis is to make a comparison between the two independent survey results and point out the observed similarities and differences in ratings for each category by the "course monitors" and students for each course. This study proposes factor analysis as an appropriate model for examination of the data. Factor analysis requires calculation of the correlation matrix among the variables for each group. Each correlation matrix would then be factor analyzed to determine how high each variable loads on the criterion (factor) under which the variables were clustered. The highest loaded variables on a criterion are the best instances (representation) of that criterion. A comparison of the variable loading for the two groups would reveal whether or not both groups have equally signified the importance of the variables in the assessment of the online course. This approach will further allow drawing conclusions on the significance of each criterion (factor) in explaining the quality and effectiveness of the courses.

Phase 4

In the fourth phase of the PRS, results of the analysis in the third phase will inform the development of the final set of criteria for the evaluation and construction of web-based courses. The experience of the WIT participants strongly suggests that high-quality web-based course design can only evolve from the collaborative efforts of faculty teams and instructional designers who can rely on a structured training facility. Frequent and thorough evaluation of web-based courses during all phases of design by other faculty engaged in the design of web-based courses, students, and ultimately course monitors, can provide valuable feedback to instructors/course designers. This feedback allows faculty/course designers to identify strengths and weaknesses in the course under design and helps to ensure quality courses.

Concluding Remarks

The PRS would improve upon current practice in several ways. First, faculty designing a course would have clear, concise, empirically validated criteria for quality Web-based courses. The guidelines in the PRS would systematize and summarize this information, shifting the faculty's job of course design to guiding and mentoring students.

The collaborative nature of PRS allows faculty to interact with their peers and draw upon their collective experience to further enhance the quality of their online courses. The WIT participants have learned first-hand how a valuable ongoing review of this sort can be.

Courses reviewed under the PRS will have the imprimatur of an independent body attesting to their quality. Prospective students shopping for courses can select from among those that have been reviewed under the PRS. Faculty who have developed these courses can receive recognition for their superior efforts. When the faculty is considered for merit raises, promotion, or tenure, this seal of approval will attest to their courses as refereed work.

Institutions contemplating committing their resources to the development of Distance Education programs want to justify those expenditures. Since no universally accepted protocol for overseeing the integrity of web-based courses exists, establishment of criteria by which these courses may be evaluated will provide universities the necessary assurances. In addition, acceptance and application of these quality assurance criteria will help facilitate the transference of academic credits between institutions. Likewise, accrediting bodies such as the AACSB, NCATE, and others are in search of standards for approving academic programs that are offered either partially or entirely online. If any national or international standard of quality for Web-based courses is to be developed, a consistent set of criteria for evaluating those courses is essential.

Works Cited

Alonso Diane.

"Forms of Controlled Interaction as Determinants of Lecture Effectiveness in Electronic Classroom." <http://lap.umd.edu/LAPFolder/Papers/dianesthesis/page1.html>.

 

American Association of University Professors.

"Statement on Distance Education." 1999 <http://www.aaup.org/spcdistn.htm>.

 

Barnett, Ronald.

"Improving Higher Education: Total Quality Care." Buckingham: SRHE and Open University Press. 1992.

 

Bickerstaff, Douglas D.

"CBT Training of CAI Developers: Conference Record for 1992 IEEE Fifth Conference on Human Factors and Power Plants." 1992. 432.

 

Boling, E.S., and Sousa, G.

"Interface Design Issues in the Future of Business Training." Business Horizon 36.6 (1993): 54 - 61.

Boshier, Roger W., et al. "Best and worst dressed web courses: Strutting into the 21^st century in comfort and style."Distance Education Journal 18. 2 (1997).

 

Broad, Molly C.

"The Dynamics of Quality Assurance in Online Distance Education." e-JIST 3.1 (1999). <http://www.usq.edu.au/electpub/e-jist/vol3no1/index.htm>.

 

Burge, Elizabeth J.

"Learning in a computer conference contexts: The learners' perspective." Journal of Distance Education 9.1 (1994): 19.

 

CHEA 2000 Annual Conference.

"Quality Assurance, Distance learning and International Perspectives and Needs." Washington DC, January 24-26, 2000.

Cronje, Johannes. "Using the Internet to facilitate co-operative distance learning." 1999. <http://hagar.up.ac.za/catts/rbo96.html>.

 

Dillon, Andrew, and Zhu, E.

"Designing Web-based Instructions: A Human Computer-Interaction Perspective." In Khan, B.H. Web-Based Instruction. Englewood Cliffs, New Jersey: Educational Technology Publications,1997.

 

Donovan, Mark, and Macklin, S.

"One Size Doesn't Fit All: Designing Scalable, Client-Centered Support for Technology in Teaching." EDUCAUSE Conference on Information Technology in Higher Education. Seattle Washington: 8 Dec. 1998.

 

Graham, Charles, et al.

"Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses" The Technology Source. March/April 2001 <http://ts.mivu.org/default.asp?show=issue&id=41>.

 

Jones, Marshall G., and Okey, J.R.

"Interface Design for Computer-based Learning Environments." 21 Feb. 1995 <http://ddi.cs.uni-potsdam.de/HyFISCH/Multimedia/Learning/InterfaceDesignJones.htm>.

 

Kearney, Thom.

"Self-directed learning at Algonquin College." 1997 <http://thenode.org/tfl/notes/kearney.html>.

 

Khan, Badrul H., (Ed.)

Web-based instruction, Englewood Cliffs, NJ: Instructional Technology Publication, 1997.

 

Mason, Robin.

"Methods for Evaluating Applications of CMC." OU Press - PLUM Paper #31. 1996.

 

McNabb, Jeffrey.

"Telecourse effectiveness: Findings in the current literature." TechTrends 39.4 (1994): 39-40.

 

Nielson, Dean H.

"Quality assessment and quality assurance in distance teacher education." Distance Education Journal 18.2 (1997).

 

Nielson, Jakob.

"Top Ten Web Design Mistakes." 1996. <http://www.useit.com/alertbox/9605.html>.

 

Overmyer, Scott.

"If only my intelligent pedagogical agent could...!" A summary of online discussion on Educational Technology &Society's Formal Discussion Board. 12 Sept. -- 7 Oct., 1998. http://ifets.gmd.de/periodical/vol_1_99/formal_discussion3.html>.

 

Parson, Robert.

"An Investigation into Instruction." 1997 <http://www.osie.on.ca/~rparson/out1d.htm>.

 

Phipps, Ronald, Wellman, J.,and Merisotis, J.

"Assuring quality in Distance Learning, A Preliminary Review", A Report Prepared for the Council for Higher Education Accreditation by the Institute for Higher Education Policy. Washington DC, April 1998.

 

Ragan, Lawrence C.

"Good Teaching is Good Teaching: An Emerging Set of Guiding Principles and Practices For the Design and Development of Distance Education." <http://www.vpaa.uillinois.edu/reports_retreats/tid/resources/penn-state.html>.

 

Scriven, Michael.

"The Methodology of Evaluation." In Stake, R., (Ed.), Perspectives of Curriculum Evaluation. Chicago: Rand McNally & Company, 1967.

 

Sherry, Lorraine.

"Issues in distance learning." International Journal of Educational Telecommunications 1.4 (1996): 337-365.

 

Simonson, Michael.

"Does anyone really want to learn ... at a distance?" TechTrends 40.5 (1995).

 

Squires, David, and Preece, J.

"Usability and Learning: Evaluating the Potential of Educational Software." Computer Education 27.1 (1996): 15.

 

Thorp, M.

"Keynote presentation for the Open Learning Forum." IET of the OU. University of Sussex/England. 1998.

 

Wagner, Ellen D.

"In support of a functional definition of interaction." The American Journal of Distance Education 8.2 (1994):.6-29.

 

Wagner, Ellen D.

"Interactivity: From Agents to Outcomes." New Directions for Teaching and Learning 71 (1997): 19-26.

 

Weiss, Kenneth R.

"California Virtual University has a Collision with Reality." LA Times, 8 April, 1999.

 

Welsh, Thomas M.

"An Event-Oriented Design Model for Web-Based Instructions." In Khan, Badrul H. (Ed.), Web-Based Instruction, Englewood Cliffs, NJ: Instructional Technology Publication (1997): 164.

Footnotes:

1. "A Survey Documents Growth in Distance Education in Late 1990s." The Chronicle of Higher Education 8 Aug. 2003:.A28  [<<Back]
2. Ibid.  [<<Back]
3. American Association of University Professors. " Special Committee on Distance Education and Intellectual Property Issues."
       <http://www.aaup.org/deguide.htm>.  [<<Back]
4. Council for Higher Education Accreditation. "Assuring quality in distance learning." Washington, DC: Institute for Higher Education Policy. Apr. 1998
       <http://www.chea.org/>.  [<<Back]
5. For a list of these models see Minnesota State Colleges and Universities, ITS Office of Instructional Technologies:
       <http://www.mnscu.oit.edu pages/qualitymodels.htm#rubrics>  .[<<Back]
6. Ragan, Lawrence, C. "Good Teaching is Good Teaching: An Emerging Set of Guiding Principles and Practices for the Design and Development of Distance Education."
       <http://www.vpaa.uillinois.edu/reports_retreats/tid/resources/penn-state.html>.  [<<Back]
7. For Distance Education, the instructor's role is more of a facilitator (mentor) than the sage, and the students' role is more of investigation than regurgitation of knowledge due to emphasis on a more learner-centered classroom environment. See Phillips, Vicky.
       <http://archive.salon.com/21st/feature/1998/01/20feature.html>.  [<<Back]
8. To review a list of such approaches see: Minnesota State Colleges and Universities. "ITS Office of Instructional Technologies."
       <http://www.mnscu.oit.edu pages/qualitymodels.htm#rubrics>.  [<<Back]
9. For example Innovations in Distance Education (IDE) project, assisted faculty at Penn State, Lincoln and Cheyney universities to create a supportive institutional culture in which the possibilities of distance education could be realized. Ragan, Lawrence C. "Good Teaching is Good Teaching: An Emerging Set of Guiding Principles and Practices for the Design and Development of Distance Education."
       <http://www.vpaa.uillinois.edu/reports_retreats/tid/resources/penn-state.html>.  [<<Back]

Academic Exchange Extra invites reader responses to any writings in this issue--especially articles advancing the scholarly debate of issues raised.

Page Viewed:   / Created: 16 September 2003 / Updated: --