Graduate Program Assessment Overview
By their nature, graduate programs are advanced fields of study that concentrate on providing new knowledge in areas that are often unique to the interests of particular faculty members within specific disciplines. However, at the graduate level, there are generally quite similar expected educational outcomes that are usually independent of the specific field of graduate study. Virtually all graduate students in almost all disciplines are assessed on their: (a) acquisition of advanced knowledge; (b) acquisition of professional, verbal, and written skills; (c) ability to undertake appropriate research, scholarly or creative endeavors, and contribute to their discipline; (d) ability to teach, often at the collegiate level; and (e) ability to find employment in their, or a related, field.
Since, in general, there are no standardized instruments for the assessment of advanced knowledge acquired by individual students in a given discipline, most graduate programs assess this parameter through summary comprehensive written and/or oral examinations. This approach is generally found to be an academically acceptable method of capturing most of the information necessary for graduate student assessment, although more meaningful approaches to assessing student learning are becoming the norm at the graduate level. For programs culminating in a project, thesis, dissertation, or other scholarly and creative works, evaluation of these undertakings by committees of graduate faculty remains the ultimate assessment standard of student success at the graduate level. In some disciplines, an assessment of graduate students’ ability to teach/instruct undergraduates or professional colleagues may also serve as an assessment tool.
In contrast, graduate program assessment is usually accomplished by having each graduate degree program undertake a “self study” of their program. This self-study evaluates program performance given a mission and goal statement, some form of program intended educational outcomes, and the availability of faculty and other resources in the program. In particular, data on student enrollment and quality, faculty scholarly activity, financial and infrastructure support, and evaluation of student learning outcomes is integrated to provide an overall view of the success of the program and plan for program advancement. These data are accumulated over a 5-year period, which at WVU corresponds to the BOG Program Review cycle, and serve as a basis for both annual discussion and for the Five Year Program Review by the University Graduate Council. Together, the faculty, the department’s director of graduate studies, and the department chair usually conduct these self studies. As a result of these reviews, appropriate changes are instituted that improve all aspects of the graduate student experience, as well as improve the competitiveness of program graduates for employment in academics, industry, or government.
Programs use assessment measures appropriate to the discipline and degree. Common to most graduate degree programs is the requirement that students must complete some type of project, thesis or dissertation, or other scholarly or creative work. These undertakings serve as excellent methods of assessing students’ ability to do research, perform advanced skills or techniques, and write or perform in a professionally acceptable manner. Some master’s and doctoral programs feel that assessment outcomes can be measured by identifying the number of degree program graduates that go on to doctoral education or to postdoctoral study. Numbers of contributions to the scholarly literature both during and several years immediately after graduation similarly have been used as a form of program assessment.
The use of assessment metrics or learning outcomes requires that these metrics/outcomes can be quantitatively assessed.
For example, a learning outcome for a master’s student in engineering might be:
Communication skills learning outcome – Upon completion of the master’s degree, a student at the master’s level will demonstrate appropriate oral and written communication skills in the technical aspects of chemical engineering.
For this learning outcome, an appropriate assessment rubric must be used, such as:
- Performance in class presentations and reports in graduate level courses;
- Successful thesis research presentation and defense;
- Research presentation(s) at technical meetings.
Across the graduate program, data on these aspects of the students program of study should be maintained and assessed collectively with other stated learning outcomes. Thus, for this example, an evaluation of success of student communication skills across all students will rely on the overall performance in class presentations (could be a mean grade), successful thesis completion (as a percent), and the number of presentations undertaken by the master’s degree cohort.
Professional programs (e.g., dentistry, law, medicine, counseling, clinical psychology, etc.) often focus on the ability of their graduates to obtain licensure. Assessment is evaluated in these programs by examining pass/fail rates and scale scores for different parts of the licensure exam. These scale scores permit the degree program to concentrate on improving particular components of the professional curriculum. Professional accrediting associations also may play a significant role in dictating or measuring or validating the degree program’s success in accomplishing its, and the profession’s, stated mission and goals.
Assessment as an Iterative Approach to Program Improvement
A reasonably consistent group of criteria is currently being used to assess graduate programs at West Virginia University. Among the educational outcome measures examined are the performance of students and the quality of the programs as reflected in the grades in given graduate courses, the performance of students on qualifying examinations, the number of refereed journal publications, the grants and contracts received, and the career advancement opportunities obtained after leaving the graduate program. Graduate programs use these assessment data to improve their course requirements, introduce new courses, modify program requirements and, in the case of several scientific disciplines, improve their research facilities and equipment.
Thus, program assessment forms the underpinning for the objective evaluation of strengths and weaknesses of a program in meeting its established educational goals. Assessment is not a means of diminishing a program’s role or goals at the University. Rather, its correct use allows program evolution and innovation, both of which provide for a vibrant and stimulating graduate experience at WVU.
Program Assessment and Review
Responsibility for assessment oversight of individual WVU graduate programs has been given to the University Graduate Council. It is this body that conducts the Five-Year Program Reviews required by the University’s Board of Governors for each masters and doctoral program at the university. A significant portion of these reviews involves an analysis of student and programmatic assessment information. Among its present duties, the Graduate Council examines the principal elements of each program’s assessment plan, with particular emphasis placed on assessing student learning and programmatic outcomes. Each degree program is asked to provide information on: (a) educational goals of the program; (b) measures of evaluating success in achieving these goals; (c) identification of the goals that are being successfully met as well as those that need attention as determined by an analysis of the data; and (d) use of assessment data to improve program quality. More information on the Five-Year Review is provided in the following sections.
Photo by WVU Photographic Services