Assessment in Engineering Education: Evolution, Approaches and Future Collaborations

ABSTRACT

This article examines the current state of assessment in engineering education in the United States as reflected in the Journal of Engineering Education. We begin with a brief review of recent developments in the assessment of engineering education and the events that have inspired change. Next, we explore assessment methodologies that have been used repeatedly in the evaluation of engineering courses, curricula, and research investigations as well as some methods that have not been used extensively but are likely to be informative. We conclude with a discussion of the importance of establishing collaborations between researchers in engineering education and educational research. Throughout this paper we highlight examples of sound and rigorous assessments in engineering education.

Keywords: assessment, evaluation, engineering education

I. INTRODUCTION

The advancement of engineering education in many ways depends on assessment. High-quality assessments can provide educators with information they can use to move the field forward. Inadequate or poorly constructed assessments can cause educators to pursue ineffective paths, resulting in the loss of time, money, and energy. Researchers have also recognized the central role that assessment plays specifically in engineering education. According to Felder, Sheppard, and Smith, "Research, by its nature, requires effective assessment. The infusion of accepted principles and practices of educational assessment are having a significant impact on the development of engineering curricula and the evaluation in terms of student performance" [1]. We use examples from recent articles published in the Journal of 'Engineering Education (JEE) to examine current assessment practices and methods in engineering education in the United States. We acknowledge the many contributions to the literature on the assessment of engineering education that have been reported in other venues, both within engineering education and external to engineering education (pre-collcge educational research, national studies, etc.) However, we have elected to limit our discussion to articles published m JEE because of considerations for the length of the paper and because JEE offers a rigorous review process with an emphasis on engineering education. The purpose of this article is not to present a comprehensive literature review about assessment, but rather to highlight good practices within engineering education as reflected through JEE and to suggest some possible future directions and challenges. Because of the central nature of assessment to engineering education, several other articles in this issue address assessment as a related topic.

The language of assessment can vary slightly from author to author. Therefore, we begin this discussion with a definition, taken from Rogers and Sandos [2], of three key terms as they are used within this paper. The term assessment is used here to refer to the act of collecting data or evidence that can be used to answer classroom, curricular, or research questions. In other words, we use the term assessment in a broader sense than measuring individual student's competencies, such as scores on a classroom exam or homework assignments. Assessment methods are defined here as the procedures used to support the data collection process and are an important consideration in any educational research design. Evaluation, which is often used synonymously with assessment, refers to the interpretations that are made of the evidence collected about a given question. A complete summary of the assessment terms and definitions used in this paper appears in the Appendix.

In the next section, we provide a brief review of recent developments in the assessment of engineering education and reflect upon the events that have inspired change. Events of interest include the accountability mandates of the mid-1980s and recent changes in the engineering accreditation process instituted by ABET. Depending upon the setting and the purpose of the assessment, different methodologies are likely to be appropriate. The majority of this paper is dedicated to a discussion of assessment methodologies that have been used in engineering education. We conclude with a discussion of the importance of establishing collaborations between researchers in engineering education and educational research.

II. RECENT DEVELOPMENTS IN ASSESSMENT IN ENGINEERING EDUCATION

Since approximately the mid-1980s, there has been increasing pressure on institutions of higher education to be accountable to their constituents, including state higher education commissions, alumni, students, parents, and accreditation agencies. Accountability in this context refers to institutions taking responsibility for and demonstrating the effectiveness of their educational programs. Many states (for example, Tennessee and Colorado) passed laws in the late '80s requiring public universities to provide annual reports on their assessment of student outcomes. A quick examination of the guidelines for any of the regional accreditators, such as the Southern Association of Colleges and Schools and the Higher Learning Commission of the North Central Association, reveals a strong emphasis on outcomes-based assessment. In addition, many professional accreditation groups, including engineering, now require the assessment of student outcomes as part of their accreditation process.

The accountability mandates led to a variety of curricular reforms and, perhaps most significantly for engineering education, the revision of the guidelines for accreditation by ABET. For a detailed description of the history behind these events, see the article by Prados, Petcrson, and Lattuca in this issue [3]. A number of key studies strongly influenced the change in the ABET engineering criteria, including Systemic Engineering Education Reform: An Action Agenda (1995) [4]; The Green Report: Engineering Education for a Changing World (1994) [5]; and Engineering Education: Designing an Adaptive System (1995) [6]. ABET, which was established more than seventy years ago, currently accredits more than 2,500 engineering programs across the United States, directly affecting 500 institutions of higher education [7]. Because of AB ET's widespread influence, the revision of the ABET engineering criteria has even had an international impact on engineering programs. Previously, ABET's engineering criteria required institutions to report numerical summaries that reflected their programs and resources and to focus on "inputs" (e.g., number of credit hours, volumes in the library, faculty with Ph.D.s) rather than "outcomes" (what students know and are able to do). With the new ABET engineering criteria, rather than providing numerical inputs to the educational process and implying educational quality, institutions must now directly demonstrate through assessment and evaluation that they are reaching the desired outcomes. Particularly challenging for engineering institutions seeking accreditation has been ABET Engineering Criterion 3:

"engineering programs must demonstrate that their graduates have

(a) an ability to apply knowledge of mathematics, science, and engineering;

(b) an ability to design and conduct experiments, as well as to analyze and interpret data;

(c) an ability to design a system, component, or process to meet desired needs;

(d) an ability to function on multidisciplinaiy teams;

(e) an ability to identify, formulate, and solve engineering problems;

(f) an understanding of professional and ethical responsibility;

(g) an ability to communicate effectively;

(h) the broad education necessary to understand the impact of engineering solutions in a global and societal context;

(i) a recognition of the need for, and ability to engage in, life-long learning;

(j) a knowledge of contemporary issues;

(k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice."