METHODOLOGY

This project is divided into three phases as shown in Figure 1. Phase1 is concerned with system definition, development of an evaluation protocol, identification of methodologies to investigate, inventorying existing instruments (or tools) that can be used to implement the methodologies, determination of innovations to assess, and formulation of an experimental design to enable innovations to be assessed at multiple institutions using alternative methodologies. 

Phase 2 is concerned with carrying out the evaluation of the various assessment methodologies. To do this, some methodologies will require further research; e.g., verbal protocol analysis and concept maps, that will be carried out during this phase. As part of our evaluation, we will assess a number of the engineering education innovations being developed and tested at one or more of our participating institutions (under separate funding). 

Phase 3 involves the evaluation of the methodologies and the preparation of the outcome assessment case studies. We will focus on dissemination in this phase, preparing web-based versions of our instruments for ease of use and scoring. Papers describing the results of these activities will be distributed. 

Figure 1: Project Framework

Phase 1:

1.1. Conceptualization of the Engineering Education Systems

This first phase will address the conceptualization of the engineering education system. We will develop a set of outcomes and relate those to educational processes. Inputs for the system will also be defined. These elements will be combined into a "working" model of the educational system (that will evolve as the project progresses). This important first step will also enable us to establish a "common language" and to agree upon definitions for all relevant terminology [16]. We will utilize this common language as we proceed. This will enable us to reach consensus more readily concerning the points in the educational process where we want to measure outcomes. During this phase we will also develop our protocol for evaluating each of the different assessment methodologies.

1.1.a. Identification of Outcomes ("ABET 11" and Others)

ABET has specified eleven "generic" student outcomes of a modern engineering education program. We will examine each outcome, adding specificity and definition. We will also propose additional, complementary outcomes; e.g., ability to integrate knowledge; ability to innovate. Each outcome will be precisely defined so that it is "transportable" across our collaborating institutions. 

1.2. Determine a set of measures for each outcome.

For each of our proposed outcomes, we will develop a set of measures. The goal is to assure that each measure has meaning; i.e., the faculty will be able to use the measure consistently to make valuable inferences about student learning and/or achievement. 

1.3. Selection of Assessment Methodologies and Instruments for Outcome Measurement

There are a number of methodologies that have been used successfully in other areas with potential for assessing engineering education outcomes [17-23]. We will catalogue appropriate methodologies and existing instruments and tools for assessment. We will rely on both our collective knowledge of the field and will utilize the resources of a growing number of national assessment "repositories"; e.g., NISE resources [24], National Center for Higher Education Management Systems [25], AAHE [26], ASEE [27], NEEDHA [28]. From our list we will select the methodologies for detailed study. Table 1 contains a list of potential assessment methodologies and potentially applicable outcomes. We will conduct an extensive review of the current state-of-the-art of this rapidly changing area with the expectation of adding to the list. A brief overview of each potential method and how it may be used in this project is presented under the Phase 2 description.

1.4. Identify Instruments (Tools) To Be Developed.

A basic assumption of this project is that some instruments currently exist for assessing educational programs, and these can be adapted to engineering. Progress in this area has been made by a number of engineering educators, including the principals of this project. However, we realize that some instrument development will be necessary. In doing this, we will follow established practices for instrument design including performing validity and reliability checks as part of this process. 

1.5. Selections of Outcomes and Innovations to Measure by Partner Institutions

Each partner institution, having specified the assessment methodologies it wishes to investigate, also will identify those outcomes of primary interest, as well as innovations that can be assessed by measuring specific outcomes. We are particularly interested in assessing specific forms of innovation at several institutions using multiple methods. For example, both Drexel [29] (E4) and RHIT [30] (IFYCSME) have well-established, highly integrated freshman curricula; NCSU offers another version of a first year innovation which will be scaled up from 300 to 1,100 students next year [31]. UTEP has yet another integrated first year that creates learning communities for its engineering students. These three institutions provide an exciting assessment opportunity that we intend to use to this project’s advantage.

Outcome Measure

Physical Portfolios

Electronic Portfolios

Student Surveys

Alumni Surveys

Student Interviews

Focus Groups

Competency Measurement

Student Journals

Concept Maps

Verbal Protocols

Intellectual Development

Authentic Assessment

Apply math, science, and engineering

ü

ü

ü

ü

ü

ü

ü

ü

Design/conduct experiments

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

Design component or system

ü

ü

ü

ü

ü

ü

ü

ü

ü

Function on multi-disciplinary teams

ü

ü

ü

ü

ü

ü

ü

ü

Identify, formulate and solve problems

ü

ü

ü

ü

ü

ü

ü

Understand professional, ethical responsibility

ü

ü

ü

ü

ü

ü

ü

Ability to communicate effectively

ü

ü

ü

ü

ü

ü

ü

ü

ü

Engineering in a global and societal context

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

Recognize need for life long learning

ü

ü

ü

ü

ü

ü

ü

ü

Knowledge of contemporary issues

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

ü

Use modern engineering tools

ü

ü

ü

ü

ü

ü

Ability to integrate knowledge

ü

ü

ü

ü

ü

ü

ü

ü

Positive attitude about  Engineering

ü

ü

ü

ü

ü

ü

Student satisfaction with education

ü

ü

ü

ü

ü

ü

ü

 

1.6. Determine "Experimental Design" for Project 

This project involves faculty from five universities examining approximately twenty outcomes using more than ten assessment methods, and applying them to a series of innovations. To maximize the value of the information obtained, we will develop an overall " quasi-experimental design" which relates outcomes to proposed methods by institution, resulting in a "triangulation" process; i.e., the use of more than one method to assess each outcome.

1.7. Establish a Process for Evaluating Each Assessment Methodology

As part of the evaluation plan for this project, we will utilize the following process to develop a protocol for assessing each of the methodologies that we will utilize. Our evaluation criteria will include documentation of strengths and weaknesses; cost to implement/maintain; reliability, validity and other standard measures.

We will review assessment methods for specific psychometric properties including reliability and validity; i.e., will the method consistently predict differences in student performance? In addition, we will identify what is known about comparative strengths and weaknesses of each of the assessment methodologies represented. This information will be used to develop informed assessments of the application of these tools to engineering education. Special emphasis will be placed on appropriateness in different academic settings, the "politics" of methodological choice (i.e., barriers and enhancers to implementation), and how easily data can be transformed into information to improve engineering programs. We will conduct this review from the perspectives of faculty, students, and administrators. 

As part of this and succeeding tasks, new methods will be tested to assure that they meet certain criteria for effectiveness. Multiple criteria to evaluate measurement strategies will be employed. A major criterion for selection is the extent that the measurement approach will yield information that is useful to the involved faculty. Collecting data in an educational environment is a relatively easy task; applying the results is another thing. Finally, ease of administration and cost effectiveness will be evaluated for any assessment methods under review.

1.8. Development of a working model of the engineering education system 

For each outcome defined above, we will hypothesize the program elements and processes that affect it. We are particularly interested in delineating the processes within the engineering education system and understanding how faculty can control important aspects of those processes. Such processes may include critical or core processes (e.g., curriculum, culture and class learning), as well as secondary processes (e.g., mentoring and advising. financial aid, etc. that support the core processes. We also will classify the inputs to the system, the students. We will consider the demographic descriptors that should be used to define student populations. Factors such as SAT scores, class rank, geographic location, quality of high school attended, level of scholarship/financial aid, admission status (e.g., direct admit, transfer, probation) ethnicity and gender will be examined. Other inputs may include financial resources of the institution, facilities, and support from state and local government as well as from private sources. 

Utilizing the outputs, processes and inputs, we will develop a "working model" of the engineering education system, that will serve as a framework for conducting this study. Here we will relate the processes that make up the system to its outcomes. Aldridge [32] and Besterfield-Sacre [33] among others have proposed such models. 

We will utilize our conceptual model as a framework for identifying relationships among the student, the educational process, and the outcomes. By modeling the system we can better interpret the results we obtain with our various assessment methodologies. Modeling will also enable us to better utilize the outcome measurements we obtain for decision making. Conversely, the assessments will enable us to enrich our "working model." To date, our educational models have enabled us to identify those students most likely to leave an engineering program during their freshman year [34]; determine the impact of co-op and internships on the educational process [35]; and develop quality control charts for tracking student attitudinal changes [36].

Phase 2: Implement and Extend Methodologies; gather and analyze data for Outcome Assessment

During this phase we will carry out the research into each of the assessment methodologies identified in Phase 1. As noted, our effort will be directed at adapting proven methodologies from other areas for the assessment of specific engineering educational outcomes, developing new instruments where needed. These are discussed below:

2.a. Physical portfolios are a collection of student work, usually gathered over time. The material collected is determined by what student outcomes are being assessed — for example, writing samples are collected to assess student writing, while project reports, team meeting minutes, student notebooks, etc. are collected to assess students’ design knowledge and skills. However, a single, well-chosen portfolio entry can be used for multiple measures. The student or instructor can make decisions about what materials are included in a portfolio; materials may be from one course, a series of courses, or the entire curriculum. Often, students are asked to include a self-reflection piece in the portfolio to help them view their own growth and areas where improvement is needed. Portfolios can be used for both formative and summative feedback. All these features of a portfolio make it a strong candidate for outcomes assessment, particularly with respect to EC-2000 outcomes design, communications, and contemporary issues [37-41].

Proposed research areas: Portfolios are an excellent triangulation device — a qualitative way of validating such quantitative measures as transcript evaluation and student attitudinal surveys. We need to develop rubrics for portfolios that will allow them to be used for cross-institutional assessment. We also need to explore ways in which electronic and other types of portfolios can be used more efficiently than traditional portfolios, which are generally time-consuming to maintain and evaluate.

2.b. Electronic portfolios (eportfolio) are a method that allows students to submit evidence of their progress toward achieving learning outcomes in an electronic format. For each student, outcome goals are defined with specific performance criteria. Students must evaluate evidence from their collegiate experience and make decisions about how it relates to the performance criteria. Students must write brief reflective statements as to how the evidence relates to specific criterion(a). The evidence may be course-based, co-curricular, or from other experiences within the collegiate experience (e.g., co-ops, internships). Teams of faculty evaluate the evidence using rubrics to rate the level of competence the student has achieved. This information will be used to make decisions about the academic program. As this process is being developed at Rose-Hulman, the student is responsible for submitting evidence to his/her own eportfolio. The eportfolio has a web interface with a secure site for each student. It has an electronic database structure that supports a variety of electronic formats (standard software packages, html, pdf, digitized video, audio, etc.) The electronic database will enable faculty assessors to search across multiple files and extract data for assessment. The Rose-Hulman model minimizes the work that faculty do in the data collection process and puts the emphasis on faculty as the ones who are responsible to evaluate the students’ work in regards to specific, focused criteria. 

Proposed research areas: What are the implementation costs of an eportfolio system? What is the educational value for students? How can you get students to participate in the project? What criteria should be used to sample student work? What is the ease of the use of eportfolios for faculty assessors? How is assessment data transformed into information that can be used to improve the engineering program? How can a cost/benefit metric be established as a measure of the eportfolio project effectiveness?

2.c. Closed-form Questionnaires - Attitudinal Surveys are a practical method for evaluating student or alumni attitudes about engineering, aspects of their education, and their self-assessed abilities and competencies. Closed-form questionnaires are less costly to develop, administer and analyze than other types of assessment methodologies, particularly if a large data set is being collected and if statistically reliable conclusions are desired. By limiting the response choices data collection can be repeated overtime. Thus, we can examine how attitudes are affected by particular interventions, change over time, or vary among groups of individuals. Like other methods described, a good closed-form questionnaire design requires considerable knowledge and skill if results are to be valid. We have developed closed-form questionnaires at both the student and post-graduation level. At the student level we have measured attitudes freshman have about engineering and their self-assessed abilities and observed how these measures change as a result of their educational experiences [42-44]. To date, this instrument has been adopted by eleven engineering programs, and is being examined by 20 others. We also have developed and implemented a closed-form questionnaire to assess engineering alumni/ae attitudes about the outcomes of their education. This latter survey reflects the EC-2000 criteria, while capturing information about the processes an individual experienced as an undergraduate student [45-46]. 

Proposed research areas: We will extend our current assessment research using questionnaires to track students’ attitudes and competencies at a number of points in the educational process. These will extend from when they enter the engineering education system, at graduation, and as professionals, and will address issues germane to EC-2000 and the educational processes they are/have experienced. In particular, we will develop sophomore and junior year assessment instruments as complements to our freshman instrument. We will develop and test questionnaires that can be used across engineering disciplines and institutions. Because self-assessed ratings are surrogates to true measures of students’ competencies, we will also explore the viability and cost effectiveness of using closed-form questionnaires over other forms of assessment. 

2.d. Open-ended Surveys and Structured Student Interviews are used to elicit in-depth information about a particular subject, especially when the subject of concern is complex, and there are a number of avenues to explore [47]. Because of their one-on-one nature, open-ended surveys and interviews allow the subject to present his/her attitudes in a more private and less restricted setting than other methodologies, such as focus groups and closed-form questionnaires. This permits in-depth information to be obtained about potentially sensitive subjects; e.g., reasons for leaving engineering, one’s position about ethical responsibility in school and as a professional engineer, one’s definition and desire towards life long learning.

Proposed research area: In terms of EC-2000, little research has been conducted in the area of assessment of professional and ethical issues in undergraduate engineering education and how students acquire an understanding of the role engineering has in a societal and international context. Two principals (Shuman and Wolfe) have conducted in-depth research in the area of ethical responsibility of professional engineers, specific to the Space Shuttle Program [48]. We will employ open-ended surveys and student interviews, along with focus groups (described below) to investigate these issues and to develop protocols and instruments aimed at assessing these learning outcomes.

2.e. Focus Groups are used to identify the attitudes and perceptions that a group of individuals has relative to a particular subject or concept. They are commonly used when one is looking for exploratory or exhaustive information about a particular issue [49]. In an educational setting, they may be employed to probe student perceptions or determine if there are communication gaps between different subject groups, such as faculty and students. Focus groups have several advantages as an assessment method. First, by providing an environment in which the subjects discuss a particular issue, they facilitate the capture of ‘real-life’ data. Second, the dynamics of a group allow for particular issues to be explored in-depth, something that is not always possible with structured questions [50]. For example, a single individual may not fully understanding how engineering fits in a global and societal context, but collectively students may provide full knowledge and understanding of this issue given their diverse backgrounds and perspectives. If conducted properly, results of a focus group have high face and construct validity [51, 52].

Proposed research area: Focus groups will be used in concert with open-ended surveys and interviews to investigate and assess the outcomes associated with professional and ethical responsibility, societal and global contexts of engineering, life long learning, teamwork, as well as attitudes about engineering.

2.f. Competency Measurement - competency-based surveying provides faculty and students with a means for quantifying performance with respect to specified knowledge, skills, and abilities. This type of assessment method is quite flexible and can be constructed for faculty-student rating and/or student-student (peer) rating. Competency surveys can also serve as a structure for self-assessment. One approach that has been successfully used in the classroom is a behavior-oriented computerized survey called the Team Developer™. Team Developer™ is designed to provide each student with developmental feedback regarding his or her effectiveness on several specific cognitive and behavioral skills. Student team members rate both themselves and their teammates on items designed to identify skills that have been found to be important for practicing engineers. Each student receives a developmental feedback report that presents his or her "self " and team ratings on each survey item and highlights overall strengths and areas for development. Gaps between self-perceptions and the perceptions of others are clearly shown. Specific suggestions for development, keyed to the behavioral areas, are provided to assist team members in developing action plans based on their personal feedback.

Proposed research area: One of the biggest challenges many faculty face when attempting to implement competency-based surveys is finding the time to collect, tabulate and then disseminate information. A computerized format like Team Developer™ helps to eliminate many of these obstacles. Using a computerized survey means that data can be collected and analyzed quickly and feedback can be provided quickly. This also means that more time can be spent reviewing information and ensuring that the feedback process is a meaningful one for students and instructors. We intend to further explore the efficacy and validity of computerized survey processes. We will address several issues in this important topic area including resource availability, ease of administration, longitudinal tracking of student performance, and confidentiality.

2.g. Student Journals are an established learning and assessment tool in college literature and writing courses and are becoming more widely used in other settings. The rationale for using journals is simple and pedagogically grounded — students learn better when given an opportunity to articulate connections between new ideas and knowledge they already possess. Journals are also an effective, non-threatening mechanism for creating a rich dialogue between student and instructor. Although primarily used as a formative assessment tool, journals can also be used for more formalized summative assessment of what students have learned and the knowledge they have constructed. For assessment purposes, journal writing assignments might include: 1) asking students to pose clarification questions on confusing or unclear topics and then attempt to answer the questions themselves; 2) asking students to apply the classroom analysis of a topic to everyday phenomena they observe; or 3) asking students to describe their thought processes for solving a problem, particularly open-ended, related to the course material [53-55].

Proposed research areas: Journals provide a rich and varied source of information about student learning processes and the ability to construct knowledge. However, we need to develop and pilot rubrics that will guide the use of journal entries for formative and summative assessment purposes. For example, how can student journals from one course or a series of courses be used to provide information about student or EC-2000 outcomes? How can a large number of journal entries be rapidly screened to collect valid assessment data?

2.h. Concept Maps are a graphical assessment tool used to evaluate cognitive structure in students by allowing them to visually describe relationships among concepts and topics in a course, series of courses, or entire curriculum. As such, concept maps may be used to probe for understanding and misconceptions as students internally structure their knowledge in a field of learning. Although originally used primarily as a classroom assessment technique, concept maps currently are being formalized as an assessment and evaluation tool. For example, maps can be used to provide information about students’ ability to integrate knowledge from different parts of their curriculum and therefore can be used to assess such EC-2000 criteria as the students apply knowledge of math, science and engineering, and the ability to formulate and solve engineering problems [56-58].

Proposed research areas: Since concept maps involve visual representation of connections among concepts, they are inherently difficult to assess and score reliably. We propose to formalize the use of concept maps by developing a set of guidelines and rubrics for administering and scoring concept maps for various purposes (classroom assessment, individual student feedback, curriculum assessment and evaluation).

2.i. Verbal Protocol Analysis (VPA) is a research method that requires subjects to "think aloud" as they perform a task [59]. It is a particularly valuable for collecting data about the processes that students use as they solve problems. Once the verbal protocols are collected via audio and video tape, they are transcribed; segmented into codable units of subject statements; coded according to a pre-defined coding scheme; and analyzed to answer specific research questions. Of the eleven EC-2000 outcomes, at least five are purely process skills while most of the others contain some process components. Verbal protocol analysis is a powerful tool that can be used to understand some of the process outcomes such as the "ability to design a system, component, or process". Atman has used this research method to assess engineering student learning of the design process and has documented differences in student design processes in three cases: 1) after completing an undergraduate engineering degree, 2) after the first semester freshman year, and 3) after the short term intervention of reading a text book [60-67].

Proposed research areas: Verbal protocol analysis is a very time consuming analysis technique that is used most frequently as a research tool. However, this type of process data is invaluable in the level of detail it can provide to guide curriculum changes. We intend to investigate several methods to obtain information relevant for design process assessment that can be accomplished with less time. Specifically, we will identify key design process variables that research has shown to correlate with design quality. We will then develop specific design problems that will allow us to observe these behaviors easily. We also plan to examine a number of analysis methodologies that may give us this assessment information with less effort than a full verbal protocol study.

2.j. Intellectual Development - Students are expected to develop intellectually in addition to acquiring knowledge and skills in a specific engineering discipline. Several ABET outcome criteria imply that students are able to think critically about their work and have developed higher order thinking skills (analysis, synthesis, evaluation as defined by Bloom) [68-70]. These include the ability to conduct experiments and analyze and interpret results, to design, formulate, and solve engineering problems, and to understand engineering in a global and societal context.

Proposed research areas: Traditional measures of intellectual development include pencil-and-paper questionnaires that have failed to demonstrate sufficient reliability and validity or expensive, time-consuming student interviews by certified experts. We are developing computer software based on neural net and expert system technology to emulate the interview process; versions of this will be available for field testing in this project. Results can be used to improve the software package for use as a program evaluation tool.

2.k. Authentic assessment and performance-based assessment methods attempt to measure how well engineering students can apply acquired classroom knowledge and skills to more realistic problems approximating "real-world" engineering practice. Examples of authentic assessments include design projects, open-ended problems and lab exercises, simulations, and portfolios of student work. The key to authentic assessment is to create a context in which the student can individually or collaboratively demonstrate an ability to apply a well-developed problem-solving strategy. This might involve problem definition, gathering relevant information, generating solution alternatives, choosing the optimum solution given implicit and explicit constraints, assessing and improving the proposed solution, and effectively reporting the results of his/her work. Authentic tasks can be used for both formative and summative assessment, and ideally can be included as part of the student’s assigned coursework [71-74].

Proposed research areas: As with other "non-test" assessment methods, authentic assessment has the advantage of providing rich data on student performance, but at the expense of increased time requirements to collect and analyze large amounts of descriptive data and observations. Development of well-designed scoring rubrics and methods for ensuring inter-rater reliability are required to make authentic assessment easier to use by engineering faculty and a viable assessment tool for ABET program evaluation. Guidelines also need to be developed which help faculty choose tasks that are good candidates for collecting authentic assessment data in engineering courses.

2.l. Modeling the Engineering Education System, specifically empirical modeling, is commonly used to draw correlated inferences and define relationships among different factors. Knowledge and insights about the relationships among the inputs, process elements, and outcomes of a system are useful, not only as an evaluation tool for better understanding the system, but also in targeting feedback of those factors most correlated with outcome. Empirically derived models may also be used to predict system outputs given information about the inputs and processes. To date, many of the empirical modeling applications in engineering education have focused on retention or performance [75-81]. Factors used in developing these models have included, but are not limited to: gender, race, geographical backgrounds, personality differences, attitudes about engineering, self-assessed confidence, as well as intellectual factors. At Pitt, we have developed regression models to predict attrition and performance in our freshman engineering program using quantified measures of student attitudes [82]. Implementation of these models has allowed freshman advisors to better inform students of opportunities that engineering offers, devise programs of study that take advantage of students’ varied interests, and set realistic retention goals.

Proposed research areas: Modeling aspects of the engineering education system has the advantage of helping us quantify, define, and evaluate relationships among aspects of the student, their educational experiences (in particular, innovative interventions), and the outcomes of their education. We have conducted some promising preliminary research in this area using attitude questionnaires from engineering alumni [83]. Using a combination of different empirical modeling techniques (e.g. regression analysis, discriminant analysis, neural networks) coupled with the combined databases from this research, we will expand this pilot work. Further, we will use empirical modeling to determine the consistency among the various assessment methodologies with respect to particular outcomes. 

Phase 3

In this final phase we will complete the case studies for each outcome, examining how various assessment methods can be used. To do this, we may focus on evaluating certain educational innovations that have been or are being implemented at our participating universities. We will complete the development of our web-based tools and instruments. Finally, we will prepare all material for dissemination.

3.1 Evaluate Each Outcome and Prepare Case Studies

We will have selected a series of outcomes for measurement using a variety of methodologies across multiple institutions. Assessments will be targeted at curricula innovations where possible. Since many innovations will be applicable to only a subset of the institution’s engineering students, this will enable us to have a comparison group at these universities in addition to comparisons with like populations of students at one or more of our partner institutions. The end product for each outcome will be a comprehensive case study. The case study will present the scenarios (innovations and/or populations assessed); detail the assessment methodologies that were used to evaluate the outcome, describe how each was adapted or refined; and give any tools or instruments that were developed to measure the particular outcome. The results of the different assessment methods will be compared as part of our "triangulation" evaluation process. We will assess the information obtained with each method, determine consistency among the methods and areas of inconsistency. These results will be further assessed using our modeling methodology. Finally we will report on the important factors of reliability, validity, sensitivity, cost to implement or modify, cost to maintain, and needed extensions. We anticipate assessing approximately fifteen output measures. We will prepare a case study for each outcome, and cross-reference the different assessment methods. This will yield a set of approximately fifteen case studies with each assessment method used for several outcomes.

3.2 Develop Web-based instruments and tools

Once instruments have been developed, piloted tested and reviewed, where possible, we will place them on the worldwide web to facilitate use by the developing institution and adoption at other institutions. For example, one of our principals, McGourty, has created the Student Developer™ and Team Developer™, computerized instruments that gives students, working in teams, peer feedback from teammates on nine learning outcomes [84]. We will also utilize the web to disseminate those instruments (in .pdf format) that cannot be computerized. We will also include complete descriptions and instructions for use. We have been successful in disseminating our Pittsburgh Freshman Engineering Attitude Survey© to over 30 universities, at least ten of whom are currently using it. We continue to analyze the data from these institutions, and hence, build up a more extensive national database. We anticipate building up similar national databases with the instruments that will be developed as part of this study.

We also intend to utilize the web to facilitate communication among our partner universities and to exchange draft instruments, tools, and monthly progress reports. In this manner, we will maintain an electronic record of our progress that will be accessible to all project staff.