• The Development of a Statement of Purpose

• The Development of Specific Objectives

• Selection of Appropriate Assessment Mechanisms to Determine the Extent to Which Objectives Are Being Accomplished

• The Annual Assessment of the Extent to Which Student Learning Objectives Have Been Met

The formulation of a sound assessment plan for any academic major or program is a fairly straightforward process that involves three specific activities:

(STEP 1) The development of a statement of purpose.  Academic majors and programs should begin with the College's Statement of Purpose, Institutional Standing Goals, and Curriculum Goals--these are statements about what the College values and what it intends to offer to students.  These statements should guide the objectives and activities of academic majors and programs.

While some academic majors and programs work directly from these College statements, most have prepared an individual and specific purpose statement.  These specific statements are consistent with the College statements but may add to them and tailor them to the specific major or program.  Just as the College statements influence all programs at the College, the purpose statement of each academic major and program should influence all related activities.  Examples of purpose statements of specific majors and programs are available elsewhere at this site.

(STEP 2) The development of specific objectives.  Each academic major and program should identify specific learning objectives for students in that field.  The following steps facilitate this process.

1. Start with individual reflection and group discussion.  Use Peter Ewell's menu of possible learning objectives (at the end of this section in 2-d) as a means for thinking through the various types of objectives that might be selected.  Individuals should reflect on:
   1. The knowledge, skills, attitudes, and behaviors expected of students in the program

   2. The achievements (education and career accomplishments, lifestyles, citizenship activities, aesthetic and intellectual involvements) reasonably to be expected of students in and graduates of the program
   
   
2. Draft a set of well-written objectives.  They should have the following characteristics:
   1. Include an action verb and a statement of ability.  Use meaningful verbs: better objectives use action verbs like paraphrase, compute, describe, and construct; poorer objectives use verbs that are too general like understand and appreciate.

   2. Involve objectives that can be operationalized and are empirically verifiable/directly observable

   3. Are attainable or feasible given the resources of the department

   4. Clarify or establish a link between what students accomplish in the program and what they do after they graduate

   5. Permit multiple paths of demonstrating mastery of program objectives

   6. Additional considerations: The objectives may be
      • short-term or long-term
      • subject-aware or subject-unaware
      • direct or indirect
      • intended or unintended
   
   
3. Examples of objectives that might be used by sociology departments for what they would want their graduates to be able to do:
   1. Explain what makes an explanation sociological

   2. Analyze current social issues from the three dominant theoretical perspectives

   3. Interpret quantitative data and abstract models as presented in tables, graphs, charts, and diagrams

   4. Identify and explain the primary strengths and weaknesses of survey research, experimental research, observational research, and content analysis

   5. Formulate a sound design for a social research project and carry out the project

   6. Present a summary of a research project at an on-campus or off-campus forum

   7. Consider self well-prepared for graduate school
   
   
4. "A Classification of Outcomes Dimensions" (Source: Peter Ewell, The Self-Regarding Institution: Information for Excellence, National Center for Higher Education Management Systems, Boulder, CO, 1984)
   

   In constructing learning objectives for students, it is helpful to think of the variety of dimensions in which growth or change in students might be desired.  The following classification system is helpful in thinking through these various dimensions.  It should be considered as a menu; departments may choose to formulate objectives relative to as many or few of these dimensions as they think important.

   1. Knowledge Outcomes--the purely cognitive aspect of educational outcomes
      1. General knowledge (breadth of knowledge)--familiarity with and analysis and comprehension of facts and principles inherent in broad areas of study.  It may include:
         1. recall and comprehension of facts in a broad area of study
         2. recall and comprehension of theories and terminology in a broad area of study
         3. recall and comprehension of investigative principles and methods in a broad area of study
         4. recall and comprehension of the history and development of a broad area of study
         5. ability to relate/integrate approaches and concepts drawn from more than one broad area of study
         
         
      2. Knowledge of specific fields (depth of knowledge)--requires mastery of the accepted body of facts, theories, language, and techniques of a particular field of study.  It may include:
         1. recall and comprehension of facts in a particular discipline or specialized field
         2. recall and comprehension of theories and terminology in a particular discipline or specialized field
         3. recall and comprehension of investigative principles and methods in a particular discipline or specialized field
         4. recall and comprehension of the history and development of a particular discipline or specialized field
         5. ability to effectively manipulate appropriate concepts, theories, and investigative methods to create new knowledge in a particular discipline or specialized field
      
      
   2. Skills Outcomes--the relative abilities of students to perform explicit tasks
      1. General competence (social functioning) skills--encompass the abilities to use knowledge, organize information, define problems, and discover and implement solutions to problems.  It may include:
         1. verbal skills, including reading, reading comprehension, writing, and oral communication
         2. quantitative skills, including mathematics, statistics, and computing
         3. leadership/organizational/human-relations skills
         4. analytical skills, including skills of problem definition, problem solving, and critical thinking
         5. invention/innovation/creative-thinking skills
         6. aesthetic-appreciation/creative-expression skills
         7. physical/motor skills
         
         
      2. Professional/occupational skills--consist of the particular, specialized skills needed for effective performance in an identified profession or occupation
      
      
   3. Attitude/Value Outcomes--the affective impacts of higher education
      1. Personal goals and aspirations--are defined in terms of levels, patterns, and directions of personal interests, desires, drives, and ambitions.  It may include:
         1. general goals and aspirations (lifestyle, social mobility, family goals, personal goals)
         2. occupational and career goals
         3. educational goals
         4. motivation and drive levels for each of the above
         
         
      2. General attitudes, values, and satisfactions--in terms of their content, extent, and internal consistency.  It may include:
         1. beliefs (including religious beliefs), belief systems, value commitments, and philosophies of life
         2. mores, customs, and standards of conduct
         3. patterns of feelings and emotions, including particular satisfactions and dissatisfactions with individuals, groups, institutions, and social situations
         
         
      3. Attitudes toward self--development of identity.  It may include:
         1. perception of self, general self-concept, self-discovery
         2. self-reliance, self-confidence, including adventurousness and initiative, autonomy, and independence
         3. satisfaction with self, psychological well-being
         4. personality/personal coping characteristics, including flexibility and adaptability, dogmatism/authoritarianism, tolerance and persistence, and so forth
         
         
      4. Attitudes toward others--related to and part of individual identity.  It may include:
         1. specific perceptions of other individuals/groups in a society
         2. tolerance for cultural and intellectual diversity, including a willingness to accept different points of view
         3. general human understanding, including empathy, sensitivity, and cooperation
      
      
   4. Relationships with Society and with Particular Constituencies--an often longer-term measure of the effects of higher education on students
      1. Relationships with educational institutions
         1. individual educational development goals
         2. patterns of enrollment, placement, and participation
         3. patterns of retention, attrition, and program completion
         4. patterns of program/institutional change or transfer
         5. levels of achievement in subsequent educational experiences
         6. quality of student effort
         7. assessed relevance and contribution of past education to subsequent educational experiences
         
         
      2. Relationships with employers
         1. individual employment/career choice goals, including change, stability, and intensity of goals
         2. first job obtained after education, including the relevance of this job to the education received
         3. long-term employment history
         4. income/earnings history
         5. promotion and job performance
         6. job satisfaction
         
         
      3. Relationships with professions/professional associations
         1. individual professional development goals, including change, stability, and intensity of goals
         2. patterns of professional certification/recognition/award
         3. patterns of subsequent professional development activities
         4. assessed contributions of past education to professional success
         5. professional satisfaction
         
         
      4. Relationships with family/community/society
         1. family roles, relationships, and child-rearing practices
         2. patterns of social affiliation, group membership, and participation
         3. patterns of voluntary contribution, including contributions of time, money, or other support
         4. patterns of citizenship activities/political participation

1. Start with individual reflection and group discussion.  Use the menu in Appendix B as a means for thinking through the various types of assessment mechanisms that are available or might be created.

2. Develop a well-rounded assessment program.  In selecting mechanisms to be included in the program, give consideration to the following:
   1. To the maximum extent possible, mechanisms selected should measure what they intend to measure.  Our assessment mechanisms should enable us to draw correct conclusions about the extent to which our objectives are being met.

   2. Both quantitative (i.e., numerical data such as scores on comprehensive exams and number of students doing independent studies) and qualitative (i.e., more subjective information such as perception of the genuine interest that majors have in the discipline or perception of students' ability to ask perceptive questions in class) should be included.  (Often, they nicely complement each other.)

   3. Assessment information should be collected from a variety of constituencies.  For example, departments might focus on current majors and minors, students in the introductory class, non-majors taking electives in the department, recent or older alums, faculty in other departments, student services staff, etc.

   4. While there is no magic number of mechanisms that should be included, assessment programs, a sufficient number should be used to ably evaluate the variety of objectives that have been formulated.  The number should not be so great as to be burdensome, but having only a couple of mechanisms is clearly not enough.

   5. Departments should avail themselves of the many excellent assessment mechanisms that have been used at the College for many years.  The Senior Exit Survey and alumni surveys are examples.

   6. Departments should construct an assessment program that seems appropriate and beneficial to them.
   
   
3. Among the assessment mechanisms currently in use at Roanoke are the following:
   1. From Current Students

      Course/Instructor Evaluations.  While typical course/instructor evaluations lack the specificity to be strong assessment measures, helpful information can be obtained through analysis of specific items and student comments.  All courses at Roanoke are evaluated anonymously by students at the end of the semester, and tallies of responses and comments are provided to every faculty member.
      

      Course-Embedded Assessment (evaluation of specific aspects of student performance in core and designated courses).  While traditional course grading is not an adequate assessment tool, the development and use of techniques to assess specific skills and abilities within courses is a common and acceptable technique.
      

      Pre- and Post-General Studies Courses Writing Samples.  All students submit a written essay in response to a reflective question early in their freshman year (prior to their General Studies courses) and again in the junior year (in the Values and Responsible Life course--the final GST course prior to the Senior Symposium).
      

      Professional Standardized Testing.  Some departments require majors to take professional standardized tests (e.g., students getting certified in education take the Praxis examinations) or certification examinations (e.g., in athletic training).
      

      Student Portfolios.  Some departments require students to maintain a cumulative portfolio of their class and co-curricular work.  Portfolios are especially helpful in evaluating intellectual growth and in ascertaining students' abilities near the end of their academic work.

   2. From Graduating Seniors

      Senior Exit Survey.  Each department designs its own Senior Exit Survey that is administered by the Career Services Office to every graduating senior.  Items typically are closely aligned with department objectives and can be altered from year to year.

   3. From Alumni

      General Alumni Surveys.  Each department designs its own alumni surveys that are administered by the Office of Institutional Research to all students one year and five years after graduation.  Until recently, these surveys contained questions related only to broad institutional goals.  Now, the surveys also contain items submitted by each major program.  Though these items tend to be more general than those found on the Senior Exit Survey, they enable a longer range assessment of the perception of students regarding the extent to which the academic goals of the Department were met.
      

      Targeted Alumni Surveys.  Each department may add surveys that are targeted to specific groups of alumni (e.g., students who have entered graduate or professional studies programs).  These surveys are useful in assessing the extent to which specific academic objectives are being met (e.g., whether or not students think they were well prepared for graduate education).

   4. From External Reviewers

      Program Evaluation.  Every five years, academic departments and programs at Roanoke undergo a program review that is conducted by faculty members in other disciplines and by external reviewers from the same discipline.  This review includes a comprehensive departmental self-study that receives intensive review by the external reviewers and is the basis for their analysis while on campus.

   5. From Institutional Data

      Departmental Annual Report.  Department chairs submit an Annual Report to the Vice President-Dean of the College.  The required components of the Annual Report are standard across all departments and include an assessment of the progress that has been made in reaching the Department's annual goals; an overview of the teaching effectiveness, professional growth, and professional service of Department members; special learning opportunities afforded to students; student evaluations and grade distributions in Department courses; and an assessment by the Chair of the major accomplishments of the previous year and areas on which the Department will focus in the coming year.
      

      Monitoring of Student Background and Academic Performance.  Throughout the year and in the process of writing the Annual Report, departments monitor a variety of student performance indicators, including the quality of students entering the major; department grades; performance in campus (e.g., Summer Scholar selections and Student Research Symposium participation) and off-campus (e.g., professional conference) competitions; participation in community internships, independent studies, and department honors projects; and success in admission to and performance in graduate and professional schools.