BU Assessment FAQs

Here are some answers to frequently-asked questions about assessment. If you don’t see your question here, please submit it to kconnors@bu.edu and we will add it to the list, along with a response, for the benefit of everyone.

What is program assessment?
Why do we need to do assessment?
What are the basic steps in the assessment process?
What are program learning outcomes?
Where can I see some examples from my discipline?
How can we check if the outcomes we have defined are acceptable?
How should we decide where to begin with assessment?
What are some ways to measure student learning?
What do I need to do to protect student privacy?
How should we document our assessment process?
Why can’t we use grades for assessment?

 

What is program assessment?

Assessment is “the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development” (Palomba and Banta 1999). While assessment takes place regularly at the course level as faculty evaluate and grade student work, program assessment takes this evaluation to the next level to ask how the various parts of the curriculum are working together to promote student learning over a course of study.

Why do we need to do assessment?

The purpose of assessment at the program level is to gather evidence of student learning at regular intervals to provide information to the faculty about student learning for ongoing improvement. It is a regular process that allows faculty to ask key questions about learning in their programs and gain information they can use to support decisions about curricular change. Assessment can help identify successes and challenges and suggest possible steps to improve the program. As an ongoing systematic evaluation by faculty of what students learn in BU’s programs, the assessment process itself also provides evidence to support the university’s re-accreditation by the New England Association of Schools and Colleges.

What are the basic steps in the assessment process?

  1. Define program goals/mission
  2. Establish learning outcomes
  3. Determine opportunities for students to demonstrate learning
  4. Establish a research question or goal and collect pertinent evidence
  5. Evaluate and interpret student evidence
  6. Based on results, create and implement a plan to improve the program and student learning
  7. Communicate results in an annual assessment report (due November 15)

What are program learning outcomes?

Learning outcomes identify what students will know and be able to do by the end of a course or program. They should be framed as statements that describe significant and essential learning that students can demonstrate and should represent your highest aspirations for your students. Even learning outcomes that seem ineffable or difficult to quantify (frequently seen in arts and humanities disciplines) can usually be assessed by some measure and should be included in program learning outcomes.

Where can I see some examples from my discipline?

For many disciplines, professional organizations have put forward examples of learning outcomes that departments can use as a reference.

The National Institute for Learning Outcomes Assessment website provides links to a range of professional organizations and discipline-specific learning outcomes from the MLA, AHA, ACTFL, MAA, ACS, ABET, and others.

How can we check if the outcomes we have defined are acceptable?

You can start by asking some of the following questions:

  • Are the outcomes aligned with the primary mission of the program? Taken as a group, do they reflect the most important results of the program?
  • Do the outcomes clearly describe the expected abilities, knowledge, values, and attitudes of program graduates?
  • Are the outcomes simply stated, using action verbs?
  • Does each statement describe just one outcome?
  • Are the outcomes measurable, whether by direct or indirect (e.g., survey) measures?
  • Are the outcomes distinctive and specific to the program?
  • Can the outcomes be used to identify areas to improve?
  • Are they written in language that describes student rather than teacher behaviors?

The language of learning outcomes should always emphasize how students will demonstrate what they know and can do. Try to avoid verbs that refer to behaviors that cannot be outwardly observed or measured (appreciate, become familiar with, know, understand) and instead use actions verbs that refer to what students will demonstrate (demonstrate, apply, explain, interpret, analyze, compare, contrast, evaluate).

For guidance, you might want to refer to Bloom’s taxonomy of educational objectives and the key words associated with each level, which you can find on the Center for Teaching and Learning’s (CTL) website.

How should we decide where to begin with assessment?

Assessment should begin with real questions that faculty would like to answer about their programs. If a program has recently participated in program review, there may be issues or questions that arose during that process that could provide a meaningful starting point for assessment.

In some cases, faculty share an anecdotal sense that some aspects of the program are either particularly successful or in need of improvement and would like to have concrete evidence to guide future decisions about the direction of the program.  If changes have been made to some aspect of the curriculum, the assessment process can be used to gather evidence about the impact of the change on student learning.

Concrete questions about the quality of the program will help suggest where to begin with assessment and help faculty set priorities and devise a timeline.

What are some ways to measure student learning?

There are many possible tools and techniques programs can use to measure student learning, including qualitative, quantitative, direct, and indirect measures.  It is best to use a combination of at least one direct and one indirect method.  It is ideal to look at graded student work from the end of a student’s course of study as a direct measure of student learning.

Direct measures: Senior theses, final papers, or other graded projects, comprehensive exams, certification or licensure exams, capstone courses, portfolio evaluations, writing proficiency exams, performance assessments for graduating seniors, locally developed pre- and post- tests, GRE subject exams.

Indirect measures: Student surveys, focus group discussions, job placement data, exit interviews, alumni surveys, tracking of alumni awards and achievements, employer surveys, graduate school acceptance rates.

What do I need to do to protect student privacy?

You do not need permission from the institutional review board (IRB) for normal assessment procedures aimed at program improvement.  Design your review of student work so that student information remains anonymous, and when you are reporting the results of assessment, be sure that individual students cannot be identified and aggregate results. If, however, if you are conducting assessment for a grant-funded project or for publication, you should check with the IRB about whether you need to obtain student consent beforehand.

How should we document our assessment process?

It is important to document what your program did to assess student learning and what actions it plans to take based on assessment results.  It is acceptable to summarize assessment activities and results (e.g. minutes from a faculty meeting), but be sure to record the following information: a description of the group that was assessed, including the sample size of the group, a description of the information or student work collected or reviewed, a description of how the review process took place, a summary of results, and a description of who interpreted the assessment results. Finally, include a summary of how the reviewers interpreted the results and what actions the program intends to take based on this information. Include all of this information in the program’s annual assessment report, which is due each November 15.

Why can’t we use grades for assessment?

Although you may well be using graded student work as the cornerstone of your program’s assessment process, it is important that the process be considered separately from the assigning of grades.  While grades are a form of assessment, they are of limited use for program assessment since they don’t provide detailed information about what to work on.