Developing an Effective Assessment Plan

An effective assessment plan should clearly specify the following items:

An appropriate, and realistic targeted achievement level for each outcome
The direct and indirect assessment tools that will be used to assess each outcome;
A timeline that specifies when and how often each outcome will be assessed. This could depend on national and/or international accreditation requirements.
The entities involved in the collection of the assessment evidence (e.g., faculty, lab instructors, supervisors, etc.)
How the evidence will be collected (e.g., unified rubrics, surveys)

The following points should also be considered while developing the assessment plan:

Effective assessment plan does not have to be complicated. The aim is to develop an assessment plan that requires little extra time and effort.
Faculty should be involved as early as possible in the plan development process, and their feedback regarding the assessment plan should be considered.
Do not overwhelm faculty and students with new tasks. The assessment works best when it is part of the regular course work and learning activities rather than additional tasks for students.
Choose the assessment methods that provide evidence that are easy to understand and interpret.
Identify which learning experiences (e.g., projects, exams, assignments, and presentations) are best suited for each assessment method.
Consider whether the selected assessment methods allow you to easily detect the improvements in the outcome achievement level.
Keep the students informed about the aim of the assessment practice. Studies show that students usually perform much better when they know what the instructor is expecting from them.
Protect the confidentiality of students whose learning will be assessed.

The first step in the assessment plan is to identify the most appropriate assessment methods that will be used to assess each outcome. An appropriate assessment method should be able to measure the competency addressed by the outcome effectively and accurately. It is essential that the selected assessment tool and the outcome belong to the same bloom’s taxonomy level. For instance a describe or explain question (Blooms level 1) should not be used to assess a design outcome (Blooms level 6).

The assessment tool should be also aligned with the content of the curriculum and take advantage of the existing teaching practices. The better the integration of the assessments into existing student work (e.g. exiting exams, capstone projects, assignments, etc.), the greater the probability that the assessment plans will succeed. Using multiple direct and indirect assessment methods is necessary to assure reliability and validity of the assessment findings.

Indirect assessment methods pertain to the constituencies (students, faculty, employers, alumni, advisory boards, etc.) perceptions of the achievement level of the learning outcomes, and their perspectives on program structure and curricular content. Examples include different types of satisfaction and exit surveys, and results of focus groups and interviews. Indirect methods may also include retention and transfer studies, graduation and transfer rates, and job placement data. Although indirect methods provide useful information regarding the quality of the learning experience, they do not simply answer fundamental questions about the degree to which students have met specific learning outcomes.

Direct assessment methods, on the other hand, include immediate evaluation of a student’s performance, such as a test, paper, capstone project, laboratory procedure, signature assignments, etc. They can be classroom-based activities, department-level exams or projects, or standardized tests relevant to the field. Student performance must be measured using explicit criteria (e.g., rubrics) connected to the learning outcomes. However, it is curtail not to use the entire grade of an exam as a direct indicator of learning outcomes achievement as it does not pinpoint what exactly which outcomes the students have or have not achieved. For example, if 70% of a class may receive a grade of C in a test. Although the overall grade show that students have achieved the target competence, students might have scored 90% in questions related to outcome one and only 60% in questions related to outcome 2, which cannot be detected if the overall grade is used.

Faculty should consider rubrics more often when designing tools for assessing students work especially when multiple sections of one course are taught simultaneously. A rubric is a set of scoring guidelines (criteria) and standards for evaluating students work and for giving feedback. Some of the clear benefits of using a scoring rubrics are:

It identifies the key elements (criteria) of the work that will be evaluated.
It indicates the differences between good and poor work (standards) on each criterion.
It is a tool to ensure that the evaluation (or scores) of work or performance are valid and reliable.
It provides both students and evaluators with a clear prospect about what is expected for excellence.

Continuous improvement of the student learning experience is the ultimate goal of the assessment process. However, it is very sensitive to the appropriate and realistic setting of target learning outcome achievement level. Recommendations for improvement and corrective actions are usually used to address learning outcomes where the actual achievement level is blow the target. Therefore, setting the target achievement levels inappropriately low increases the risk that the students will constantly exceed them. Hence, halt the continuous improvement cycle as no recommendations for improvements or corrective actions will be required. The improvement process will remain halt until the target achievement level is revised or the assessment methods are checked and adjusted if needed.

If no previous assessment data is available to guide the setting of the target achievement level, it is recommended to set the initial achievement level according to the following guidelines:

Undergraduate CLOs, direct tools: 70% of the students score ≥ 70%
Undergraduate CLOs, indirect tools: 70% of the students score ≥ 3 on a four point scale, or ≥ 4 on a five point scale.
Graduate CLOs, direct tools: 70% of the students score ≥ 80%
Undergraduate CLOs, indirect tools: 80% of the students score ≥ 3 on a four point scale.
Target attainment level for specific direct tools such as (exit exam) could be lower than that calculated based on official exams

It is expected that target achievement levels for several outcomes will be revised by the assessment committees to ensure the continuation of the improvement cycle. Consequently, different outcomes will eventually have different target achievement levels.