FAQs

• What are the Program Learning Outcome (PLOs)?

Program learning outcomes are specific statements that describe the knowledge, skills, and competencies the students should have or be able to demonstrate upon the successful completion of the degree requirements.

• What is a program goal and how does it differ from a program learning outcome?

Program goals are broad, general statements that identify what you want to accomplish at the program level; they may state the program purpose. Goals describe a state of being and are not measurable, while outcomes identify results in behavioral terms and are measurable.

• What are course learning outcomes?

The course learning outcomes describe clearly what learners will know and be able to do at the end of the course. They are performance–based and results oriented. They describe learning that is significant and related to what learners will be expected to do in the "real world". Each course learning outcome should align with one or more of the program learning outcomes.

• What is a SMART learning outcome?

It is a learning outcome that is specific, measurable, attainable, relevant (to the program), and time bound.

• How do program learning outcomes influence curriculum development and program review?

Faculty should have a clear understanding of how courses are expected to contribute to the achievement of the program learning outcomes. The course learning outcomes and learning experiences in the course need to reflect this alignment. Faculty should also ensure that the program learning outcomes are current and relevant for the workplace. Faculty should demonstrate that the program's curriculum and how it is delivered guides learners to achieve the program learning outcomes and provides students with several opportunities to achieve the program learning outcomes.

• What is learning outcomes assessment?

Assessment is the process of gathering and discussing information from multiple and diverse sources to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences. Assessment is a continuous cycle that uses a feedback loop for identifying desired results; collecting and analyzing relevant information (data); and then using the findings to make changes to the program. The process cycle culminates when assessment results are used to improve subsequent learning.

• Why do we need to assess student learning?

Faculty design educational programs around the specific knowledge, skills and abilities they expect graduates of the program to acquire. The assessment process seeks to determine how well students are learning. It is essential not only to guide the development of individual students but also to monitor and continuously improve the quality of the program. It also provides feedback to students, educators, parents, policy makers, and the public about the effectiveness of educational services.

• Would learning outcome assessment help me to improve as a teacher?

Two of the major characteristics that distinguish effective higher education teachers are (Bain, 2004):

• When they prepare to teach, they usually begin with questions about student learning objectives rather than about what the teacher will do.
• They have some systematic program to assess their own efforts and to make appropriate changes. They assess their students based on the primary learning objectives rather than on arbitrary standards.

• What are some other benefits of assessment?

When done well, assessment leads to numerous benefits apparent in the short- and long-term. Assessment can help faculty:

• evaluate effectiveness and alignment of curricula
• improve the program and its standing
• inform planning and decision making
• understand the impact of program changes
• highlight program successes
• provide evidence of need when requesting resources
• advise students with a set of learning outcomes
  
  
• Is "grading" the same as "assessment"?

No. Grades alone may not tell us much about student learning, because a letter or numeric grade does not express the content of what students have learned. Grading and assessment criteria may differ, and grades may not reflect all learning experiences. While grades certainly play a role in the assessment process, they are usually not sufficient for answering questions about whether specific learning goals have been achieved. In addition, grading standards may be vague, idiosyncratic, or inconsistent.

• What is an assessment plan?

A document that outlines and describes in advance what will be assessed, when it will be assessed, how it will be assessed, and how the results of the assessment will be used for improvement. The assessment plan helps the faculty to focus on their efforts and to establish priorities for assessment activities.

• What is formative assessment?

Formative assessment (assessment for learning), is a range of formal and informal assessment procedures employed by teachers during the learning process to modify teaching and learning activities to improve student attainment. The primary focus of formative assessment is to identify areas that may need improvement. Formative assessment helps students identify their strengths and weaknesses and target areas that need work. It also helps faculty recognize where students are struggling and address problems immediately. These assessments are typically not graded and act as a gauge to students’ learning progress and to determine teaching effectiveness. Types of formative assessment may include:

• draw a concept map in class to represent their understanding of a topic
• submit one or two sentences identifying the main point of a lecture
• turn in a research proposal for early feedback
• Observations during in-class activities on students’ non-verbal feedback during lecture

• What is summative assessment?

Summative assessment (assessment of learning) employs a variety of tools and methods for obtaining information about what has been learned. It takes place after the learning has been completed and provides critical information about students' overall learning as well as an indication of the quality of classroom instruction. The main purpose is to determine whether the outcomes have been achieved and to provide information on performance for an individual student or statistics about a course or program for internal or external accountability purposes. The results of the summative assessment are usually used by policy makers, program planners, supervisors, and accreditation agencies. Examples of summative assessments include:

• Embedded questions in midterm and final exams
• Term papers
• Capstone projects
• Students Portfolios
• Standard tests

PS: Information from summative assessments can be used formatively when students or faculty use it to guide their efforts and activities in subsequent courses.

• What is embedded assessment?

Embedded assessment means that assessment activities are integrated into the routine of teaching, learning, or research. The intent is to make assessment activities regular, ongoing, and as unobtrusive as possible. An embedded assessment approach also emphasizes using information you already have or collect as much as possible.

• Why do we need to define a target attainment level for each outcome?

A target attainment level is the percentage of students who achieved a predefined level of performance. For example, 70% of the students scored 75% or above in a direct assessment tool related to a specific outcome. Defining a reasonable and realistic attainment level is crucial for the whole assessment process as it drives the continuous improvement cycle. There is a risk that setting the attainment level too low will hinder the continuous improvement process as the outcome will be always attained and no improvement will be applied to the program. The program may set different target attainment levels for different outcomes. The program may also set different attainment levels for direct and indirect assessment tools.

• What is the difference between direct and indirect assessment?

Direct assessment involves examination of products produced by students such as term papers, presentations, capstone projects, portfolios, exit exams, embedded questions, standardized tests, etc. While indirect assessment is the gathering of information about student learning by looking at indicators of learning other than student work output. This assessment approach is intended to find out about the quality of the learning process by getting feedback from the student or other persons who may provide relevant information (e.g., faculty, internship supervisors, employers, alumni, advisory boards, etc.). Indirect methods can provide additional information about what students are learning and how this learning is valued by different constituencies. However, they do not provide reliable evidence that the learning outcomes have been attained. The use of surveys and focus groups may lead to improvements in a program but do not directly provide evidence of student learning.

• What is the difference between quantitative and qualitative assessment methods?

A quantitative assessment method collects data that can be analyzed using quantitative methods. Includes methods that rely on numerical scores or ratings. A quantitative measurement uses values from an instrument based on a standardized system that intentionally limits data collection to a selected or predetermined set of possible responses. Quantitative assessment methods work by the numbers, collecting, analyzing, interpreting, and charting results, trends, and norms. On the other hand, qualitative methods collect data that does not lend itself to quantitative methods but rather to gathering information that focuses on describing a phenomenon in a deep comprehensive manner. Includes methods that rely on descriptions rather than numbers such as interviews, open-ended questions, or focus groups. Using qualitative methods allows faculty to understand how individual students are doing, what their experiences are, and recognize important antecedents and outcomes of interest that might not surface when surveyed with pre-determined questions. Qualitative measurement is more concerned with detailed descriptions of situations or performance. Therefore, it can be thought of as subjective, when pooled across several participants it provides a conceptual understanding and evidence that certain phenomena are occurring with particular groups or individuals.

• What is the best way to find the time for assessment?

The best assessment is ongoing and embedded in regular work processes. As much as possible, collect your evidence from learning or program activities that already occur. For student learning outcomes, that means using scores from student coursework or extracurricular activities. Standardize collection processes so that they become routine. Look for data that produces information that is usable; if it’s not usable, don’t collect it. Taking the time up-front to plan assessment activities will minimize the time it requires to implement them.

• How much data do I need to collect and how often?

Collect enough information to determine whether you are meeting program objectives and outcomes. If you are not sure how you will use the information to measure student learning or program effectiveness, then consider whether it is worth your time to collect it. The frequency of data collection depends on how you plan to use the results. In general, data collection should be regular and ongoing. This does not mean to collect all data constantly, spread out data collection over time so that you collect certain pieces each semester in a regular cycle that covers multiple academic years.

• What approach should we take to analyze assessment results?

Analysis helps make meaning from the data by describing general trends and pointing out differences and similarities. It relates results to the goals and outcomes/objectives, explores the relationships among multiple measures, and draws inferences. With analysis, you can address questions such as: Do the results tell us what we already know or tell us something new? What do we think about what has happened so far? Are our targets meaningful? Academic programs can answer these types of questions: - What does the evidence say about student mastery of the subject matter or crucial skills?

What does the data say about students’ preparation for the next step? Where are our students consistently strong or weak? Administrative programs can ask: Are our activities and events achieving what we want them to? How can we be more strategic in our operations? What places do we see where we can be more efficient?

• Do assessment results have to be used for the purpose of improvement?

In any given year, it might not be necessary or appropriate to launch a program improvement initiative based on assessment results. Over time, however, assessment results will be used for curricular and program improvement. Using assessment data to make decisions about programs keeps it relevant, even innovative. Additionally, data-based decisions improve efficiency, strengthen mission, and ultimately, positively impact student learning or operations.

• Is learning outcomes assessment here to stay?

Yes, learning outcome assessment is here to stay. It is essential for WACS, and CAA accreditation. It is also a basic part of the new periodic program review process, and any program professional accreditation. One of the major objectives of the program and curriculum office is to improve the assessment culture within the UAEU community such that assessment is considered inseparable from curricular, or program improvement and it is the best way to demonstrate student learning and program quality.

• Does university administration use outcomes assessment results as an additional way to evaluate faculty performance?

No. Outcomes assessment focuses on measuring student learning across a program, not on evaluating individual faculty members. Assessment results are not tied to the promotion and tenure process.