The Wildland Resources Department at USU has four undergraduate majors: Conservation and Restoration Ecology, Forest Ecology and Management, Rangeland Ecology and Management, and Wildlife Ecology and Management. In attempting to assess undergraduate learning, nine years ago (~2007) we developed 26 undergraduate learning objectives that were primarily related to our commons courses, a set of 8 courses and 21 credit hours that are taken by undergraduate students across all 4 of our majors. In 2016 we consolidated the 26 learning objectives into 5 overarching objectives (learning objectives table) that allow for more reliable and program-specific assessment of student achievement of the objectives by the faculty, and by the students themselves.
OPM Evaluation and Learning Objectives
The curricula for our undergraduate programs were designed with an awareness of the federal government’s Office of Personnel Management (OPM) requirements. Descriptions of how our curricula and courses at USU can be used to meet OPM requirements are provided on our website under the “Undergrad Programs” link, and can be accessed by clicking on a specific major. When a student graduates with a degree in one of our majors, they meet the qualifications of the federal agencies for employment as wildlife biologists, foresters, range conservationists, or other natural resource professions.
Overall, we want our graduating students to become professionals that can excel at the work society expects them to do. Our updated learning objectives move us toward this overall goal, which was determined by our faculty based on their knowledge of the professions and agency hiring requirements, through input from professionals and former students, and based on requirements of accrediting organizations.
Going forward we will assess student progress on the learning objectives at the program (major) level, and for each student within each program. The learning objectives are very similar among majors because all of our undergraduate students enroll in a commons curriculum that is shared across majors. No single course or professor can assess a student’s achievement of all the learning objectives, but they can be mapped to a set of courses for each major (course table). For all courses, learning objectives are stated in corresponding course syllabi and, for our courses, are included in the IDEA Student Ratings assessment program.
But our educational assessment plan is not restricted to assessing the accomplishment of our stated learning objectives. Rather, it involves a multi-faceted approach that includes gathering and summarizing data on achievement of each learning objective for each student in each major, but also involves evaluating patterns in employment and continuing education among our graduates, learning about professional society expectations of our major programs and how we meet those expectations, sharing assessment outcomes data amongst ourselves and with peers, and using analyses of assessment outcomes data to make decisions about changes in our major programs and learning objectives.
Student Progress on Learning Objectives
Following a ‘self-study’ of the department in 2007 that was conducted in relation to NWCCU accreditation, the department decided to develop a single capstone course for all four majors that would encompass all of the 26 (historical) learning objectives. Faculty envisioned that this single course could provide a comprehensive assessment of how well students meet the learning objectives by the end of their respective degree programs. After several iterations, we found that student enrollments were too large and student needs were too disparate to accomplish what we wanted in that one course. We now have separate capstone courses for each of our four undergraduate majors. These courses require each student to become involved in the analysis of real-world environmental problems. How students fare in these capstone experiences should depend in part on learning in previous courses. Although faculty who teach these capstone courses will be able to assess student achievement for several of the learning objectives, it would be unrealistic to ask them to assess performance on all of them.
Instead, graduating senior attainment for each learning objective (learning objectives table) will be assessed by faculty teaching the courses that are most relevant to an objective (course table). The faculty will score each student’s progress for a given learning objective using a 1-4 scale (4 = Achieves Mastery, 3 = Achieves Proficiency, 2 = Approaching Proficiency, 1 = Lacks Proficiency) that is calculated by averaging scores from courses and particular assignments. For non-WILD courses the 1-4 score will be assigned based on the student’s final letter grade in the course (A- or higher = 4, B- to B+ = 3, C- to C+ = 2, less than C- = 1). For WILD courses that map to a single learning objective, faculty members will have the option of using the final letter grade in their course to assess student progress on the learning objective, or performance on specific assignments and exams. The faculty in charge of “commons curriculum” and “capstone” courses teach their classes holistically to meet the learning objectives of the program. These courses emphasize one or more of the learning objectives across majors, and to help guide appropriate assessment, faculty have been asked to describe the percentage time directed at the learning objectives described herein (e.g., W5 [50%], W4 [20%], W3 [30%]). For these classes, faculty should report separate student assessment scores for each pertinent learning objective using appropriate assignments and/or exams; e.g., through the Canvas learning management system (by creating “Assessments” for departmental use, but not for grading). Results from faculty assessment of students will be posted on our Outcomes Data page by major program by department staff.
Other Assessment Approaches
IDEA Student Ratings
The IDEA course evaluation system adopted by USU has been useful to us in pointing out courses and teaching methods that may need to be modified. IDEA focuses on student assessment of instructors and course material, rather than faculty assessment of students, but administrators and faculty pay close attention to IDEA scores. Moreover, the IDEA system allows instructors to place greater weight on certain survey questions by selecting the learning objective categories that are most emphasized in a given course. Although not a one-to-one link to our program learning objectives, the IDEA instructor ratings provide a source of existing data to track program performance. IDEA scores for specific instructors and courses are provided by the USU AAA office, and summaries are provided on our Outcomes Data page.
Graduating Senior Survey
The department head meets informally with graduating seniors at an end-of-semester luncheon. During the luncheon, the students and department head discuss high and low points of their educational experiences at USU. In the past, students attending the luncheon filled out an anonymous survey to self-assess their accomplishment of the 26 historical learning objectives. We will continue to post results from these surveys on the Outcomes Data page; however, beginning in fall 2016 students will self-assess their accomplishment of the new, contemporary learning objectives using the same scale used by the faculty. Because graduating senior participation in the exit survey has been low in the past, ranging from 34% to 69%, we are evaluating strategies to increase the response rate. Going forward, results from the student self-assessment survey will be posted on our Outcomes Data page by major program by department staff.
Employment & Continuing Education of Graduates
USU Career Services conducted an employment and continuing education telephone survey across graduates from all USU colleges and departments in 2011-12, 2012-13, 2013-14, and 2014-15. The department also conducted a survey in 2008. We feel that these or similar data are very important because they provide an indication of how well we are recruiting and preparing our students to become working natural resource professionals. We are exploring how frequently these university-wide surveys will be conducted in the future, and also to find out what other information is being gathered (if any) and how we can access it. Data from these surveys, and analyses of such data, will be posted on our Outcomes Data page such that we can assess ways to alter our recruitments strategies, placement into specific degree programs, and modification of the degree programs themselves in order to keep up with employment and continuing education demands.
Professional Society Accreditation
We feel that an important part of our program assessment is the periodic external assessment by professional accrediting bodies, namely the Society of American Foresters (SAF) for our Forest Ecology and Management major and the Society for Range Management (SRM) for our Rangeland Ecology and Management major. The Wildlife Ecology and Management and Conservation and Restoration Ecology degrees do not have associated accrediting bodies. Though SAF and SRM accredit those two majors, the entire department, and to some extent the entire college and university, are expected to participate in these assessments of what we are capable of and what we are accomplishing. If successful the SAF and SRM accreditations are valid for 10 years. Our last SRM site visit was in 2008 when we were fully accredited for 10 years, so our next visit should be in 2018. Our last SAF site visit was in 2010 when we were fully accredited for 10 years, so our next visit should be in 2020.
The SAF report in 2010 mentioned that faculty are actively engaged in program planning and assessment through annual retreats and regular meetings during the academic year. Accreditors commented that outcomes assessment developed by forestry faculty is used to guide curriculum development. Assessment results had been cross-referenced with SAF competency areas, and this approach was considered particularly noteworthy, along with our adaptive management model of program improvement.
Learning Objective Assessment
We have just formulated our plan for assessment of student attainment of our contemporary learning objectives. This year we will finalize how we are going to involve staff and faculty to collect these data and calculate results, and our intention is to first do this for the spring program graduates in 2017. Results and some analysis will be presented and discussed at our retreat in August 2017.
IDEA Student Ratings
IDEA summaries for the Wildland Resources Department are available here for Spring 2013, Spring 2014, Fall 2014, Spring 2015, Fall 2015, and Spring 2016. These show our overall response rates, the objectives we use and ratings of importance and progress on each objective, a summary of our IDEA scores and how they compare with others, and ratings of teaching methods and styles.
Graduating Senior Survey
This link takes you to survey results for historical learning objectives for our graduating seniors for 2011-12, 2012-13, 2013-14, 2014-15, and 2015-16. We started using our contemporary learning objectives in 2016-17.
Employment & Continuing Education of Graduates
This link takes you to survey summary documents on our graduates’ employment and/or continuing education and includes data from a 2008 survey the department did, and surveys done by USU Career Services in 2011-12, 2012-13, 2013-14, and 2014-15.
Our most recent SAF and SRM program accreditation reports are available below. Overviews for the SAF accreditation process are available here and for the SRM accreditation process are available here.
Volume I: A Self-Evaluation Report
Volume II: Appendices
Committee on Accreditation Summary Findings and Action
Volume I: A Self-Evaluation Report
Analyses of program assessment outcomes data are shared with faculty annually at our department retreat, and discussed during subsequent departmental meetings. Based on these analyses and discussions, suggested changes to courses, curricula and programs are identified. In large part, the details of any such changes are handled by the departmental Undergraduate Curriculum Committee. Changes made as a result of trends in the program assessment outcomes data are then documented and highlighted in this section of our website.
Example – 2016-17; Increasing Future Emphasis on Technical Writing in English 2010: An important source of information, though it is unavoidably biased, is the students themselves when the department head talks with them at a luncheon just before graduation. Though many are quiet, the students readily speak about problems they encountered. An example this last year is their expression of a perceived need to get more instruction and experience with technical writing while at USU. We are not sure where this perception is coming from. However, upon reflection on our current curriculum, and also after visiting with the English department head about what is, and what is not, included in the English courses that our students are currently required to take, we have begun discussions with the English Department about designating a section (or two) of English 2010 to better meet these needs.
Example – 2016; Increasing Rangeland Resources (RLR) Major Flexibility to Increase Enrollments: One of our faculty, Eric Thacker, talked to a faculty member at Snow College in Ephraim, UT, Chad Dewey, about having students take a portion of their range courses at Snow College. These students would transfer to USU for their senior year having taken a mix of courses that include a) Snow College courses with content coordinated with us such that credits transfer readily and meet our requirements, b) USU broadcast courses that students take before coming to the USU Logan campus, and finally c) courses taken on the USU Logan campus during their senior year. The need for this was supported with our enrollment data for RLR indicating that, although the numbers are increasing, they are still lower than our other major programs. We discussed this idea at our August 2016 retreat and agreed to try it out beginning Fall of 2016.
Example – 2015 and 2016; Changes in Commons Course Prerequisites: Based on student retention data obtained from the USU AAA office we determined that, while enrollments in our undergraduate programs are increasing, especially for our Wildlife Science major, an unacceptably low proportion of our majors graduate (48%). This means we are busy teaching students who will not complete their degree with us, which indirectly negatively affects other students through high student:teacher ratios. Partly as a result of this, and also because students are avoiding some classes until too late in their programs, we added prerequisites to some classes to force students to take some critical classes earlier (like calculus), and raised the level of what counts as having satisfied a prerequisite in other cases. For the latter, we realized that requiring students to take and pass Biology 1610, for example, often was not really meeting ours or the students’ needs, since passing requires only a “D” final grade. We therefore raised the requirement to meet the prerequisite to a C- for several courses. This should moderate our enrollment growth and also should force students to face challenging, required courses earlier so that if they aren’t going to make it to graduation, they and we know that sooner such that students can consider alternative educational and career options. Based on our Fall 2016 enrollment data for our commons entry course, WILD 2400, it appears that this may have worked. After several prerequisite changes in the past year, enrollment in WILD 2400 went from 58 to 44.
Example – 2014; Undergraduate Curriculum Revision: In regard to decisions we have made based on assessment outcomes data, our periodic revision of our undergraduate curriculum is an example. We extensively revised our undergraduate curricula in 2014, changed course requirements, and added several courses to our commons curriculum while removing others. The curriculum committee that designed this revision took into account student survey data and other information (e.g. cancellation of courses offered by other departments). They also used knowledge gained from interviews with agency personnel who hire our students and knowledge of who is being hired in what jobs.
Example – 2014; Change in GIS Requirement: A specific example of a decision to change our curriculum is our now requiring all undergraduate majors to take a GIS class before they graduate. The Forest Ecology and Management and Rangeland Ecology and Management majors had required GIS up to 2014 but the Wildlife Ecology and Management and Conservation and Restoration Ecology majors did not. We made the change to require GIS of everyone because of our changed understanding of what students are expected to be able to do when they get summer and permanent positions, and also what we were requiring in later classes that depended on them having GIS skills.
Example – 2010; Increased Emphasis on Timber Harvesting in Forestry Major: After going through professional society accreditations, we are provided with programmatic recommendations and deficiencies. When the department met with the SAF accreditation team in 2010, they identified a need for our Forest Ecology and Management students to learn more about forest harvest operations. A course on this topic used to exist, but due to changes in the expertise of our faculty and other priorities in our curriculum, that subject was no longer being addressed. We partially addressed the SAF requirement by having the forestry Extension team take students on the annual timber harvest tour that is held in the fall and other programs that Extension puts on, like webinars on the subject.