The Wildland Resources Department at USU has four undergraduate majors: Conservation and Restoration Ecology, Forest Ecology and Management, Rangeland Ecology and Management, and Wildlife Ecology and Management. The Wildland Resources faculty have designated five similar but unique learning objectives (LOs) for each major (click here for LO table). Briefly, these LOs are 1) knowledge of biology and ecology, 2) ability to collect and analyze data, 3) understanding of the social context of their field, 4) ability to communicate, and 5) understanding of and ability to apply what they have learned to problems in their field. We feel that, if a student can accomplish all of that by the time they graduate they will be ready for their next step(s), whether those are getting a professional position, going to graduate school, or being an ecologically fluent citizen.
Overall, we want our graduating students to become professionals that can excel at the work society needs them to do. To measure achievement of this goal, we assess student attainment of the LOs for their majors. We also look at a number of other indicators that can show success, but less directly, like employment, program accreditation, and student self-assessment.
Student Progress on Learning Objectives
Attainment of LOs for our students is indicated by how they do in a combination of the courses we teach and courses that others teach. For our courses, elements of the LOs are incorporated into course syllabi and into the course objectives we specify in the IDEA Student Ratings program (go to www.usu.edu/aaa/idea.cfm for a description of the IDEA program). For required and certain elective courses we don’t teach we have examined their syllabi and have determined which LOs they address. To quantify student achievement of the LOs for their major, after they graduate we obtain their final grades for a set of required and elective courses, chosen from all of the courses they take, that we think best-indicate success for a particular LO and major (click here for Course Table with Learning Objectives). We average those course grades for each student for each LO and those averages are their LO scores. Some course grades are used without weighting while others are down-weighted (multiplied by a weighting factor between 0.10 and 0.75) if we determine that they should contribute less to the total. Some LOs have many courses that are averaged to calculate their score (CREC LO1 has 13) while some have only a single course that goes into the score (FEMA LO5 has one course that goes into its score).
Because the scores are means of grades on a 4 point scale they essentially are grade point averages (GPAs), but instead of us using a student’s overall GPA we calculate a GPA for each LO and major. We convert the 4-point scale to Achieves Mastery (AM) = 3.33 to 4, Achieves Proficiency (AP) = 2.66 to 3.33, Approaching Proficiency (ApP) = 2 to 2.66, and Lacks Proficiency (LP) < 2. Note that the last score should not really occur very often though, since we are only doing these calculations for students who graduated, and they are required to get a grade of at least 2.5 in all of their NR courses. Results from these calculations are posted by student (with identifying information removed) and averages also are posted for each major.
Other Assessment Approaches
IDEA Student Ratings
The IDEA course evaluation system adopted by USU has been useful to us in pointing out courses and teaching methods that may need to be modified. IDEA focuses on student assessment of instructors and course material, rather than faculty assessment of students, but administrators and faculty pay close attention to IDEA scores. Moreover, the IDEA system allows instructors to place greater weight on certain survey questions by selecting the learning objective categories that are most emphasized in a given course. Although not a one-to-one link to our program learning objectives, the IDEA instructor ratings provide a source of existing data to track program performance. IDEA scores for specific instructors and courses are provided by the USU AAA office, and summaries are provided on our Outcomes Data page.
Graduating Senior Survey
The department head meets informally with graduating seniors at an end-of-semester luncheon. During the luncheon, the students and department head discuss high and low points of their educational experiences at USU. In the past, students attending the luncheon filled out an anonymous survey to self-assess their accomplishment of the 26 historical learning objectives. We will continue to post results from these surveys on the Outcomes Data page; however, beginning in fall 2016 students will self-assess their accomplishment of the new, contemporary learning objectives using the same scale used by the faculty. Because graduating senior participation in the exit survey has been low in the past, ranging from 34% to 69%, we are evaluating strategies to increase the response rate. Going forward, results from the student self-assessment survey will be posted on our Outcomes Data page by major program by department staff.
Employment & Continuing Education of Graduates
USU Career Services conducted an employment and continuing education telephone survey across graduates from all USU colleges and departments in 2011-12, 2012-13, 2013-14, and 2014-15. The department also conducted a survey in 2008. We feel that these or similar data are very important because they provide an indication of how well we are recruiting and preparing our students to become working natural resource professionals. We are exploring how frequently these university-wide surveys will be conducted in the future, and also to find out what other information is being gathered (if any) and how we can access it. Data from these surveys, and analyses of such data, will be posted on our Outcomes Data page such that we can assess ways to alter our recruitments strategies, placement into specific degree programs, and modification of the degree programs themselves in order to keep up with employment and continuing education demands.
Professional Society Accreditation
We feel that an important part of our program assessment is the periodic external assessment by professional accrediting bodies, namely the Society of American Foresters (SAF) for our Forest Ecology and Management major and the Society for Range Management (SRM) for our Rangeland Ecology and Management major. The Wildlife Ecology and Management and Conservation and Restoration Ecology degrees do not have associated accrediting bodies. Though SAF and SRM accredit those two majors, the entire department, and to some extent the entire college and university, are expected to participate in these assessments of what we are capable of and what we are accomplishing. If successful the SAF and SRM accreditations are valid for 10 years. Our last SRM site visit was in 2008 when we were fully accredited for 10 years, so our next visit should be in 2018. Our last SAF site visit was in 2010 when we were fully accredited for 10 years, so our next visit should be in 2020.
The SAF report in 2010 mentioned that faculty are actively engaged in program planning and assessment through annual retreats and regular meetings during the academic year. Accreditors commented that outcomes assessment developed by forestry faculty is used to guide curriculum development. Assessment results had been cross-referenced with SAF competency areas, and this approach was considered particularly noteworthy, along with our adaptive management model of program improvement.
Learning Objective Assessment
We have implemented our plan for assessment of student attainment of the learning objectives for each major. Data for the 2015-16 school year is presented above. Results will be presented and discussed at our retreat in August 2017.
IDEA Student Ratings
IDEA summaries for the Wildland Resources Department are available here for Spring 2013, Spring 2014, Fall 2014, Spring 2015, Fall 2015, and Spring 2016. These show our overall response rates, the objectives we use and ratings of importance and progress on each objective, a summary of our IDEA scores and how they compare with others, and ratings of teaching methods and styles.
Graduating Senior Survey
This link takes you to survey results for historical learning objectives for our graduating seniors for 2011-12, 2012-13, 2013-14, 2014-15, and 2015-16. We started using our contemporary learning objectives in 2016-17.
Employment & Continuing Education of Graduates
This link takes you to survey summary documents on our graduates’ employment and/or continuing education and includes data from a 2008 survey the department did, and surveys done by USU Career Services in 2011-12, 2012-13, 2013-14, and 2014-15.
Our most recent SAF and SRM program accreditation reports are available below. Overviews for the SAF accreditation process are available here and for the SRM accreditation process are available here.
Volume I: A Self-Evaluation Report
Volume II: Appendices
Committee on Accreditation Summary Findings and Action
Volume I: A Self-Evaluation Report
Analyses of program assessment outcomes data are shared with faculty annually at our department retreat, and discussed during subsequent departmental meetings. Based on these analyses and discussions, suggested changes to courses, curricula and programs are identified. In large part, the details of any such changes are handled by the departmental Undergraduate Curriculum Committee. Changes made as a result of trends in the program assessment outcomes data are then documented and highlighted in this section of our website.
Example – 2016-17; Increasing Future Emphasis on Technical Writing in English 2010: An important source of information, though it is unavoidably biased, is the students themselves when the department head talks with them at a luncheon just before graduation. Though many are quiet, the students readily speak about problems they encountered. An example this last year is their expression of a perceived need to get more instruction and experience with technical writing while at USU. We are not sure where this perception is coming from. However, upon reflection on our current curriculum, and also after visiting with the English department head about what is, and what is not, included in the English courses that our students are currently required to take, we have begun discussions with the English Department about designating a section (or two) of English 2010 to better meet these needs.
Example – 2016; Increasing Rangeland Resources (RLR) Major Flexibility to Increase Enrollments: One of our faculty, Eric Thacker, talked to a faculty member at Snow College in Ephraim, UT, Chad Dewey, about having students take a portion of their range courses at Snow College. These students would transfer to USU for their senior year having taken a mix of courses that include a) Snow College courses with content coordinated with us such that credits transfer readily and meet our requirements, b) USU broadcast courses that students take before coming to the USU Logan campus, and finally c) courses taken on the USU Logan campus during their senior year. The need for this was supported with our enrollment data for RLR indicating that, although the numbers are increasing, they are still lower than our other major programs. We discussed this idea at our August 2016 retreat and agreed to try it out beginning Fall of 2016.
Example – 2015 and 2016; Changes in Commons Course Prerequisites: Based on student retention data obtained from the USU AAA office we determined that, while enrollments in our undergraduate programs are increasing, especially for our Wildlife Science major, an unacceptably low proportion of our majors graduate (48%). This means we are busy teaching students who will not complete their degree with us, which indirectly negatively affects other students through high student:teacher ratios. Partly as a result of this, and also because students are avoiding some classes until too late in their programs, we added prerequisites to some classes to force students to take some critical classes earlier (like calculus), and raised the level of what counts as having satisfied a prerequisite in other cases. For the latter, we realized that requiring students to take and pass Biology 1610, for example, often was not really meeting ours or the students’ needs, since passing requires only a “D” final grade. We therefore raised the requirement to meet the prerequisite to a C- for several courses. This should moderate our enrollment growth and also should force students to face challenging, required courses earlier so that if they aren’t going to make it to graduation, they and we know that sooner such that students can consider alternative educational and career options. Based on our Fall 2016 enrollment data for our commons entry course, WILD 2400, it appears that this may have worked. After several prerequisite changes in the past year, enrollment in WILD 2400 went from 58 to 44.
Example – 2014; Undergraduate Curriculum Revision: In regard to decisions we have made based on assessment outcomes data, our periodic revision of our undergraduate curriculum is an example. We extensively revised our undergraduate curricula in 2014, changed course requirements, and added several courses to our commons curriculum while removing others. The curriculum committee that designed this revision took into account student survey data and other information (e.g. cancellation of courses offered by other departments). They also used knowledge gained from interviews with agency personnel who hire our students and knowledge of who is being hired in what jobs.
Example – 2014; Change in GIS Requirement: A specific example of a decision to change our curriculum is our now requiring all undergraduate majors to take a GIS class before they graduate. The Forest Ecology and Management and Rangeland Ecology and Management majors had required GIS up to 2014 but the Wildlife Ecology and Management and Conservation and Restoration Ecology majors did not. We made the change to require GIS of everyone because of our changed understanding of what students are expected to be able to do when they get summer and permanent positions, and also what we were requiring in later classes that depended on them having GIS skills.
Example – 2010; Increased Emphasis on Timber Harvesting in Forestry Major: After going through professional society accreditations, we are provided with programmatic recommendations and deficiencies. When the department met with the SAF accreditation team in 2010, they identified a need for our Forest Ecology and Management students to learn more about forest harvest operations. A course on this topic used to exist, but due to changes in the expertise of our faculty and other priorities in our curriculum, that subject was no longer being addressed. We partially addressed the SAF requirement by having the forestry Extension team take students on the annual timber harvest tour that is held in the fall and other programs that Extension puts on, like webinars on the subject.