Assessment

Learning Objectives

Environmental Studies

By the end of their program, students can….

1. Describe and explain environmental issues from a rigorous interdisciplinary perspective by integrating insights based on principles, theories, and information from the natural sciences, social sciences, and humanities.
2. Demonstrate understanding of ethical dimensions of environmental issues.
3. Apply, evaluate, and interpret appropriate quantitative methods (e.g. geographic information systems, statistics, and systems modeling) to gain information and understanding of environmental problems. 
4. Synthesize and integrate material in interdisciplinary team-structured activities.
5. Communicate information necessary to understand environmental problems and solutions in written, oral, and visual formats to professional and lay audiences in a manner that is both scientifically accurate, relatable and understandable by the audience.
6. Demonstrate understanding of how the interactions between humans and the environment affect human health, ecosystem health, resource availability, sustainability, and political stability.
7. Demonstrate understanding of how environmental policy is made and how, together with economic incentives, it structures people’s use of environmental resources.

Learning Objective Centered Courses

Course	Learning Objective	Assignments/Exams
ENVS 2340	1	Multiple assignments
ENVS 3010	1	Mid-term exam grade
ENVS 2340	2	Multiple assignments
GEOG/WILD 1800	3	Labs
ENVS 3500	3	Find and assess a correlation interpreted as causation, a "false" graph (or a graph that due to scaling showcases inaccurate information) and a biased study within the news and/or social media outlets. Explain why the mistake is made and what the article is actually suggesting.
ENVS 3010	3	Benefit-cost assignment
ENVS 2340	4	A group discussion, group oral presentations, and group posters
ENVS 4700	4	Service Learning Project
ENVS 2340	5	Group discussions, group presentations, group posters, and individual NRSIA article analysis
ENVS 4700	5	Service Learning Project
ENVS 5000	5	Final project report: students are asked to apply the full suite of course concepts to a nonprofit organization in which they have been performing volunteer service during the semester.
ENVS 2340	6	Two tests, final exam, NRSIA article analyses, no-pressure quizzes
ENVS 4700	6	In-class discussions, online discussions, service learning project
ENVS 3010	7	Mid-term and grade on legal case briefs

Geography

By the end of their program, students can….

1. Demonstrate proficiency in application of basic geographic terminology, principles, and concepts.
2. Explain the diversity and interdependence of regions, places and locations.
3. Interpret connections between the natural world and human society.
4. Draw knowledge, understanding and a diversity of approaches from other disciplines to synthesize them in a geographical context.
5. Apply geographic information systems (GIS) to the analysis of geographic data and spatial relationships.
6. Apply and interpret appropriate basic statistical and other quantitative analyses for geographic data, including spatially-explicit data.
7. Communicate geographical ideas and information effectively and fluently by written, oral and visual means. 

Learning Objective Centered Courses

Course	Learning Objective	Assignments/Exams
GEOG 1000	1	Exams
GEOG 1300	1	Exams
GEOG 4100	1	Research paper
GEOG 4210	1	Critical reflection paper
GEOG 4210	2	County presentation
GEOG 4100	3	Research paper
GEOG 4210	3	Field trip written assignment
GEOG 4400*	3	Disaster presentation
GEOG 4120*	4	Third essay
GEOG 4210	4	TBD
GEOG 4400*	4	Midterm and final exams
GEOG 1800	5	Final project
WATS 4930**	5	Labs
WATS 4931**	5	Research project
STAT course	6	Course grade
GEOG 1800	6	Final exam; final project
WATS 4930**	6	Lab on spatial statistics
WATS 4931**	6	Research project
GEOG 1800	7	Final project
GEOG 4120*	7	Third essay; group presentation
GEOG 4210	7	County presentation; critical reflection paper; group project and presentation

*course required only for the Human-Environment Geography Emphasis
**course required only for the Geographical Information Science Emphasis (name change pending approval)

Recreation Resources Management

By the end of their program, students can…. 

1. Identify and articulate central foundations, theories and ideas, and best approaches and practices in RRM.
2. Utilize theories, principles, and knowledge of RRM to address management issues and challenges.
3. Utilize theories, principles, and knowledge of related disciplines to address management issues and challenges.
4. Quantify and analyze recreational use and associated impacts utilizing research approaches and methods, sampling and measurement, and data analysis techniques for managing recreation resources.
5. Write logical and analytical papers supported by appropriate research.
6. Determine, apply, and interpret appropriate basic statistical or other quantitative analyses to RRM data
7. Productively conduct group/team work to deliver professional quality presentations and reports.
8. Demonstrate basic competency in the use of geographic information systems and field data collection using global positioning systems.

Learning Objective Centered Courses

Course	Learning Objective	Assignments/Exams
ENVS 3300	1	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 4130	1	Writing Assignments- Final Research Project Paper
ENVS 4500	1	Writing Assignments- Final Research Project Paper
ENVS 4600	1	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 3300	2	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 4130	2	Writing Assignments- Final Research Project Paper
ENVS 4500	2	Writing Assignments- Final Research Project Paper
ENVS 4600	2	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 4920	2	Final Report
ENVS 3300	3	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 4130	3	Writing Assignments- Final Research Project Paper
ENVS 4500	3	Writing Assignments- Final Research Project Paper
ENVS 4600	3	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 4920	3	Multiple assignments (quizzes/exams, exercises, writing assignments)
ENVS 3500	4	Final synthesis project
ENVS 4500	4	Final Research Project
ENVS 4550	4	Final Research Project Report and Presentation
ENVS 4130	5	Writing Assignments- Final Research Project Paper
ENVS 4500	5	Writing Assignments- Final Research Project Paper
ENVS 3500	6	TBD
ENVS 4550	6	Final Research Project Report and Presentation
ENVS 4130	7	Final Research Project Presentation
ENVS 4500	7	Final Research Project Presentation
ENVS 4550	7	Final Research Project Presentation
GEOG/WILD 1800	8	Final project

Graduate Degrees

On completion of the requirements for each MS degree, students can:

1. Demonstrate mastery of relevant subject material at a level appropriate to master’s attainment.
2. Conduct, present, and defend a body of research.
3. Conduct scholarly activities in an ethical manner.

On completion of the requirements for the PhD degree, students can:

1. Demonstrate mastery of subject material at a level appropriate to doctoral attainment.
2. Produce, present, and defend original contribution to knowledge.
3. Conduct scholarly activities in an ethical manner.

Curriculum Assessment Plan

At the start of the 2016-2017 academic year, the ENVS faculty transitioned from departmental learning objectives to program-specific learning objectives.  This transition was accompanied by the adoption of a new assessment plan, to measure learning outcomes at the program level. The ENVS Department will begin implementing the new assessment plan during the 2016-2017 academic year, with the first set of data collected for spring semester 2017.

The core of this new plan is the addition of course-embedded assessment for the measurement of student outcomes for the program-specified learning objectives. For each learning objective, the ENVS faculty have selected at least one course for embedded assessment. Over the course of the 2016-2017 academic year, the instructor for each of these courses will identify the most appropriate assignment or test/exam for assessment use. The table below provides details on this course and assignment specification. The Environmental Studies program curricula are currently undergoing revision by the faculty. As such the course selections listed in the table below are tentative and subject to change as soon as the revisions are completed. The Recreation Resources Management program faculty have not yet completed course selection for the course-embedded assessment. Details for this program will be posted as soon as selection has been completed.

At the end of each semester, these instructors are responsible for submitting the outcomes data to departmental staff, for students in these courses in the designated program. Instructors will convert assignment or test/exam grades (or the relevant portions of grades) to a scale of 1 to 4 and report a frequency distribution:

• 4 = achieves mastery
• 3 = achieves proficiency
• 2 = approaches proficiency
• 1 = lacks proficiency

In addition, the department will continue to employ the following sources of additional data:

1. IDEA course evaluations (see summary of prior assessment plans)
2. Graduating student exit interviews (see summary of prior assessment plans)
3. USU’s Career Services Alumni Survey, conducted 1 year following graduation

Data from all sources will be reviewed regularly by the faculty, or by subsets of faculty for each program (Environmental Studies, Geography, and Recreation Resource Management). These data will guide faculty decisions for regular program revisions and may lead to specific recommendations for curricular changes at the course-level, as well as at the program-level. 

Summary of Prior Assessment Plans (2009-2016)

Previously, program assessment occurred through several mechanisms, based on a set of 34 learning objectives, shared across all programs within the Environment and Society Department. These mechanisms included:

1. A survey administered to students in all ENVS courses at the course level. This survey asked students to assess their own achievements relative to the 34 learning objectives, based on the course in question. The survey was administered at the same time as the prior USU standardized course evaluation instrument, which has since been replaced by the IDEA instrument. Data from this survey were regularly collated by the departmental staff and periodically reviewed by the faculty at their annual retreat each August. Usually, these data were reviewed by a subset of the faculty in conjunction with a process of program revision, initiated by other factors, such as a changing market, enrollment trends, or feedback from student exit interviews (see below). The department stopped administering this survey instrument in 2011, when the university moved to the IDEA course evaluation system. The rationale for ceasing use of our own instrument was the inclusion in the IDEA instrument of student assessment of progress on instructor-selected learning objectives.
2. The IDEA course evaluation system. Starting in 2011, the department began using the data from the new campus course evaluation system to assess both faculty teaching performance and student-reported progress on the IDEA learning objectives. The department head retrieved and analyzed this data for all departmental courses from the website of USU’s Office of Analysis, Assessment and Accreditation on a semester or annual basis. The results were reviewed and discussed by departmental faculty at the annual retreat and/or a faculty meeting.
3. Exit interviews conducted with graduating seniors. Each April/May, the department head conducted exit interviews with graduating seniors to elicit qualitative feedback on student satisfaction with their programs. These interviews occurred in person and could include one or more students at a time. Interview data were summarized in a report form and shared with faculty as judged necessary by the department head. Often only a small subset of students scheduled the exit interview.

Outcomes Data

Undergraduate Progams

The newly adopted assessment plan (AY 2016-17) for the undergraduate programs includes the collection of course-embedded assessment data. This data collection will begin spring semester 2017. Program assessment outcomes data will be posted here as they become available.

IDEA Course Evaluation Data

Summary data from course evaluations provided by IDEA are available at the Departmental rather than the program level; these are summarized in the table below. These scores measure student’s perceptions of their progress on learning objectives and the quality of the course and its instructor rather than objective measures of learning. They are especially valuable in identifying what students find to be outstanding teaching (higher and especially much higher than average) as well as below standard teaching (lower and especially much lower). Scores in the “much lower” range merit a response from the Department Head, starting with identifying what students in the course perceive to not be meeting their educational expectations.

ENVS students find their progress on relevant learning objectives to be substantially better than the IDEA Database and USU norms, with a few problems in spring 2016 semester than have been corrected by changing instructors. They rate over 80% of ENVS instructors similar or higher than the IDEA and USU norms, and find the level of excellence of ENVS courses to be quite substantially higher than the IDEA and USU norms. The averages among these three measures show in fall semester 2015 for ENVS, compared to an anticipated 10%, 20%, 40%, 20%, 10% distribution, 22% much higher, 48% higher, 26% similar, 4% lower and 0% much lower. This places ENVS courses far above the anticipated performance level. For Spring semester 2016 ENVS student perceptions of ENVS courses are somewhat less outstanding with 12% much higher, 35% higher, 41% similar, 6% lower, and 6% much lower.  IDEA data thus show that students have a very favorable perception of the ENVS courses they take and indicate that ENVS faculty are very good teachers in student’s eyes.

Distribution of IDEA scores in ENVS Relative to IDEA Database

Distribution of IDEA scores in ENVS Relative to USU

Exit Interviews Data

Feedback from student exit interviews as well as focus groups held with undergraduate students during the strategic planning process indicate that ENVS courses and faculty are well-received by USU students, but curricular integration across degree programs has flaws. While perspectives among students naturally vary, a central theme was well articulated by one student:

“What I’ve noticed throughout my 5 years as an ENVS student is that there is a lot of overlap in course materials (e.g., ENVS 2340, 3330, 4000). I believe this is also what some of the other students meant about us being very well informed on the history of humans and the environment, and not so much on the skills aspect of it.”

Additional feedback indicates that students desire more instruction in skills such as GIS, economics, legal interpretation, survey design, and communicating and engaging with stakeholders.  This theme has guided our revision of the curricula in Geography and Environmental Studies.

Alumni Survey Data (collected by USU Career Services)

Career Services Annual and Graduate Reports for 2013-14 and 2014-15 show that ENVS graduates are successful in gaining employment, with 81.3% of graduates employed in each year surveyed.  ENVS alumni are working throughout the U.S., about half in Utah, with the majority holding titles the reflect their undergraduate training. These data are comparable to the Quinney College of Natural Resources and substantially better than employment rates for USU students as whole.  From 2013-14 to 2014-15, there was a substantial shift of non-employed graduates to continuing education status, with no ENVS graduates unemployed/still seeking in 2014-15.  Note that the response rate on the alumni surveys that generate these data is 55% in 2013-14 and 56% in 2014-15 for QCNR.

Employment status of QCNR graduates

Graduate Programs

Data on applications, acceptances, matriculations, and graduations in five ENVS masters and two PhD programs from AY 2011-12 to 2015-16 shows 169 total applications, of which 88 (52%) were accepted, of which 37 (42%) matriculated.  There is thus a need to improve the proportion of accepted students who choose to begin graduate school in ENVS. Graduations (37) equal to matriculations is an indicator of consistency over time and a high retention rate. 

Among the five master’s programs, Bioregional Planning has been ceded to LAEP and Ecology, in which ENVS plays a secondary role, had only 1 applicant and no acceptances.  Human Dimensions of Ecosystem Science and Management (renamed Environment and Society in 2015), Geography, and Recreation Resource Management (RRM) each received 25-52 applications, accepting 11 to 27, and matriculating 4 to 10 in the five-year period.  The low proportion of students accepted in Geography who decided to enroll (7 of 27; 26%), and the low number of graduations demonstrates a need for improvement in the pipeline from applicants (52) to graduations (2), compared to 15 graduations from 30 applicants for HDESM/Environment and Society and 6 graduations from 25 applicants in RRM.  Among these Masters programs, mean years to graduation varies from 2.5 years for Geography to 3.5 years for HDESM/E&S, with the vast majority completing their degrees within the three-year target window.

At the PhD level, HDESM (renamed Environment and Society in 2015) is the primary program in ENVS, with participation in Ecology as well. Data show a pipeline of 27 applicants yielding 13 acceptances, yielding 5 matriculations over the 5-year period.  Eight graduations took a mean of 5.38 years, a reasonable mark, with half within the target period of 4 years from Masters to PhD and 6 years from Bachelors to PhD.  One PhD in Ecology graduated in 5 years.

These data indicate two areas in need of improvement: recruiting more and better applicants, and getting a higher percentage of those accepted to come. ENVS has upgraded its recruitment efforts by instituting a Graduate Selection Committee formed of three rotating faculty, and soliciting by e-mail from the GRE-taker database, among other measures.  Comparing applicants to GRE-taker solicitations provides evidence that this was effective when first employed in 2015-16.  Offering to pay travel expenses for top candidates to visit is the next step to be taken, especially in Geography where matriculations rates are low.

Data on graduate student publishing and presentations will be available after January 10.

Applications

Year	MS Bioregional Planning	MS Ecology	MS HDESM/Environment & Society	MS Geography	MS Recreation Resources Management	PhD HDESM/Environment & Society	PhD Ecology	Yearly Total Applications
2011-2012	5	1	6	5	5	3	1	26
2012-2013	2	1	9	19	6	8	2	47
2013-2014	6	1	5	9	5	5	1	32
2014-2015	3	1	4	12	7	7	0	34
2015-2016	5	3	6	7	2	4	3	30
Total Applications per Program	21	7	30	52	25	27	7	169

Acceptances

Year	MS Bioregional Planning	MS Ecology	MS HDESM/Environment & Society	MS Geography	MS Recreation Resources Management	PhD HDESM/Environment & Society	PhD Ecology	Yearly Total Acceptances
2011-2012	5	0	4	3	2	2	0	16
2012-2013	0	0	7	6	3	1	1	18
2013-2014	6	0	2	5	1	4	0	18
2014-2015	2	0	1	7	4	4	0	18
2015-2016	2	1	4	6	1	2	2	18
Total Acceptances per Program	15	1	18	27	11	13	3	88

Matriculations

Year	MS Bioregional Planning	MS Ecology	MS HDESM/Environment & Society	MS Geography	MS Recreation Resources Management	PhD HDESM/Environment & Society	PhD Ecology	Yearly Total Matriculations
2011-2012	3	0	2	2	2	1	0	10
2012-2013	0	0	3	2	0	0	0	5
2013-2014	3	0	1	0	1	1	0	6
2014-2015	2	0	1	1	1	2	0	7
2015-2016	2	0	3	2	0	1	1	9
Total Matriculations per Program	10	0	10	7	4	5	1	37

Graduations

Year	MS Bioregional Planning	MS Ecology	MS HDESM/Environment & Society	MS Geography	MS Recreation Resources Management	PhD HDESM/Environment & Society	PhD Ecology	Yearly Total Graduations
2011-2012	2	0	3	0	3	1	0	9
2012-2013	2	0	4	1	1	1	1	10
2013-2014	0	0	2	0	2	0	0	4
2014-2015	1	0	3	0	0	4	0	8
2015-2016	0	0	3	1	0	2	0	6
Total Graduations per Program	5	0	15	2	6	8	1	37

Time to Degree in ENVS Graduate Programs

Degree	Mean Number of Years	Percent Below Target
MS Bioregional Planning	2.6	80%
MS Ecology	n/a	n/a
MS HDESM/Environment & Society	3.53	80%
MS Geography	2.5	100%
MS Recration Resources Managment	2.67	83%
PhD HDESM/Environment & Society	5.38	50%
PhD Ecology	5	0%

Target is 3 years for masters, 4 years from masters to PhD and 6 years from Bachelors to PhD.

Departmental Assessment

• 2020 Departmental Self-Study