Testing An Approach for Prioritizing Programs in Tough Financial Times

Since the Great Recession began in 2008, one cannot seem to find a day in which the media do not highlight the continued difficult financial issues facing higher education. While institutions have confronted tough fiscal problems before, including the recessions of the 1980s, the current period may be unrivaled for the severity of cuts to programs and staff, including the jettisoning of academic programs and faculty.

Budget realities coupled with the drive to be more accountable and outcomes-driven-even as larger, more diverse, and often unprepared student bodies enter-has forced many institutions to make serious decisions about changing their programs and services and ensuring that they align with their mission as well as their market. Program reviews and analyses have often been demanded when a sudden budget cut was apparent or an institution faced an accreditation or other review, but the current budget problems are of a different scale. While there will never be a truly bias-free method for reviewing programs, the process offered by Robert C. Dickeson (2010) is robust, inclusive, transparent, and uses qualitative and quantitative measurements that, in essence, review the art and wisdom of programs as well as their outputs.

As a long-time student affairs professional, I know that programs other than those that are truly academic have been cut without much attention to whether the program succeeded or failed in meeting the needs of the institution. While my former institution, Mt. Hood Community College (MHCC), has also faced budget cuts, there was no mandate for a campuswide or divisionwide program-review process. However, when the Introduction to College program was eyed for possible elimination, I proposed using an updated model to not only review it but also to test Dickeson's (2010) prioritization model in the process. While his framework calls involving all institutional constituencies to set criteria, apply the mission, and come to agreement, in this situation, I was the sole evaluator. As such, I worked only with the criteria application process. Since I also had been one of the senior implementers of the program I planned to evaluate, I followed the American Evaluation Association's Guiding Principles for Evaluators (2004).  

Dickeson (2010) has posited that his schema, which identifies 10 criteria to consider in prioritizing academic programs, enables evaluators to more cogently review a program because it allows for richness of text, quality, and reflection in addition to numerical outcomes. I gathered data on Introduction to College, a one-credit course that has been offered at no cost to incoming students since 2004. The course has existed in various forms for nearly 30 years, however. In addition to data on course completion, costs, faculty and staff involvement, institutional mission and goals, and student persistence rates, I also used my historical and theoretical knowledge and experience in best practices to review the program. I rated each criterion with a grade of excellent, satisfactory, or poor.

1. History, development, and expectations of the program: This criterion allows the evaluator to dig into the philosophy and background of the program's initiation, growth, and maturation. I was able to review the history of the program, which has had several iterations over nearly 30 years, from its humble yet prescient beginnings to its more mature, theoretical, and best-practice driven curriculum and outreach milieu. Of particular note was the richness of the updated course material and pedagogy as well as the involvement of staff members and faculty of various levels from departments and disciplines across the campus. The support of the board of education and the college president, both financially and philosophically, also were inherent. While aspects of the program's development had not been as successful as had been hoped, a high school outreach component in particular, the use of analytics ascertained the success of the program relative to board goals and in tandem with national research. Overall, this criterion was rated satisfactory.

2. External demand for the program: While this criterion is probably more relevant to programs that have a well-focused external presence such as athletics, certain academic programs, and curricular structures, I contended that the literature around persistence and, increasingly, completion argue for strong intervention in the first year of college, especially in terms of extended orientation programs (Patton, Morelon, Whitehead & Hossler, 2006). As a result, I evaluated this criterion as satisfactory.

3. Internal demand for the program: Like the second criterion, there was some interest on the part of various campus constituencies to expand this type of course. While others noted that many areas covered by the course fulfilled a need for persistence education, there was no specificity about the course serving those needs. I attributed the absence of this information as a lack of knowledge about the nature of persistence and the research concerning it. I rated this section as satisfactory because this type of orientation program is an example of the kind of intervention many students need to persist beyond their first year.

4. Quality of program inputs and processes: This area encompassed the measurement of various factors. Faculty quality was essential because the nature of the course required teachers who would be able to work with a range of students in a subject, and perhaps pedagogy, that may be outside their sphere. They must also be able to dedicate more time than they usually would to a one-credit course, such as offering more general help to students outside of class in addition to class-related office time. While the program was able to recruit and train full-time and part-time faculty to participate, it was difficult to encourage them to spend more time on the class than their collective bargaining contracts allowed. While we were able to attract teachers from various staff levels, it was difficult to recruit an ethnically diverse group of faculty members to participate. Efforts to increase the number of faculty who were male, of color, and from scientific disciplines were somewhat successful. Perhaps inherent to this particular issue of faculty involvement was the lack of support for enrollment and course scheduling and planning among the deans and between the student affairs and academic staff. Campus politics seemed to play a role. The quality of students also came into play. As the course was not mandatory for any student, enrollment depended entirely on student outreach, marketing, and communication with staff, advisors, and faculty members. With regard to first-term grades, the limited research analysis provided by the college's Research and Planning Office, which did not include pre-college scores on the ACT or similar tests, revealed no difference between students who took the course and those who did not. (Mt. Hood Community College, 2005). The breadth and depth of the curriculum was also taken into account; it was robust, perhaps too much so for one credit hour. There were discussions about increasing the course credits, but doing so would bring up difficult issues-financial, functional, and otherwise. Adaptability to technology was another factor in the evaluation. While the class goals included being able to demonstrate an aptitude in using the library and various computer applications, the technological abilities of individual faculty members varied. Additionally, while equipment and facilities appeared to be adequate, the scheduling of the one-hour, multi-section course sometimes posed logistical problems for other classes. The course was also dependent on the largesse of functional areas for presentations on library service, financial aid, and advising; good relationships among the course's faculty and other staff members were essential. While the program met the criterion's basic qualifications, many areas within it were on shaky ground. The program needed to become a more formal and integral part of the curriculum that was embraced by all the functional areas involved with it. Therefore, I gave the program a low satisfactory rating.

5. Quality of program outcomes: Since this one-class program did not specifically measure general education outcomes, the outcomes were premised primarily on whether it met MHCC's pre-2009 goals. As part of a larger body of pilot programs focused on increasing persistence, the program surpassed the goals set for a yearly increase in overall college student persistence and also indicated a 20% increase in persistence in one cohort year among students who took the course compared with those who had not (Mt. Hood Community College, 2005). While these certainly are considered successes, these studies focused on small samples and were measured in intermittent years. This criterion was rated as satisfactory.

6. Scope, size, and productivity of the program: While the program was able to run adequately with the staff available, it required a more consistent review and evaluation process. A thorough complex, analysis was necessary to ensure the program was in line with MHCC's goals and other initiatives. While the program was given a satisfactory rating, the complexity of its scope needed more attention.

7. Revenues and resources generated by the program: This area was not applicable because none were generated.

8. Costs and other expenses associated with the program: There was some difficulty finding costs associated with all of the expenses since some, such as faculty salaries, were credited to a primary department. Analyses of non-direct funds were not available. While the program was costly, it was reasonably efficient in its use of direct funds. However, a more complex analysis needed to be carried out in comparison with other departments and in relationship to cost per student to glean effectiveness in this area. Due to a lack of available or relevant cost data, it was rated as poor.

9. Impact, justification, and overall essentialness of the program: As discussed previously, student persistence is an important goal not only at MHCC but also in higher education, student affairs, and, increasingly, in national conversation. This importance provided support to justify the program, its impact, and essential role on campus. However, its structure, politics, and cost required additional attention. As a result, a measured satisfactory rating was given.

10. Opportunity analysis of the program: The program adhered to MHCC's mission and goals and provided effective strategies to meet the explicit and implicit needs of constituencies. It provided adequate program planning and implementation in a variety of ways, but budget, structural planning, and policy issues needed to be addressed. The program had lost some of its robustness over its nearly 30-year evolution. However, it remains a program that benefits the increasing numbers of unprepared incoming students entering an environment that is foreign and scary to many. Again, it received a measured satisfactory rating. Overall, the program met the desired outcome goals of increased persistence and was in line with research and empirical data. In general, this exercise was very beneficial in teasing out the many facets of the program's strengths and weaknesses. It offered definitive directions for increasing program effectiveness and identified various policy choke points for a variety of initiatives, particularly in the chasm between academic and student affairs. While this process worked well, it would have been better to have ready access to complete reports, analytics, and metrics. I would encourage student services staff to undertake such an exercise as they annually review their programs for efficacy and relevance.    


References

American Evaluation Association. (2004). Guiding Principles for Evaluators. Retrieved from http://www.eval.org/Publications/GuidingPrinciples.asp 

Dickeson, R. C. (2010). Prioritizing academic programs and services: Reallocating resources to achieve strategic balance. San Francisco: Jossey-Bass.

Mt. Hood Community College. (2005). Report to the Board of Education. Gresham, OR.

Patton, L.D., Morelon, C., Whitehead, D. M. & Hossler, D. (2006). Campus-based initiatives: Does the emperor have clothes? In E.P. St. John & M. Wilkerson (Eds.), Reframing persistence research to improve academic success, New Directions for Institutional Researchers (30), pp. 9-24. New York: Wiley.