Improving the ability to set and assess learning outcomes for students has been a challenging task and not one that has been well documented or executed. This article presents a protocol for the development of a content- and skills-based management curriculum that is based on the Malcolm Baldrige National Performance Criteria and the U.S. Department of Labor's O*NET database as a means to define learning outcomes and to assess skills-based performance measures. The O*NET database has career-related data for 965 occupations. Because the data are collected from thousands of job incumbents in a representative sample of the U.S. labor force, this database can be used to help determine the desired learning outcomes of employers. In this article we first look at existing means to evaluate the quality of an academic management program through the popular press ranking criteria and the Association to Advance Collegiate Schools of Business (AACSB) standards. Then the Malcolm Baldrige National Quality Program Criteria and Balanced Scorecard will be applied to the establishment of standards of performance as an alternative way of measuring learning. Next, the U.S. Department of Labor's O*NET database will be used to demonstrate how to define specific needed skills for an MBA or undergraduate management program. Bringing together the elements of the Baldrige Award criteria and O*NET database will help focus management education into a learning outcome mentality that better meets AACSB accreditation standards and managerial job requirements.
Learning outcomes, assurance of learning, O*NET, Baldrige, Balanced Scorecard
The Association to Advance Collegiate Schools of Business (AACSB) and other accreditation associations are slowly focusing on the need to assess outcomes in management education. However, these attempts seem to lack the comprehensive approach that is needed to focus on the total development of the student. This is critical in management education. Not only does management education have the content dimensions of the discipline, there is the additional behavioral skill building responsibility that is not prevalent in some other business school disciplines. Failure to define the key measurable outcomes of management education and to translate them into an aligned set of learning goals will make it more difficult to meet the needs of the student and future employers in preparing the student for the necessary content and skill development needed to meet the challenges of a highly competitive workforce.
The Malcolm Baldrige National Quality Program Criteria and the Balanced Scorecard will be used to develop aligned learning metrics for assessing course and curriculum outcomes of a management education program. In addition, the U.S. Department of Labor's O*NET database will be used to help define specific knowledge and skill sets needed for MBA-level leadership skills and for undergraduate general management skills. Examples will be given to show how the O*NET database can be used to find needed skills. In addition, as student involvement is critical in the learning process, ways to assess student engagement will be reviewed. Together a more integrated set of measures can be developed to assess content and skill learning and to assess student satisfaction. The methodology proposed can support a more defined learning effort through clear learning and student engagement options.
There have been a number of calls to improve the quality of graduate and undergraduate management programs. Chia and Holt (2008) raise concerns that some of the current approaches to learning focus too much on abstract causal explanations over providing skills and practical applications. They call for "knowledge by exemplification" as an approach by faculty that is demonstrative, creative, and "performative" (sic). Students need to learn through examples and the approaches used by the faculty that become more directly involved in the process rather than abstract theoretical lectures.
Rynes, Lawson, and Ilies (2003) raise a similar issue over the content of management education. Their concern focuses on the overstressing of technical knowledge and skills at the expense of important conceptual and behavioral knowledge and skills. The need to balance the technical and behavioral is essential for meeting recruiters' needs to find candidates that have the behavioral skills required to effectively work in groups.
Pfeffer and Fong (2004) raise a different sort of concern related to the market focus and lack of professional standards and values in U.S. business schools. Sadler-Smith and Shefy (2007) share similar concerns, although they focus on current business schools' disregard for intuitive awareness. They propose a protocol for developing and evaluating intuitive skills as a means to make business programs more relevant.
Whetten (2007) focuses on designing courses to improve the relevancy and value to students. More recently, Rubin and Dierdorff (2009) raise similar concerns of relevancy of MBA programs, that knowledge acquired may not be what is required for desired careers. It is to these concerns that we apply the Baldrige Approach and O*NET database as a means to better define and assess program needs and how one might use these concepts to improve their own course design. First, we will track current program assessment approaches.
Current Program Assessment Approaches
The AACSB and other accreditation associations are slowly focusing on the need to assess outcomes in management education. In addition, the popular press has created processes to assess business schools. Each offers a different perspective on how to evaluate the quality of business schools' outputs.
Popular Press Rankings
U.S. News & World Report ("Best Colleges 2009: Undergraduate Business Programs," 2008; "Best Graduate Schools: Guide to Business Schools," 2008) defines business school goals in terms of satisfying administrators, faculty, students, and employers. For full-time MBA programs, the goals are student selectivity (25% of a program's total score), student job placement (35%), and program quality as assessed by business school administrators (25%) and recruiters (15%). The part-time MBA program rankings are totally determined by a survey of business school administrators. Undergraduate business school rankings are determined by surveys of administrators and senior faculty.
BusinessWeek ("Best Undergrad B-Schools," 2008; "Business School Rankings & Profiles," 2008) defines business school goals from the perspectives of students and employers. MBA program rankings are based on student surveys (45% of the total ranking), recruiter surveys (45%), and faculty members' intellectual capital (10%). Undergraduate business program rankings are also based on student surveys (30%) and recruiter surveys (20%), but they include additional student-oriented data: median starting salaries (10%), number of graduates admitted to the top 35 MBA programs (10%), and academic quality (30%), which includes average SAT/ACT, faculty/student ratio, average class size, percentage of students with internships, and number of hours per week students spend studying. Some of the evaluation data are more objective, but much of the rankings are based on attitude survey data that are less stable measures of learning.
As flawed as they may be, these rankings are important. Potential students may use the rankings to help them decide where to apply for admission. If they are admitted to more than one school, they may use them to decide which one to attend (Clarke, 2006). Faculty members may prefer to teach at higher ranked schools. Donors may be interested in giving money to well-ranked schools. If a school falls in the rankings, the board of trustees may call for someone's head. Still, the rankings leave much to be desired (Adler & Harzing, 2009).
We argue that one of the biggest flaws in these rankings is that they do not help provide guidance on how to improve student learning. They do not help faculty members determine what should be in a business school curriculum, and they do not help us determine what should be in particular courses.
This article advocates a method to help faculty members follow a process for developing a curriculum and courses in that curriculum. We suggest the use of a systematic method that will tie business school curricula and management courses to what important stakeholders (students and employers) say is important on the job. This method gives faculty members enough guidance to help set learning goals but leaves them enough flexibility to address various business school missions and various faculty interests. AACSB's curriculum standards include some of that guidance.
AACSB International's Curriculum Standards
The AACSB is a stakeholder for many business schools. Its standards for accreditation can be viewed as a set of criteria that define the quality of business education. The school must practice strategic management; it must engage its staff, faculty, and students; and it must assure that learning occurs.
The strategic management standards say that a school must develop a mission statement enunciating its long-term goals, one of which must be the production of intellectual contributions. The long-term goals must be supported by short-term goals and action plans to accomplish them. There should be an alignment of university, college, and course goals.
The participant engagement standards address staff, faculty, and students. There must be sufficient numbers of staff members and faculty members to fulfill the school's mission. Faculty members must maintain their academic or professional qualifications. Faculty, staff, and students must work together to perform various educational processes such as setting high expectations for learning, assuring student-faculty interactions, and evaluating the effectiveness of instruction.