Simon School of Business

Full-Time MBA

Life at Simon
The Simon Advantage
Academics
Careers
Scholarships & Tuition
Admissions
ContactSimon Profile Login

Assessment of Learning

5. Assurance of Learning

I. Summary

In this section, we summarize the Simon School Assessment of Learning (AOL) activities to date and outline the plans for the next two academic years.

The Simon School completed its first full year of AOL activity in 2010. We started by examining our full-time MBA and core courses were initially singled out for application of a consistent five-step AACSB process. The results to date are:

  • The spirit of cooperation and seriousness of purpose across core faculty was notable. The AOL process was followed rigorously and the results documented.
  • The course AOL evaluations yielded few surprises but identified some opportunities for continuous improvement. Among other things, faculty expectations are higher than the student outcomes.
  • The involved faculty all noted an increased attention to the alignment of their core course with program objectives. Various faculty meetings, town hall discussions, and AOL documentation offered vehicles to share information about how to improve student outcomes. Increased experiential learning and student participation have been particular areas of focus.
  • Overall, Simon faculty have become aware of AOL and adopted a reasonable level of acceptance. There is a growing awareness that this is not an exercise but an integral component of the way we do business.
  • The MBA study plan has been significantly revised as of the 2012-2013 academic year. The change was motivated by the desire to improve student employment outcomes and is consistent with AOL objectives.
  • An AOL evaluation of the most popular MBA electives will be conducted during the 2012-2013 year.

II. Learning Goals for the MBA Program

Table H: Learning Goals for the MBA Program

Upon graduation, our students will be able to:

Learning Goals

Learning Objectives

Core Courses

Apply knowledge based on a rigorous research-based, economics-focused core curriculum

(a)Explain and apply key concepts, tools, and theories in Managerial Economics.

STR 401

(b)Explain and apply key concepts, tools, and theories in Corporate Financial Accounting.

ACC 401

(c)Explain and apply key concepts, tools, and theories in Capital Budgeting and Corporate Finance.

FIN 402

(d)Explain and apply key concepts, tools, and theories in Marketing Management.

MKT 402

(e)Explain and apply key concepts, tools, and theories in Operations Management.

OMG 402

(f)Explain and apply key concepts, tools, and theories in Computers & Information Systems.

CIS 401

Execute decisions, and design organizational processes and structures, based on the economics of organizations and markets

(a)Predict how people respond to incentives both within organizations and in the marketplace.

MKT 402, ACC 401, STR 401, STR 403

(b)Design effective systems of performance measurement for people and teams.

STR 403, CIS 401

(c)Determine how to allocate tasks and decision rights to people and teams in different situations, based on their expertise and their contribution to the larger task at hand.

STR 403, CIS 401

(d)Determine the implications of a firm’s market environment on organizational structure and market decisions.

STR 403, CIS 401, FIN 401, STR 401

Frame complex decision problems in new and unfamiliar circumstances through a conceptual understanding of relevant disciplines

(a) Proactively monitor the business environment to identify new opportunities or the need to respond to changes.

MKT 402, CIS 401

(b)Develop hypotheses about the causes of problems; e.g. is our falling stock price related to financing decisions or a strategic problem? Is a knowledge management problem IT-related or organizational?

ACC 401

(c)Build on the frameworks and theories relevant to a decision problem to develop more detailed questions that guide the analysis.

ACC 401, CIS 401, FIN 401

(d)Identify objectives, decisions, constraints, and parameters in a business problem.

STR 401, CIS 401, GBA 411, MKT 402, FIN 401

(e)Collect and organize the cogent facts, qualitative and quantitative, required to investigate the nature of a decision problem, or to investigate specific hypotheses about causes.

GBA 411, ACC 401, CIS 401

Make decisions based on available data and by thorough qualitative and quantitative analysis

(a) Recognize the degree of uncertainty in the analysis and learn how to cope with it.

GBA 411, FIN 401

(b)Apply qualitative and quantitative analytical techniques and tools to distinguish between competing hypotheses about a decision problem.

ACC 401, GBA 411, CIS 401

(c)Develop possible solutions to a decision problem, including details of implementation.

MKT 402, GBA 411, CIS 401, FIN 401

(d)Apply qualitative and quantitative analytical techniques and tools to evaluate different solutions.

STR 401, MKT 402, GBA 411, STR 401

Communicate effectively in different business situations

(a) Effectively communicate (to superiors, clients or other parties) a recommended course of action and what considerations lead to the recommendation.

MKT 402, MGC 401-403

(b)Effectively take and defend a position, both in presentation and debate situations.

MGC 401-403

(c)Effectively present an analysis or argument in writing.

MGC 401-403

III. Assessment Procedure and Sample Result

Individual course goals were developed by the relevant faculty. Learning goals pertaining to exam questions were identified and rubrics for adequate learning were established so as to assess whether the learning goals were met.

A sample Learning Assessment Table for a particular course, GBA411 Framing and Analyzing Business Problems I, is shown below.

Table I: Learning Assessment Table for GBA 411

Learning Goals & Objectives

Learning Objective

Metric: Exam Question, Project, Assignment Identification

Rubric: Adequate Learning Score

Percentage of Class at or above

3.(d)

Identify objectives, decisions, constraints, and parameters in a business problem.

Q2b

Q3b

10/15

10/15

23%

39%

3.(e)

Collect and organize the cogent facts, qualitative and quantitative, required to investigate the nature of a decision problem, or to investigate specific hypotheses about causes.

Q2a

Q3a

10/15

10/15

85%

50%

4.(a)

Recognize the degree of uncertainty in the analysis and know how to cope with it in the analysis.

Q3a

Q3b

Q3c

10/15

10/15

4/10

50%

39%

18%

4.(b)

Apply qualitative and quantitative analytical techniques and tools to distinguish between competing hypotheses about a decision problem.

Q1a

Q1b

10/15

10/15

67%

73%

4.(c)

Develop possible solutions to a decision problem, including details of implementation.

Q2All

Q3All

22/30

25/40

68%

32%

4.(d)

Apply qualitative and quantitative analytical techniques and tools to evaluate different solutions.

Q2b

Q3b

10/15

10/15

23%

39%

After completing the AOL evaluation the core faculty were asked to reflect on the AOL process and offer their comments. Below is a sample of a comment (unedited) received from Prof. Edieal Pinker who taught the GBA411 section that participated in the AOL process:

GBA411 - The AOL process gave me a structure to organize the goals for the course. These are to develop the students’ abilities to: 1) frame a business problem and organize the relevant information in a way that is conducive to developing a spreadsheet model; 2) create a workable prototype of a spreadsheet model and validate it; 3) model uncertainty; 4) infer conclusions from sample data; and 5) optimize. As a result, I have been able to ask myself in which areas is the course more or less successful. In my opinion, objective (1) is an area in which students continue to struggle, for good reason, and thus I have sought ways to strengthen this element of the course. What makes it difficult is that it is not a distinct subject but rather a skill that is developed throughout the course. Therefore, to do better on this objective requires re-evaluating all the activities in the course and asking what can be changed in the way they are done to strengthen their contribution to objective (1). I have taken a first pass at this by taking many of the in-class examples done throughout the course and modifying them to have somewhat less structure and information so the starting point is vaguer. This enables the class to have some more discussion of the framing of the problem before diving into solving it with some technique. This has led to a reallocation of class time that has posed some challenges and is very much a work in progress. At a higher level, the AOL process terminology of “assessment” and “learning” begs the question of how to balance these two activities. Examinations are clearly for assessing what the students have learned. But what is the purpose of homework? I have taken the position that homework is to be used primarily for learning and not for assessment or evaluation of the students. Therefore, I do not grade homework rigorously but do assign a lot of challenging work to give motivated students the opportunity to learn by doing. What about class participation?; We want the students to get practice speaking and reasoning but should this be graded?; If they are practicing a skill to develop it then it is learning. If we are grading them on it then it is assessment. It seems to me that the grading is there as a motivational factor to encourage participation but this is not really appropriate for true assessment. That would require, at the end of the quarter, to give an oral exam which is not practical. This leads me to think that there are significant things that we are trying to teach in our courses that we are not in a position to fully assess and that the AOL process has many gaps that do not have obvious solutions.

The AOL effort and the ongoing quality improvement process that Simon employs in its programs resulted in a number of curricular changes that are described below.

IV. MBA Program Changes

A. Career-based Sequences of Core Courses

Two new sequences of core courses were developed to allow the students to take courses relevant to their internships earlier in their first year study plans. These sequences were based on the areas of concentrations by students, functional area of internships, and the commonly chosen electives. This helped students better prepare for securing internships and successful performance. Details of this change are provided in Appendix H.

B. Changes in the Fall 2011 Core Courses

As a result of AOL, specific changes were made to our efforts to improve experiential learning and our students’ communication skills as of fall 2011 including:

  • The use of clickers to stimulate full participation from all students in a class.
  • Encourage creative thinking through the “Think-Pair-Share” classroom framework which encouraged small group creative problem solving and class discussion.
  • The modification of labs to emphasize the application of theory.
  • Workshops led by second-year students to emphasize small group collaborative problem solving.
  • Communication Learning Center workshops were developed to provide extra help for some international students.
  • The core management communication and strategy courses were re-engineered and better connected with the objective of making the covered topics more relevant to current management issues. This experiment was partially successful but further modifications are required.

C. New Academic Calendar and Other AOL Driven Changes

Using recruiter feedback and AOL evaluations, the faculty and the administration conducted an extensive evaluation of the proposal to start the fall term earlier and decided that this change would:

  • Create appropriate learning outcomes earlier in the fall recruiting season.
  • Allow students to discuss unstructured business problems earlier in the curriculum.
  • Increase the experiential content of the courses and encouraging more classroom participation in the MBA classes.
  • Develop a new academic calendar that moves up the start of courses so that our students have a comparable number of weeks of instruction to what is provided by peer schools.

The changed study plan is being introduced this academic year. Further details regarding the curricular changes and some of the discussion leading up to the change are contained in Appendix H.

This calendar change is a novel application of AOL principles. It is based on employer feedback. The recommended course of action was primarily a change in our first-year fall curriculum and calendar. The entire faculty was engaged in reviewing and approving the change. Feedback leading to continuous improvement can be expected after the change is implemented this fall.

Many of the core courses have been redesigned for the new curriculum. This redesign provides an excellent opportunity for the faculty to incorporate their findings from our initial AOL evaluations.

V. FIN 413 - Corporate Finance

As evidence of continuing activity and Assessment of Learning (AOL), the following is our latest assessment report provided in Spring 2013 on an MBA elective, Finance 413.

Course description: This course provides an intensive analysis of the effects of various corporate financial policy decisions on the value of the firm, including a discussion of the effects of taxes, bankruptcy costs and agency costs on these decisions. It then examines the interrelation of financing policy with executive compensation, hedging, and payout policies. The course provides an understanding of the theoretical issues involved in the choice of these policies.

Key concepts

  1. Modigliani and Miller (M&M) Theorem
  2. Debt tax shield
  3. Bankruptcy costs, direct and indirect
  4. Debt overhang
  5. Asset substitution
  6. Restructuring
  7. DIP financing
  8. Debt covenants
  9. Manager-shareholder conflicts
  10. Corporate governance
  11. Empire building
  12. Free cash flow problems
  13. Incentive compensation
  14. Information asymmetry
  15. Signaling
  16. Adverse selection
  17. Payout policy
  18. Risk management

Students completing FIN 413 should be able to:

Learning Goals

Learning Objectives

Related program goal

1. Apply the Modigliani and Miller (M&M) Theorem to analyze corporate financial policies

(a)explain the economic intuition of the M&M Theorem,

1c

(b)state the M&M Theorem in its contra-positive form,

1c

(c)apply the M&M to analyze the effect of corporate financial policies on firm value.

1c

2. Analyze and compute the tax benefits of debt

(a)Identify the source of debt tax savings, 

1c

(b)apply the adjusted present value method to compute the present value of the debt tax shield,

1c

(c)analyze the effect of investor taxes on the tax benefits of debt.

1c

3. Analyze the effect of debt-equity conflicts on firm value

(a) explain bankruptcy costs, distinguish direct and indirect costs, and identify specific bankruptcy costs,

1c

(b) identify the debt-equity conflict in debt overhang and analyze its effect on firm value, 

1c

(c) identify the debt-equity conflict in asset substitution and analyze its effect on firm value, 

1c

(d) analyze the debt-equity conflict under Chapter 11 restructuring,

1c

(e) describe DIP financing, debt covenants, and other mechanisms, and explain how they mitigate debt-equity conflicts.

1c

4. Analyze the effect of manager-shareholder conflicts on firm value

(a) identify and explain specific manager-shareholder conflicts,

1c

(b) describe corporate governance and analyze how it mitigates manager-shareholder conflicts,

1c

(c) identify the manager-shareholder conflicts in empire building and free cash flow problems, and analyze how debt mitigates the problems,

1c

(d) analyze the tradeoff between incentive and risk-bearing in executive compensation.

1c

5. Analyze the effect of information asymmetry on firm value 

(a) explain how information asymmetry arises between managers and investors,

1c

(b) identify information asymmetry in signaling and analyze the effect of debt as a signal on firm value,

1c

(c) identify information asymmetry in adverse selection and analyze the effect of equity issuance on firm value.

1c

6. Analyze the effects of taxes, agency costs, and information asymmetry on the payout policy and risk management

(a) analyze the effect of taxes,

1c

(b) analyze the effect of agency costs,

1c

(c) analyze the effect of information asymmetry.

1c

     

Learning Goals & Objectives

Learning Objective

Metric: Exam Question, Project, Assignment Identification

Rubric: Adequate Learning Score

Percentage of Class at or above

1.(c)

Explain and apply key concepts, tools, and theories in Capital Budgeting and Corporate Finance.

Exam
Midterm Question 1

Final Question 4


20/25

23/30


78%

79%

2.(a)

Predict how people respond to incentives both within organizations and in the marketplace.

Midterm Exam
Question 4


14/20


57%

3.(d)

Identify objectives, decisions, constraints, parameters in a business problem.

Case: Bankruptcy and Restructuring at Marvel Ent. Group

85/100

79%

4.(a)

Recognize the degree of uncertainty in the analysis and know how to cope with it in the analysis.

Problem Set 3
Question 1 and 2

85/100

86%

4.(d)

Apply qualitative and quantitative analytical techniques and tools to evaluate different solutions.

Case: Arundel Partners - The Sequel Project

80/100

86%

5.(a)

Effectively communicate (to superiors, clients or other parties) a recommended course of action and what considerations lead to the recommendation.

Case: American Barrick Resources Corp.

80/100

79%

FIN 413 – The AOL process pushed me to think in depth about the learning objectives of the class as well as the tools in place to attain these objectives. The course is composed of several main topics. Every topic has explicitly stated learning objectives. Also, I relate every individual topic objectives to the rest of the course objectives. The goal is to help the students understand how the topics are inter-related and provide them with an integrated understanding. To attain the course learning objectives, I rely on class discussions, homework, and exams. I encourage class participation and regularly call on students. This helps me assess their level of understanding on a continued basis. Also, I give in-class group problems that students have to solve on the spot. This helps me assess where students stand. This active learning exercise also serves students as a reality check. The homework is composed of problem sets as well as case studies. The problem sets are typically a bit more challenging than examples discussed in class and help the students practice the concepts introduced in class. The case studies provide the opportunity to apply these concepts to real life situations. Both problem sets and case studies are built so that to help students reach the learning objectives of the class. However, problem sets are tilted more towards assessment, while case studies more towards learning of these objectives. The grading of the homework takes into account this aspect. If class size permits, the students present the cases. Student presentations are followed by class discussions. These discussions represent yet another way for assessing the learning of students but from an alternative perspective. First, presenters are challenged by the students. Second, the quality of the discussion is revealing for assessing if learning objective are reached. Finally, the exams are meant to measure student performance and assess learning objectives attainment.

Prof Boris Nikolov

VI. Future Plans

Preliminary plans for the next four years have been developed. Future activities will build on our progress to date. During the next four years we will:

  • Evaluate the core courses in the new curriculum. We plan to review the MBA courses on a three-year cycle.
  • A programmatic review will be conducted to assess the variation in learning outcomes across the MBA program. The review will assess how well the MBA program is meeting its objectives and identify areas for improvement. Outside information from alumni, employers, and other schools combined with the MBA course evaluations will be used to make an overall assessment. This will be the benchmark for future AOL evaluations.
  • The AOL process will be expanded to include our MS programs. Many of these programs require several core MBA courses, so we anticipate the AOL process can move more quickly for these programs.
  • The AOL process will also evaluate the EMBA program. While the courses are unique, they have many common elements with our full-time MBA program.
  • The PhD program review will be scheduled after 2015.