Q&A with Professor Gerard P. Cachon, Fred R. Sullivan Professor of Operations & Information Management

 

We’ve heard that there are going to be significant changes to the manner in which Wharton allocates courses to students in the next few years, can you sum up for us what these changes will be?

 

Instead of using an auction with multiple rounds of bidding, we are considering a system in which students report their preferences for courses during a single round. Unlike with the current system, with the new system a student only needs to focus on her own preferences – there is no need (or value) to think about the preferences or actions of other students. As a result, it is in the best interest for a student to report her own true preferences for courses. With this new system every student receives the best course schedule the student can afford, that is, the schedule that maximizes her reported total value given her available budget. To imagine how the new system works, suppose with the auction you could choose your courses after observing all of the clearing prices. In that ideal situation (i) you don’t have to worry about how other people will bid (because you can already see the clearing prices), and (ii) you would never bid more than or just below the clearing price. In other words, you would bid exactly the clearing price for your most preferred set of courses given your budget. The new system operates like this imaginary auction system. The system then buys you your best schedule based on the preferences you report because the system knows your course preferences and their clearing prices. It knows the clearing prices because it solves for the clearing prices using sophisticated optimization mathematics that were only recently developed.

 

What stage are you in the process of designing the new course allocation system? Do you foresee making any more major changes or is the design basically complete at this point?

 

We have many of the high level design issues worked out, but there are a number of additional, and important, details that need to be finalized. What will the student user interface look like and what functionality will it have? How will budgets be assigned and how will the system evolve over multiple semesters? How will waivers be treated? How will students make changes to their schedule once the semester starts? Et cetera. So, nothing has been set in stone and a number of elements are still in the early discussion stage.

 

When are these changes supposed to go into effect? Which Wharton class will be the first to use the new course allocation system?

 

We are targeting Fall 2013 as the first semester to implement the new system. Hence, MBA15 would be the first class to use the new system throughout their time at Wharton and MBA14 would use the system in their 2nd year. We will explain this transition in detail to the MBA14 class even before they start bidding for their first year classes using the auction so they are fully aware of how the transition to the new course allocation system will be handled.

 

What brought about these changes? Has an overhaul of the course allocation system been in the works for a while or did something specific spur the initiative?

 

A number of events have led us to this point. For example, with the new curriculum, students will choose a larger fraction of their total set of courses. Hence, this creates additional incentive to make sure that our course allocation system is as good as it can be. As already mentioned, the technology behind the new system was only recently developed, so this new system wasn’t really an option even two to three years ago. Finally, overall satisfaction with the auction, based on stakeholder surveys, is not as high as we would like – which motivated us to consider alternatives that may improve overall student satisfaction.

 

Who are the designers of the new system? That is, who has had the strongest hand in developing the changes in terms of students, faculty and staff?

 

It has been a team effort. Everyone at the school has a stake in making sure that our course allocation process is as good as it can be. As a result, from the beginning of our initiative we have had active participation from students, faculty and staff on our committee. For example, our two student representatives, Kathryn Scarborough (WG ‘12) and Paul Nolen (WG ‘12), have provided critical advice on the student’s perspective as users of the course allocation system. Next, we were lucky to have the support of more than 130 students who participated in our beta tests. On the faculty side, besides the faculty serving on the committee, we have received feedback from the Dean’s Advisory Council, Department Chairs as well as other faculty groups. The MBA program office and the computer developers from WCIT have also been actively guiding our efforts.

 

We heard that there was some beta testing done with the proposed course allocation system, what were the results of those tests (i.e. did students fair better or worse than with today’s auction system)?

 

We ran eleven sessions in which current students participated. Students were told to pretend that they were choosing classes for the spring semester of their second year from a set of 25 actual courses. They did so with two different systems, one that closely resembles our current auction system and the other that resembles our proposed new system. In all of the sessions, the majority of students preferred their schedule with the new system over their schedule from the auction. There was also considerably less envy with the new system – envy occurs when student A prefers student B’s schedule over the one she received. Envy is a useful important measure of fairness and we were encouraged to see that the new system scored well on this dimension. (We measured envy by literally asking students to compare pairs of schedules, one of which was the schedule they received and the other was a schedule another student received in the same session. Each student made up to 19 of these schedule comparisons.) Finally, while we have observed a gender gap with the Auction (women like it less than men), there was no gender gap with the new system (women and men liked it equally).

 

What were student comments after the beta testing?

 

Students thought the new system was “quite user friendly” and “equitable”. Furthermore, they liked not having to think strategically: “It was much easier to think about what I wanted than what I would need to bid in addition to what I wanted”. But learning a new system that is substantially different than an auction left some students with the impression (understandably) that it was a “black box”. From the set of comments we received we developed some good ideas for improving the user interface. In addition, it is clear to us that we will have to invest in a considerable amount of support and training. Once we do so, we believe we will be able to substantially reduce (and hopefully nearly eliminate) the “black box” concerns.

 

Have there been significant changes since the beta testing?

 

No. We have been pouring over the data and we think we have determined several changes that could improve the system further, but nothing that fundamentally changes the design.

 

What has been the response of the faculty to the new course allocation system?

 

Remarkably positive – faculty like to debate but the overall reaction has been uniformly favorable. There is strong theory supporting the new method (that is, it is not “ad hoc”) and the beta testing gave us very encouraging data. Business school professors love the combination of sound theory and supportive data.

 

What were the most controversial points of the system and how did you work around them within the design team?

 

There really haven’t been any “controversial” points that have bogged us down or led to difficult disputes. From the beginning we have all wanted to see if we can design a system that would lead to better course allocations and higher student satisfaction. When there has been a debate, we have looked to find theory and/or data, but especially data, to resolve the debate. Our sources of data included the beta test as well as our own actual course auctions. There is no question that good data makes it easy to resolve issues.

 

We’ve heard that the new course allocation system will be able to provide data in terms of what classes students are signing up for, and how much they value it, etc. that the auction system is not able to provide. Can you explain a bit more about this and how you foresee this impacting the courses that Wharton itself provides?

 

This is potentially one of the most exciting features of the new system. While it seems that prices in the course auction system provide good signals of what students value, we discovered that they provide only weak signals. The main reason for this is that students should bid strategically in the auction. For example, suppose you really like course A and while you are willing to take course B, it will not be the cornerstone of your education. Furthermore, suppose course A has a fantastic professor but the topic doesn’t have a broad appeal, so there is a limited number of students who want to take the course. Hence, you expect that it will clear for a low price. In contrast, course B, has historically cleared at a high price (maybe because its class size is limited to 36 seats). If your bids corresponded to your preferences, you would put most of your points on course A and a smaller number on course B. But most students rationally realize that they should bid the majority of their points for course B even though they prefer course A. So the auction doesn’t really reveal to us true preferences.

 

In contrast, the new system does reveal true preferences. And that will allow the school to learn much more about which courses are truly desired. We can begin by asking ourselves simple questions like – if we could add a few seats to course A, will we be able to improve student satisfaction? This could influence how we assign rooms to courses – which courses get the larger rooms, which the median size rooms and which courses are best suited for seminar style rooms. With the new system this can be done just after we learn student preferences because doing so doesn’t influence how students would have reported their preferences. In contrast, which the current auction, expanding a course’s capacity at the end of the auction may create disappointment with some students as they bid and paid a high price under the pretense that the course would have limited capacity.

 

 

Are there any other major impacts you foresee the new course allocation system having on students/faculty/etc.?  For example, it definitely seems like it will save students time.

 

Our primary goal is to ensure that we allocate courses as effectively as possible (i.e., the students who want a course the most are the ones that get to take that course) and as fairly as possible (e.g., to minimize the amount of envy). But as you mention, time is also an important consideration. We believe that the new system may reduce the amount of time a student must spend on course allocation for several reasons. For one, one round will tend to occupy less time than 8 or more rounds. Next, because with the new system a student doesn’t have to worry about how other students behave, there is no need to look up historical clearing prices or to try to figure out a bidding strategy. Instead, students should devote their time to understanding their own preferences – which courses do they want to take and how strongly do they value those courses relative to others. So a student’s time should be spent on learning about the courses, such as looking up syllabi, viewing sample videos of classes and talking to people about their experiences. Overall, we hope that students will be able to spend less time and the time they spend will be directed towards a more productive use.