The value of grades

An interesting discussion of different ways to scale grades using algorithms over at Worthwhile Canadian Initiative.  The comments are at least as interesting as the post.  In the comments, the value of grades emerges as a central concern.

Grades have two main uses:

1. Social
2. Pedagogical

Both are important, but it's very clear that the main social use of grades--as a socially visible and efficacious mark which sets apart the "smart" kids from the not-so-smart ones--is predicated upon the sound functioning of the main pedagogical use--as a form of critical feedback that lets a student know relatively how s/he's performing. Grade inflation is driven by the democratization of the university and the concomitant rise of the social importance of grades. Since a university-level education is now regarded as a sine qua non for a decent job, and since universities use high-school grades as an admissions test (i.e., to make invidious distinctions between students), it's no surprise that enormous pressure is placed on teachers to give higher marks. Similarly, since university students now presume (rather unimaginatively, in my opinion) that a graduate degree is the key to social success, and since graduate schools look at university grades as an admissions test, university students correctly deduce that good grades are key to their social success. And since grades are presented with little context, enormous pressure can be brought to bear on teachers, since it hardly matters how a student gets good grades. The succesful wheedler can expect a level of social success (status and salary) equivalent to that of the class genius.

Students believe that this equation represents the world perfectly.  They are not entirely wrong.

With that in mind, the crucial context that makes it possible to put grades to any use is the relative expectations that presumably animate the teacher's instruction. I always think of this context as having three layers:

1. The student's performance vis-à-vis his/her peers in this particular course (during this semester, with these students)
2. The student's performance vis-à-vis other cohorts taking more of less the same course (i.e., compared to all students who've taken this course with me)
3. The student's performance vis-à-vis the universe of students who have ever taken, are now taking, or will take a course more or less equivalent to this one, in any institution and with any instructor.

Comparing students only among their immediate peers can give a false impression of their performance, since cohorts and classes can and do differ in relative strength. Some groups should skew higher or lower, because the groups are stronger or weaker than other groups. I find this kind of contextualizing to be very difficult, and while algorithms can be helpful as a method, they are no substitute for the judgment that decides which of them to use or whether to use them at all.

There are several universities in the US at which students are given feedback but no grades. Hampshire College, for example, provides no grades: at the end of each course, the teacher and the student both draft a 300-400 word narrative discussing the student's performance. These narratives form the body of the student's "transcript." These transcripts are made available to the entire student body as well as to other institutions at the student's request.  I took two courses at Hampshire, and I found the students to be engaged, engaging, and highly motivated. (Since I attended a different school, which did give grades, my teachers gave me a grade, but I can tell you that the narratives they wrote are far more precious to me. I still have them.)

Even though they still publish evaluations of each student's performance, such institutions obviously have a strong position on the social value of grades. An interesting thought experiment that really pushes the distinction between social and pedagogical uses to its limit is to imagine a university that gives grades, but does not publish them. Students are told what grades they have received, but the records are then destroyed, so that no one can "prove" anything. The students' transcripts are simply the lists of the courses they've taken. Anyone could say he'd gotten an A, but only he and his teacher know for sure, and no on can prove anything. Wouldn't teachers and students then simply regard the grades are a rather autistic and reductive form of feedback? Would giving grades be worth the trouble? Would teaching per se be easier or harder? Relative performance would still need to be graded, in the strict sense of the word, but since the social value of the grade has been eliminated, the only value left is its pedagogical value. So what IS the pedagogical value of a grade?

Kinds of businesses and kinds of business(wo)men

Paul Kedrosky discusses some of the consequences coming down the pipe for the VC industry.  Paul draws much of his inspiration from his friend, Bill Stensrud, who's a VC investor himself.  Thinking about the current state of the VC industry, Bill observes that an overweening interest in getting to an exit (read: finding a buyer) has come to replace an interest in cash flow (read: making money).

What [all VC] firms have in common is that they exist to buy and sell equities.  They both buy from entrepreneurs and they both sell to acquirers or (very infrequently these days) to public shareholders.  They are, at their very core, traders.  Their job is to buy low and sell high.  This fundamental truth about the venture business informs every action they take whether mainstream of super-angel.  It also informs the culture of the businesses they create.  Everyone is looking for a pot of gold at the end of the rainbow - the life changing - all consuming - EXIT!!  The nature of their business model demands it.  These are close-end funds.  They have to return money - cash - to their investors.

In this blog over the next several months I am going to explore another - even more ancient - model for company creation.  This is art and practice of building and running a business for POSITIVE CASH FLOW.  Before there was venture capital and before there were EXITS, people built businesses to make money so they could pay their bills.  I will argue that re-discovering this model drives a corporate culture which is much healthier, more robust and more survivable than the EXIT-focused culture created by the venture capital model.  I will also argue that the cash flow model can engage the employees, the critical human capital asset of every business, to significantly greater efficacy than equity models.  Lastly, I will argue that we can modestly scale this model to the point that it can become a significant factor in new business creation.

One of the consequences of thinking about business environments as ecologies is that it makes it relatively easy to think the relationship between the people who run businesses and the generic conditions in which those businesses operate.  It becomes easy to see how VCs, in actively selecting with an eye toward the exit, might over time change the population of entrepreneurs they partner with.  It might be obvious that business ideas that make money, but have no clear exit strategies, do not fare well securing VC in today's market.  But business ideas are developed by and instantiated by businesspeople.

Today's VC climate actively selects against entrepreneurs who want to "build and hold" profitable businesses.  "Build and hold" doesn't just describe a business model--it describes a temperament:

• Thoughtful - concerned with long-term secular trends rather than high-velocity volatile fads
• Prudent - husbands scarce resources for the long haul--including and especially managerial stamina (contemporary VC expects managerial burnout, though it hopes to exit before it happens)
• Patient - satisfied to build a strong foundation for big success by stringing together a long series of small, cumulative successes

If we want businesses that are conceived and constructed as long-term money-making ventures, we need entrepreneurs with the right temperament.  If modern VC's intensive focus on the exit has changed the character of our pool of important and interesting business ideas, well, so what?  New ideas are easy to come by.  But I fear there may have been a more subtle and more fundamental change in the character of our entrepreneurs (as a group, not as individuals).  We now have one, maybe two generations of top-tier entrepreneurs (with the right experience and connections) who think of starting a business as aiming for an exit.

In order to make best use of our limited resources, we have created a streamlined system whereby everyone must exit at Easy Street.

Our current business culture of get-what-you-can-while-you-can follows directly from the preferences of the VC investors who hire people with that kind of temperament to build and run their businesses.  As go new businesses, so goes all business.  It's hard to see how we get sensible businesspeople to run our businesses until investors stop thinking like traders.

No apologies for Aristotle

I'm aware that it would be pretty easy to get the impression, reading this blog, that Aristotle is the only philosopher I read or care about.  While this is grossly incorrect, I refuse to apologize for portraying the big A as the most important ethical philosopher in history.  He is.  While this is only my opinion, it is not my fault.

Social attitudes usually reflect social conditions (not moral preferences)

It turns out that personal finance isn't so "personal."  Much of it has to do with prevailing attitudes toward financial conditions.  Bank of America CEO Kenneth Lewis recently worried in public [WSJ; behind firewall] about people who might walk away from their mortgages now their homes are worth less than their loan balance.

Calculated Risk puts up some scary numbers, which indicate the potentially seismic consequences of large-scale shifts in social attitudes. How scary?  How does TWO TRILLION strike you?

If every upside down homeowner resorted to "jingle mail" (mailing the keys to the lender), the losses for the lenders could be staggering. Assuming a 15% total price decline, and a 50% average loss per mortgage, the losses for lenders and investors would be about $1 trillion. Assuming a 30% price decline, the losses would be over $2 trillion.

Not every upside down homeowner will use jingle mail, but if prices drop 30%, the losses for the lenders and investors might well be over $1 trillion (far in excess of the $70 to $80 billion in losses reported so far).

There's a huge social component to personal ethics--much larger than we usually suppose.  Is walking away from your mortgage bad behavior?  Well, what if not only makes financial sense, but many people are doing it?  What if it were recommended?  Shame depends on the visibility of certain personal behavior in the eyes of a public, and there seems to be much less shame associated with cutting one's mortgage losses.  The big lesson (apart from the shame I hope the banks are feeling) is that personal character is developed in symbiosis with social conditions, not in spite of them.

Living through the digital revolution

Clay Shirky says more or less everything that needs to be said about why newspapers are in trouble and what might replace them in the future. He concludes:

In craigslist’s gradual shift from ‘interesting if minor’ to ‘essential and transformative’, there is one possible answer to the question “If the old model is broken, what will work in its place?” The answer is: Nothing will work, but everything might. Now is the time for experiments, lots and lots of experiments, each of which will seem as minor at launch as craigslist did, as Wikipedia did, as octavo volumes did.

[...]

Society doesn’t need newspapers. What we need is journalism. For a century, the imperatives to strengthen journalism and to strengthen newspapers have been so tightly wound as to be indistinguishable. That’s been a fine accident to have, but when that accident stops, as it is stopping before our eyes, we’re going to need lots of other ways to strengthen journalism instead.

When we shift our attention from ’save newspapers’ to ’save society’, the imperative changes from ‘preserve the current institutions’ to ‘do whatever works.’ And what works today isn’t the same as what used to work.

[...]

For the next few decades, journalism will be made up of overlapping special cases. Many of these models will rely on amateurs as researchers and writers. Many of these models will rely on sponsorship or grants or endowments instead of revenues. Many of these models will rely on excitable 14 year olds distributing the results. Many of these models will fail. No one experiment is going to replace what we are now losing with the demise of news on paper, but over time, the collection of new experiments that do work might give us the reporting we need.

As Shirky so astutely points out, the economic reality of newspaper publishing is shifting beneath the feet of the industry. Probably today's newspapers can't dance nimbly enough to save themselves. And that shouldn't worry us very much. It's just that one crucial function of newspapers--namely, reporting--remains a pillar of democratic society. We don't need to care who does it, but we need to care passionately that it gets done.

The economists have no clothes

Over at The Atlantic, Gregory Clark admits, rather refreshingly, that academic economists have no clothes.

The current recession has revealed the weaknesses in the structures of modern capitalism. But it also revealed as useless the mathematical contortions of academic economics. There is no totemic power.

As a discipline, economics proposes models, which are by definition incomplete. That is, they exclude some details and highlight others. To think that economic theories actually describe reality--as opposed to offer an image of reality that is useful for some purposes--is to mistake the map for the world.

Ceci n'est pas le monde.

Further, the dismal science has all too often provided models whose validity is impossible to ascertain, since it has often built its theories on the basis of premises that are false prima facie. The point here is that, logically speaking, false premises do not yield false conclusions; rather, false premises render the truth values of an argument's conclusions indeterminate. It isn't that economic models are false, but rather that the falseness of their premises means that can know nothing with certainty about their conclusions. We can't say whether economic models are true, false, or some determinate mix of the two. Logically speaking, they're mere speculation, with the same logical status as wishful thinking.

You mean we came all this way and we don't even know if we're wrong?

For example, the theories of classical economics generally accept as axioms (i.e., they accept as true without argument) the following:

1. All economic actors are rational.
2. All economic actors have perfect information about the markets in which they act.
3. All resources are scarce.

These are bad axioms, since they're obviously not true. As in, there's no doubt at all that these are false. Of course, not all branches of modern economics still accept these premises without qualification, but historically speaking these assumptions lie at the foundation of all economic thought. This is precisely the main reason I never studied economics in college. Who can take seriously a discipline that, wherever it ends up, begins with nonsense? While there's no doubt that the phenomena we think of as economic are intrinsically interesting, I remain skeptical that the formal discipline of economics has a great deal to offer beyond the obvious. As Clark notes:

The debate about the bank bailout, and the stimulus package, has all revolved around issues that are entirely at the level of Econ 1. What is the multiplier from government spending? Does government spending crowd out private spending? How quickly can you increase government spending? If you got a A in college in Econ 1 you are an expert in this debate: fully an equal of Summers and Geithner.

Common sense cloaked in jargon and equations. Even the economists' invisible clothes look rather shabby these days.

The value of inconvenience

By almost any meaningful performance measurement, the U.S. economy in the 20th century performed better under Democratic administrations than under Republican ones. Reflecting on this seeming paradox, Christopher Carroll suggests that

perhaps the best explanation has to do with attitudes, not doctrines: Maybe capitalism works better when its excesses are restrained by skeptics than when true-believers are writing, interpreting, judging, and executing the rules of the game. (The Democrats are surely the more skeptical of our two parties).

Most would agree that restraining the excesses of almost anything counts as good sense, but this is only a preliminary step toward a bigger and more interesting idea:

Capitalism works better when it is being held accountable to some external standard than when left to its own devices.

The whole system works better when "held accountable to some external standard," when it is, in a word, constrained. Optimal performance, in other words, is the fruit of struggle. Make things too easy and performance declines.

Relaxed external standards? Check. Highlight reel material? Not so much.

Consider how effective coaches pull outstanding athletic performance from their players. Good coaches don't let their players do whatever they want, without accountability or oversight; they create rules and systems of accountability. A good soccer coach makes you use your weak foot in order to develop it. A good swimming coach pushes you to hold your breath longer. Optimal athletic performance depends upon the measured application of psychological and physiological pressure. (Go watch Gavin O'Connor's Miracle to see a dramatization of great coaching.) Good coaches don't remove limitations--they use them.

Or consider architectural and industrial design. The famous designer Charles Eames (yet another famous St. Louisan) once remarked:

Design depends largely on constraints.

We tend to believe that creativity is best served by removing constraints. If we could just somehow make the process of invention easier for the inventor, we imagine that she would be more inventive. But the opposite is usually true. People get creative--truly creative--when challenged to negotiate constraints. Budgets (within reason) push architects to develop new strategies to solve old problems. (The story of how the design of Seattle's new public library building developed is a great example.) The particularities of manufacturing processes push industrial designers to find solutions which challenge convention. (The story of the how the first commercially viable computer mouse was designed is a textbook example.) Constraints drive creativity.

Artist Andy Goldsworthy creates astonishing ephemeral works using only the materials he finds on site during his wilderness hikes: creativity driven by constraint.

The same is true in business. Real innovation happens where someone discovers a new way to scratch an old itch, where someone thinks through a problem in a new way (even if it's simply a new application of an old technology). When we talk about innovation happening "at the edges" of a market or industry, what we mean is that innovation happens where business rubs up against constraints. (The "mainstream" of anything is where things flow smoothly, right?) We can't have innovation--and capitalism's greatest strength as an economic system is its powerful incentives for innovation--unless we have the right kind of rules and restrictions. (The question of "more" regulation versus "less" regulation is puerile. The kind of regulation matters more than the amount.) The free market, to put it pointedly, is only as free as its constraints force it to be.

Of course, constraints are damned inconvenient. And that's precisely the point. It's often--if not always--in response to inconveniences that people are most creative, most inventive, most innovative. And so we're led inevitably to the conclusion that inconveniences can be useful.

"Traffic lights are just the Man keeping us down! We will not be constrained!"

Some inconveniences, naturally, are more useful than others, but that hardly obviates the necessity of inconvenience for optimal performance. It's easy enough to see how inconveniencing others might be worthwhile, but it's one of the marks of emotional maturity to see the value of inconvenience for oneself. Politics--in the largest possible sense of the word--is only possible because we deliberately accept to be inconvenienced in certain ways (e.g., we don't simply use whichever car is closest, use guns to force our crushes to go out with us, or lynch elected officials from opposing parties). We recognize that our condition is collectively better when we all accept to be inconvenienced in certain ways. (Again, the kind of self-regulation matters more than the amount.)

Finding--and enforcing--the right kind of constraints is key to getting the most out of people, as innovators, as politicians, as artists, as designers, and even as citizens. We would all of us do well to remember that inconvenience--yes, even our own--often serves us much, much better than convenience.