Philosophical Meditations on Outcomes and Assessment

Jeffrey Paris
Assistant Professor of Philosophy
University of San Francisco
Email:  paris@usfca.edu

Part I: Meditations on Outcomes and Assessment

      As I complete my third year as a professional in educational facilitation and information transference for a comprehensive state university in the largest advanced public education system in the country, I turn my sights to a promising gauge of success—the assessment of students and the production of outcomes.  Have I, a philosopher, effectively and successfully been preparing students for the professional world?  For business, management, teaching, or whatever is the employment of their choosing?  What are the tools by which I can measure such success?  Having previously placed these essential reflections on the back burner, I, like Descartes in his famous Meditations, today propose to momentarily clear away my cares and meditate on this concern.

     I would like to have a direct and transparent assessment method, such as a comprehensive examination, but since few students are likely to embark on a concerted philosophical journey an exam seems unwieldy and misses the mark.  Instead, one could perhaps compare student salaries and student satisfaction surveys and index them against one’s grant acquisition skills, but in today’s hectic and unstable economy such a project would require longitudinal studies that stretch well beyond my adjunct’s term of contract.  Existing total quality management strategies translate poorly into student outcomes and can be quickly rejected.

     Falling into a dizzying uncertainty, I wonder who might have previously encountered such a problem and point me towards a way out?  Being a skilled philosophical technician, I immediately grab my tattered copy of Plato’s Republic; if ever there was an esteemed colleague concerned with conquering uncertainty it was Plato.  For Plato recognized that an effective educational system, like an effective and just polity, leaves little or nothing to the winds of chance.  Having witnessed his dear mentor stripped of his few remaining years and unjustly put to death by the ill-intentioned Athenian masses, Plato hoped to develop a system in which such excesses would be prevented.  No more would cretins like Meletus and Anytus slide their way into positions of power and influence.  One must ensure that the leaders of a society make the correct decisions and that the business of ruling (or, the ruling of business) is left to those who best understand its complicated and dangerous terrain.  The builders of a city must incorporate a system of assessment to find such leaders, educate them, and prepare them for their responsibilities.  Plato’s Republic, it now appears to me, is nothing less than a full-blown mechanism for the determination of outcomes and their assessment.
 
     Plato argues famously that no person is suited to do a multiplicity of tasks well.  And while we might reject this assumption if taken unilaterally, who can deny that some are better fitted than others to be doctors, or carpenters, or warriors, or street sweepers, or artists?  To fail to make use of the talents of its members is a form of inefficiency a society can ill afford, in Plato’s day and ours.  And if we are to make effective use of the available pool of talents, we must devise ways to identify the specific talents as early as possible.  Such considerations led Plato to his controversial gender egalitarianism—who knew where one would find talent?  Not even a woman (as Plato would have put it) is a priori unsuited for tasks of leadership and valor.  Similarly, we cannot rely on the marks of birth, for the sons and daughters of the working classes may indeed grow up to be leaders of industry and politics. Thus, each youth needs to have access to the best possible educational system, at least until we can best determine their natural talents and steer them toward their appropriate career.  To teach skills that will take a youth down the road to a position for which she or he is ill suited is, of course, a road to individual unhappiness and social chaos.

     Plato, however, faced a difficult task, for it is not immediately obvious how we are to discover the natural talents of the individuals engaged in a course of training.  The simplest course would be if one could see from the physical attributes of an individual that special role for which she or he is best suited.  This has clearly been a happy chance for empiricist philosophers such as Aristotle, who noted in his Politics that those suited for slavery have “strength for the menial duties of life,” while the free men are “upright in carriage.”  The sorting of people is something analogous to the means by which a child sorts different marbles by color and weight.  Occasional breakdowns in this divine arrangement may occur, just as if a heavy marble is internally unbalanced, but we have little recourse to such knowledge except by experience and testing.  As Aristotle notes, “it is not as easy to see the beauty of the soul as it is to see that of the body.”1  Plato, on the other hand, rejects the claim that the body can tell us anything about the soul; after all, how else could such wisdom spring from one as ugly as Socrates himself?  Plato’s sorting faculty, it turns out, is more like the well-known phenomenon of chick-sexing, in which mysteriously trained individuals can identify physically undifferentiated male and female day-old chicks, the females destined for egg production, and the males, well, I prefer not to interrupt my meditations...

     Indeed, my reflections are now paying off.  To assess the basic nature of the individual and properly guide it to its suited end requires a novel faculty of vision; an inner seeing that gives us knowledge inaccessible to either the uninitiated or the untalented.  I am encouraged to inquire more closely in order to discover what I must do to apply my evaluative faculties correctly and fairly.  I shall begin with Plato’s startling (to his contemporaries, anyway) notion that men and women are each potentially suited for all positions within a society.  In the well-known Book V of the Republic, Socrates argues that women are suited to be guardians of the polis, since the obvious differences between a man and a woman are irrelevant to the virtue of guardianship (as the differences between a bald and hairy cobbler are irrelevant to shoe-making and, hopefully, given the visible state of my own still youthful pate, to philosophy). As “a man and a woman who have a physician’s mind have the same nature . . . the women and the men, then, have the same nature in respect to the guardianship of the state.”2

     I turn now to Plato’s familiar metaphysical conception of the person, composed of body and soul, the latter made up of the three elements: reason, spirit, and desire.  These parts of the soul are harmoniously integrated, sort of a Mr. Boston Bartender’s Guide to person making (1 part reason plus 1 part spirit and 1 of desire, add a twist of lime and pour over ice in a highball).  Like a good drink, a good soul is well balanced, with reason providing the kick: as long as reason rules, irrational desire will not, like too much vermouth in a martini, ruin the taste.  Entrusted with the affairs of the polis and its defense, guardians are strong in reason, but this is not to sneer at those who are not so well endowed, for they too have important roles as the work force and support system for the guardians.  (Since access to luxuries and the ever-increasing desires they produce are at the root of all power grabbing, the life of the guardians is surprisingly Spartan; however, this is more than made up by the respect and power accorded their position in society.) 

     The soul is not only an indicator of the role that a person is suited to play in the polis; it is also a mark of the power to have knowledge.  True knowledge depends not on things empirical but on one’s ability to see beyond the ever-changing world of becoming and rise into the world of the forms—of truth and beauty and justice.  We must not allow our bodies to get in the way of such a magnificent pursuit.  The knowledge gained by means of the soul, in fact, is invisible and unchanging, much like the soul itself.  Anyone with an appropriately harmonious soul, a philosopher-king or -queen, is “capable of apprehending that which is eternal and unchanging.”3  My day’s meditation is now complete, for the way forward is clear.  My special talents as a philosopher are not wasted, for I am granted the ability to see the unchanging natures of others in order to assess what their social position ought to be.  Like a chick-sexer, I must turn away from the alluring world of experience and look deeply into the souls of my students.  Success is guaranteed, and my (non-tenuring) position remains secure.

* * *

     I wake up confident that I can complete my meditations today.  Yet, the more I dwell on matters, the more it appears that I have a serious problem remaining.  For I look and look but fear I don’t know exactly how to see the souls of my students, and to distinguish the reasoned souls from those overcome with spirit and desire.  If some souls are, in fact, destined for advancement, how am I to determine which?  I feel no more advanced than when I began these meditations, for the special faculty of philosopher’s wisdom seems now to be embarrassingly withheld from me.  I must, I think, turn again to my trusty (if now even more tattered) Plato.

     Even Plato, I now see, did not really believe in a direct access faculty that displays for us the souls of our students. Instead of a genuine chick-sexing faculty, there is a normative connection between the body and the soul that offers us the necessary empirical access to information on the harmony of the invisible soul.  Some bodies just act improperly balanced, while others embody soberness and wisdom and courage and all the other nice virtues.  This is a consequence of the fact that, although one cannot tell from the form of a particular body what kind of soul it has, a well-balanced soul is demonstrated in the actions of the body.  When a woman has the kind of a soul appropriate for a philosopher, she must in fact act as a philosopher ought to act.  Failing to behave in such a manner clearly demonstrates that she is the bearer of an ill-balanced soul.4

     Furthermore, there would be no purpose to the evaluative education so specifically laid out by Plato if this were not for the ultimate purpose of allowing students to demonstrate their inner talents.  Again, philosophers are not chick-sexers, for they are not able to separate out the various talents of newly born humans until they are sufficiently grown to be tested in various ways.  Thus, we shall place them in competition with one another and offer them opportunities to demonstrate their inborn abilities.  We shall test their abilities to mimic the existing philosophers who have already proven to be most able in reason and the responsibilities of rule.  We shall set tasks for them to perform and those that competently master such tasks will, by necessity, be those who are well prepared by nature for their execution.
 
     Yea, I have quickly come full circle, and any doubts about the nature of assessment have been effectively put to rest.  Well and clearly established criteria are the safest way to guarantee the reproduction of the society as a whole for its advancement and defense.  If I simply follow the appropriate testing methods then the best souls will excel, else they were not formed as those of the best.  Our compelling interest here is advancement, thus I will distribute those markers of success we call grades which will serve as tokens representing the nature of the student and reminders of past performances.  A poor grade is, after all, no slur on the true talents of those not suited for the world of philosophy, and they can indeed find other professions in which to flourish.  I may not, in the course of these Meditations, have established precisely what manner of evaluation is best, but I have indeed placed the heretofore open question of such evaluation on a firm and level foundation.

Part II: Plato’s Reality

     Obviously, one need not ascribe to Plato’s theory of the soul, or his single-talent theory, in order to develop a justification for assessing student-learning processes along predominantly traditional lines.  Yet, my investigation into some nuances of his system of education and its purposes reveals a number of important issues.  The first, I think, is the overall goal of an educational system to guide students into a system of politics, culture, and economy, with the consequence that such a system has a mimetic function, since the secure place of the judges is recapitulated in the forms of knowledge deemed useful.  By itself, this is not necessarily a pernicious feature of an educational system; rather, it merely raises the question regarding to what extent this will be an open (self-critical, evolutionary) system and to what extent a closed (regimented, deterministic) one.  Plato’s explicit preference for the latter appears to mirror the current paradigm of assessment, even if this is not an uncontested paradigm.

     Second, and more important, is the idea that whatever tokens are distributed to meritorious performers are generally considered to be indicators of the very nature of the student.  Most, though not all, professors explicitly deny that this is the case: “After all, it’s only a grade, indicating no more than one’s accomplishments in a particular class!  A lousy grade may be more a testament to one’s life circumstance during that course than one’s proficiency with the subject matter itself.”  Such a response, I think, undermines the very justification for the token/grade provided, since the grade is given precisely to sort the students into those with mastery and those without.  Closely entwined with this is my third claim: there is a probably insurmountable difficulty of using anything other than empirically demonstrated indicators of performance, indicators that must be standardized since their purpose is ultimately to sort and compare.  How one organizes these indicators will always remain contingent (could have been otherwise) and thus the sorting will depend on peculiarities of the individual instructor or the overall goals of the institution.

     Fourth, this analysis gives us a possible direction to investigate the role that difference plays within the educational institution at large.  For example, Plato’s purported egalitarianism ends up being largely qualified by his normative expectations of the “proper” performance of men and women.  Expectations (of oneself and others) play a huge role in scholastic performance, creating subtle shifts in student responsiveness, penchants toward failure, and the “charitable” actions of an instructor who “knows” that some particular student) is “capable of much more” and shifts the grade accordingly.  Fifth, then, a powerful consequence is that students who perform as if they are interested and willing are in fact behaving in such a manner precisely because they know how effective such an approach is to generating a positive teacher-student relationship; this accords with the sense of entitlement often present among students of more privileged backgrounds.

     While I cannot, in the remainder of these reflections, hope to address each of these multiple and complex questions, questions that to some extent every teacher must grapple with, I do think that my Meditation has raised them in ways that will seem unusual and even provocative, allowing a fresh approach to the still almost entirely conventional and unhelpful approach that has thus far been taken with regard to recent innovations in student outcomes and assessment, at least as they have been implemented by universities and accrediting agencies.  But there are other reasons to begin with a meditation on Plato.

     When I first taught Plato’s Republic I spent a good bit of time explaining how his picture was that of an ideal city, an imaginative exercise in individual and political excellence, but not one that was to be found anywhere in the world at large.  This was a way to encourage students to take seriously his vision of justice while not being waylaid by the hierarchical aspect of the Republic, an aspect bound to turn them off.  Similarly, the huge role played by censorship in ensuring that Athenian students are not misled is deeply problematic, and contemporary students are usually troubled by this deeply coercive element.  Indeed, I had convinced even myself that Plato’s theory was a strange and archaic vision to haunt the dreams of free moderns, since our intuitions about justice and appropriate education are incomplete without a robust sense of freedom.

     Later, however, I was forced to rethink the Platonic imaginary when I came across Kathy Ferguson’s description of contemporary hierarchical organizations:

Modern bureaucracies are organizations designed to seek certainty in both their internal structure and their environment.  The formal structure of the organization is aimed at ensuring this condition.  However, even the most rule-governed and rationalized organizations cannot totally eliminate uncertainty, so the rules must be supplanted with norms and informal patterns of behavior to further ensure stable and predictable patterns of behavior on the part of the members.5

     I raised my eyebrows, as I had already considered the way in which Plato’s educational system fills the gaps (i.e., supplants with norms) that cannot be accounted for by the mere tri-partite division of society into commoners, auxiliaries, and guardians/philosophers.  The educational system was not only organized to divide, but also to align the goals and practices of individual citizens along with the goals of the state; most importantly, all citizens would have to voluntarily support the hierarchical division. If Plato’s genuine desire was the elimination of uncertainty, then it may be that his vision is, ironically, closely matched in contemporary bureaucratic systems that have carefully designed mechanisms to elicit certain behaviors.  Plato’s philosopher-kings, as described above, must identify specific patterns of behavior that demonstrate the innate talents of the students; analogously, the successful and aspiring entry-level worker has her or his own task readily available, again described by Ferguson:

In order to please supervisors, the upwardly mobile bureaucrats must develop the skills of impression management; they must learn to present the appropriate image, to anticipate the requirements of their superiors or of the organization in general, and to comply with those requirements in order to earn approval and promotion.6

     In Plato’s Republic, the aspiring philosopher-kings must display the appropriate signs and signals that indicate special worth, for the philosophers really have no means available other than these bodily indications.  Since the teachers are the ones who hold the esteemed positions in society, aspirants learn, if they are sharp, to emulate the patterns of behavior of those teachers.  Successful advancement in the Republic as well as in IBM is fundamentally dependent on image management and, in particular, image management that approximates that of the supervisors:

The more similarity there is in outwardly identifiable characteristics, such as race, sex, dress, language, and style, the more likely is an aspirant to be seen as the “right kind of person” and given access to positions of discretion and power.  This benefits those applicants who can utilize impression management successfully in that it gives more access to upward mobility; it also benefits supervisors because it provides reassurance that the goals of the lower-level members coincide with their own...7

     Plato’s ideal city, when looked at as a diachronically self-reproducing political entity rather than a synchronous final moment, requires this kind of bureaucratic identification; in fact, because of Plato’s desire to reduce all uncertainties, he has successfully found a way to implement the coordination of aims in all members of the society.  Once members of the society have accepted the myth of the metals (the naturalistic, meritocratic belief that persons come in three basic types), the philosophers identify the specific characteristics that best fit the already existing system and provide advancement to those members.  I think this is a rather terrifying vision, made all the more fearsome for having been so successfully implemented.  Plato’s ideal is more of a reality than he himself had ever expected.  And, as I want to show, it is a reality in the educational workplace no less than in the corporate workplace.

Part III: Teaching Students Flexible Accommodation

     Sadly, Plato’s ideal functions not only in the bureaucratic office-space (or, what is much the same, the shop floor), but is, I wish to argue, the basis for the existing evaluation scheme in the modern university.  Upward mobility is guaranteed through the production of grades.  Such would all be well and good were it the case that grades could, like Plato’s vision, adequately represent the students’ true natures.  And just look at the face of a student who receives a C on their first paper to realize just how much they believe that grades represent their nature.  Instead, they represent only (or at least primarily) the students’ skills at what Ferguson calls impression management, or what I prefer to call “flexible accommodation.”8

     In fact, it is interesting to see that the university’s concern with outcomes and assessment only mirrors what students have known all along: that it is really only the outcome of any class that matters and that outcome is measured solely by the grade obtained.  As Jerry Farber wrote over 30 years ago:

Academic success, as everyone knows, is something that we measure not in knowledge but in grade points.  What we get on the final is all-important; what we retain after the final is irrelevant.  Grades don’t make us want to enrich our minds; they make us want to please our teachers (or at least put them on).  Grades are a game.  When the term is over, you shuffle the deck and begin a new round.  Who reads his textbooks after the grades are in? What’s the point? It doesn’t go on your score.9

     College students, faced with a harshly competitive world, are well prepared to engage in the pursuit of grades, but they have one significant obstacle: the teachers.  The biggest problem with teachers, as student after student will confirm, is that they are simply so subjective in their criteria for grading (which is not to say that they aren’t fair).  Some have very specific standards regarding format, while others are slack.  Some look for length, others for quality.  Some prefer mere repetition of the class lectures, others grade high for originality.  Some utilize the textbook, others ignore it entirely.  Some, it must be admitted, are inconsistent in their evaluations, and these are the ones that really bother the students, since it injects too much uncertainty in the system.  And I’m forced to agree that such inconsistencies are one of the most egregious faculty behaviors—even if standards are necessarily iconoclastic, they must be so fairly.  But most importantly, some are harder than others are, and it is impossible to know one way or another until one has some experience with that particular teacher.  With four to six classes, a thirty-hour a week job in a local mall, and a car and cell phone to maintain and nurture, a student hardly has time to waste and must quickly size up the needs of any teacher they confront.  “What will I need to do to get an A in this class?” is the oft-heard refrain, and teachers are advised to take it seriously as the primary motivation for student learning.10

     But is it actually the case that students have nothing to go on and thus are forced either to guess at the standards of the teacher or to over perform on their first assignment?  Actually, students are reasonably well equipped to evaluate, from the small clues that can begin before the class begins (reputation), by the form and descriptions on the syllabus, and by their impressions of the instructor.  Their primary goal is: what are the specific needs of this teacher, and how can I meet them?  Those who are especially skilled in quickly identifying the subjectively organized criteria of an instructor and meeting them are the ones who are guaranteed success in college.  This is why I call it flexible accommodation: students with the maximum range of flexibility in matching their work to the desires of the instructor are receiving the real lesson of our educational system.  Success in college is actually an excellent gauge of the likelihood of one’s success in a bureaucratically organized professional world where advancement is dependent on identifying and emulating the needs and desires of those who have the power to distribute advancement and favor. The skill of flexible accommodation is, therefore, not only effective as a means to maintaining a high GPA; it is an end in itself, for it is the skill that will lead to success in the post-university world—be it in education, civil service, engineering and business, or graduate school.

     We can identify a number of central problems in the system as thus far described.  The first is that it appears to replace learning with a motivational structure based on rewards that are not intrinsically linked to the subject of learning.  Second, it serves to prepare students for unthinking acceptance of their role within a bureaucratic organization, reproducing that system by selecting out any critics and nonconformists.  Third, it teaches that success is indeed a function of imitation and flexible accommodation, and whatever critical lessons are presented in classes are nullified by their obvious inability to help a student succeed.  And, finally, well-intentioned instructional faculty accept the norms of success that our bureaucratic system has produced, since alternatives remain unrewarded and are likely to disturb the existing distribution of good to bad students.  Anyone who thinks that more students than the norm deserve and earn high grades in their class, regardless of their standards, will surely wonder whether or not they are contributing to the demon grade inflation. And since untenured faculty are often evaluated, among other things, according to how well their grade distributions match those of the university at large, and if too many students receive A’s, they will be more likely to fall under suspicion for being too easy (perhaps to inflate their student evaluations) than commended for teaching their students so well.  

Conclusion: Against the Tyranny of Grading

     Consider, then, the uncertainty thrown into the system when grades are either entirely removed or are altered so as to no longer provide the primary motivation in a class.  Such an experiment has been tried at both the institutional and the individual classroom levels with varying degrees of success. To be honest, what I would like to see is an entire educational system based on the following statement that I have, on numerous occasions, spoken to students, bringing invariably an unbelieving stare and often laughter: “I just don’t care about grades, so give yourself one, if you think you need it.”11

     An additional problem, of course, comes when one considers that there are some within this revised framework who manage to get by with a minimum amount of work while fooling me with their demonstrated commitment.  (And this makes me different from other professors… um… how?)  This problem, combined with a professed lack of success in getting students into competitive graduate schools, recently led to the University of California Santa Cruz faculty senate vote to eliminate their tradition of narrative evaluations and to replace them with letter grades.  But a letter in the Chronicle of Higher Education, the University of Redlands Dean of Admissions Nate Budington claims that their narrative evaluation system has been successful.  He writes: “With no G.P.A. and with inch-thick transcripts and self-designed, self-titled degrees, our graduates have consistently gained entrance into the nation’s most prestigious graduate schools.”12  To the criticism that these narrative evaluations allow space for "slackers" to slip through the cracks, Dean Budington responds: “And the slackers?  Yeah, we’ve got them.  But they’re a lot more colorful than the slackers at Dartmouth College.”13

Endnotes