Internalizing the External Evaluation Approach
I’m finishing up a draft of a department self-study for an external review of our library. It’s the third time I’ve been involved in one of these, and the second time I’ve been primary author. It’s making me feel reflective about this enterprise we are part of, the nature of change, and questions of purpose and agency.
Deep thoughts, in other words.
External reviews are an interesting practice. Our first review in the early 1990s brought a number of overheated issues to the surface, not unlike a volcanic event. I don’t recommend bottling everything up until a review team arrives. It’s hard on the reviewers and a mess to clean up. Some good things came out of it, ultimately. The review amplified what an accreditation team concluded at approximately the same time: the library’s budget has to be increased. I’d forgotten what a huge difference a few years of budget growth makes. The review also kicked off a few years of difficult internal discussions that led to a complete redefinition of the librarians’ roles (adopting a more holistic and shared set of duties) and the development of a collegial management model for our library organization, something I still find exciting. It works for us, and it mystifies me that more libraries don’t try it.
The second external review in 2003 was less eventful. No skeletons tumbled out of closets, no Gordian knots had to be whacked apart. We got answers to some questions we posed, we got an endorsement for the things we wanted, and we felt many of our efforts affirmed, particularly in terms of what we did to promote and support student learning.
Working on the documentation for the third external review reminds me that some of the projects we’re working on now got their start ten years ago. The reviewers noted our shelving was near capacity and warned us that we’d have to start planning accordingly. The library was packed to bursting with students who clearly felt ownership of the place and were occupying every available space. We would have to figure out how to preserve that space for students to interact in groups, study in solitude, or spread out as they worked on research. The weeding we’ve been doing wholeheartedly for the past few years stems from this recommendation.
But it also reminds me that some things are in our control, and some are not. Those budget increases leveled off and have been flat for over a decade. (Our staff costs, particularly for health care, have no doubt increased, though our staff is smaller and younger than it was.) Having ever-increasing subscription costs and a stagnant budget makes us constantly tinker with our collection, trying to cut anything that isn’t necessary. In some ways this is a healthy form of simplicity. You really need to know what your priorities are, and you have to involve the entire faculty in defining them. In good times, there’s no real need to be so analytical or reflective. We’ve gotten good at both. And we've gathered a lot of data.
A team of librarians, staff, and students conducted large-scale ethnographic study of our virtual and physical space, which also drove many of the changes we've made since the last review. We redesigned our website, we reorganized space, we created a new reference desk where we could sit side-by side with students to talk through their questions. We have a trove of information, from student and faculty surveys, to seating pattern studies, to interviews and focus groups, to picture associations and photo diaries. Lots and lots of lovely data, and it has been really useful.
But there are limits to what you can do with data. You can say “;look, students say they need more space, they need more places for solitary study, more group study areas. They’ve been saying this for 25 years. Can’t we do something about it?” Well, here’s what we can do: we can scour campus storage areas for tables that can be scrubbed up and used in place of underutilized carrels. We can create nooks in the stacks that are a little like study rooms, though never quite as popular. We can move furniture around and empty some shelves and patch things up. Because that’s in our control and requires time and imagination, but not money.
We can nip and tuck and cancel this and that to cover budget gaps. We can stop buying anything that isn’t a high priority and patch holes by canceling a thousand journal dollar subscription, making up for it by buying one $ 40 article at a time. Every time a vendor promises a cheaper version of an essential database, we can make a switch and hope the savings last longer than the time it takes to fix all the broken links. But data doesn’t lead to a bigger budget.
We academics have a habit of using the term “;bean counters” when condemning those soulless individuals in administration buildings who don’t invest in programs that aren’t performing, who run the numbers before they make decisions. How I wish more administrators were bean counters! How I wish they cared about data, about evidence, about careful stewardship of scarce resources. It’s useful stuff, data. Every improvements we’ve made in this library was a response to the data we’d gathered. But outside the library? Decisions seem to be made by some other means.
I remember thinking during the University of Virginia debacle, a power struggle that seems to be happening all over higher education, that what rankles the most when watching those who have a bit of power try to reserve it all to themselves, is that they’re so bad at using it. The things we care about in higher education -; ethics, the logical consequences of actions, the use of evidence in decisions, the virtues of equality and a belief that rational people can make decisions for the common good, that stuff we started believing during the Enlightenment, is being replaced by something else. Something blind and careless and full of confidence in the righteousness of power.
I suppose we could end each department meeting with the serenity prayer, but there are things that need changing, no matter how powerless we feel. So on we go, trying to introduce students to values that have been around a long, long time and may still, some day, come back in fashion. We can hope.