Getting Started with Evaluation

Finished October 21
Getting Started with Evaluation by Peter Hernon, Robert E. Dugan, and Joseph R. Matthews

This book starts with a definition of evaluation, looking at different ways to measure, benchmarking, and best practices. EBLIP, evidence-based library and information practice is discussed. There is a chapter on library metrics, looking at where we get the data we use. Following chapters look at subsets of this data.
Internal Evaluation for Planning and Decision Making includes a road map and a discussion of the three different types of data to collection: input, process, and output. External Evaluation to Inform Stakeholders and to Guide Continuous Improvement talks about the importance of thinking first about what and how you want to show the data, and about the quality of the data you collect. Types of data here include partnerships, customer satisfaction, supporting education and reputation.
There is a chapter dedicated to measuring satisfaction, looking at the different gap models that can be used, methods of data collection, and some samples. Measuring service quality is differentiated and gets its own chapter, looking at the two components: what is provided and how it is delivered. This discussion includes e-services.
The authors then move to ROI, looking at the economic benefits of direct use, indirect use, and non-use. It also includes ways to look at the cumulative impact on the community in areas such as community development, social inclusiveness, and an open democratic society. Measuring the value of the library and its service is also differentiated, and split into many categories: cognitive results, affective results, meeting expectations, accomplishments in relation to tasks, time aspects and money aspects. There is specific mention of the public library here with its role both as a traditional library, and as a community member.
Chapter 10 looks at what to do with the data we collection, including both how to use it and how to communicate it. This is split into sections of who to communicate to, what to communicate, and how to communicate it. It discusses the importance of aligning the three parts so that the what and the how are appropriate to the intended audience. Raw data should never be used, and libraries should be selective in the use of graphics. If a paragraph has more than five numbers in it, a table would be preferred to text. graphics are best with minimal text, and simple is always better than complex. Libraries should do test runs to ensure that the tool communicates the message that was intended. Another good reminder here is that colour can also play a subtle role, and choosing the appropriate type of chart for the message is important. An example of this is the guideline that pie charts show proportions, bar charts show relationships, and time series charts show change.
Any communication is to influence a particular group or groups of stakeholders. The timing of communication relates to the method. Libraries can be creative in the use of human interest stories placed in newspaper articles, written reports should always include an executive summary, presentations should be thought of in terms of visual appeal and good design, and newsletters should include information on progress towards goals.
The final chapter is a look at how evaluation leads to positive organizational change.
Every chapter has exercises, so this book would work well for classes in library school as well.