Each graduate of the Master of Library and Information Science program is able to evaluate programs and services using measurable criteria;
The evaluation of program and services is an important measuring tool in understanding the needs of users. It is also a useful way of understanding how users respond to its current programs and services available. Not everything we implement in our institutions is needed for the types of patrons we serve because our communities have different needs at different organizations. As a result, it is necessary to examine the needs of users and make adjustments accordingly. We also need to examine the satisfaction of users because their satisfaction equates to the utilization of available programs and services. If users are not using our resources, time and funds are not been channeled in the right areas that would benefit users. Therefore, we need to implement ways of measuring how effective our programs and services are through a combination of assessments and evaluations from information professionals and patrons.
Information professionals in the Library and Information Sciences field are understanding that looking at business and marketing models is an important way of understanding customer satisfaction. Like any type of service whether for profit or not, organizations typically want to retain loyalty from consumers. According to Hernon, Nitecki, & Altman (1999), loyalty is one of the most prominent marketing trends that date back to the 19th century (p. 9). In a service-oriented environment, like libraries and archives, retaining loyalty from users is just as important as it is for the profit sectors both then and now. For nonprofits, this loyalty translates into the institution’s value within the community. One way of attaining this loyalty is through customer satisfaction with the overall quality of services and programs. Loyalty is not a tangible value that we can easily measure so it is necessary to find ways to measure the satisfaction of users.Organizations want users to come back and use their programs and services to designate how we are enriching the lives of the community members we serve. Therefore, it is important that we measure the quality and performance of our activities so that we recognize areas of improvement to better serve the needs of our users.
Measurable criteria
Assessments and evaluations are important for institutions because they allow a means of measuring information about the current state of affairs while providing insights into the efficiency of programs and services. Finding measurable criteria can be a daunting task for institutions that need to compare and evaluate information based on subjective information. According to Evans & Ward (2007), this is based on performances and processes, rather than products, and is based on user expectations. This makes it difficult to measure because performance varies from staff members and evaluations often vary from person to person based on their current situations (p.237). Meeting the expectations and perceptions of users is a difficult task because it relates to various experiences from users that often contain discrepancies that are difficult to measure. Balancing customer expectations and perceptions helps to close gaps that are formed between the two and helps create a higher quality of service from the perspective of users. It is important to balance the perceptions and expectations of users with the quality of service that an institution provides because it helps retain customer satisfaction (Nitecki, 1996 p. 181-182). Regardless, It is well worth the effort finding assessments and evaluations that align with the mission, goals, and values of an institution as well as, being feasible to implement. Assessing and evaluating programs and services is necessary for collecting information that is valuable to the future growth of its institution. This prepares institutions for the changing needs of its users and allows them to be better equipped in accessing how to implement changes and updates to programs and services.
Methods
Assessments and evaluations are useful tools in which institutions can juxtapose information collected from internal and external resources that are then compared with other measurable criteria. Measurable criteria can take many forms and includes both qualitative and quantitative information. When evaluating information organizations, it is not enough to measure content and collections but rather the performance of activities, programs, and services (Nitecki, 1996 p.182). There are several approaches that information professionals can utilize to help evaluate and assess their programs and services. One way of examining programs and services is through quality management. Quality management is one way of examining different components of a service or program to measure the “quality.” Evans & Ward (2007) discusses Parasurman, Zeithamal and Berry’s (1985) research that describes areas to evaluate when assessing quality. These include:
- Reliability / Consistency
- Responsiveness
- Competence
- Access
- Courtesy
- Communication
- Credibility
- Security
- Understanding customer needs
- Tangibles (facilities and personnel)
(p.237-238)
All these areas are useful in deciding how to evaluate quality that helps information professionals to look at the service and program from different angles.
There are also other approaches that information professionals can use to evaluate and assess programs and services. Benchmarking is another method of evaluation that compares the performance of services with similar organizations to help improve quality. This involves sharing information to help organizations establish standards in which to compare from (Evans & Ward, 2007 p. 241). Gathering the opinions of users is also a vital form of evaluation that provides a great deal of insight concerning quality, user expectations, and their needs. “The assessment of how well a library succeeds depends on the user as a judge of quality (Nitecki, 1996 p. 183). Therefore, it is important to not overlook user’s perspectives on how they judge services. Evans and Ward (2007) explain that people have four criteria in which they judge services:
1. Expect to get what they want
2. When they want it
3. At an acceptable cost
4. and delivered in a way that meets their expectations
(Evans & Ward, 2007 p. 224)
Examining the criteria in which users judge services is helpful in designing evaluations that will help us collect information that will improve the overall quality. Suggestion boxes and boards, surveys, questionnaires, interviews and focused groups are some of the ways that we can collect information from users (Evans & Ward, 2007 p.249). There are also evaluation services and tools that help collect information from users and measure performance. Some of the most prominent include LibQUAL+ and balanced scorecards. For example, LibQUAL+ collects statistical information from different methods that include questionnaires, surveys based on measurement systems such as scales. Scorecards collect information from both internal and external resources and analyze them based on metrics developed for the organization ( p.241,243). These tools help collect data that allows us to discover trends, patterns, and problems that would not be easy to collect without the aid of these systems. Although useful, costs of such services may be a factor that informational organizations must consider.
A combination of various evaluations and assessment methods is useful information to collect to help make significant changes and updates that are beneficial for users and the informational organization. This information can be implemented in strategic planning sessions that can help make significant changes. The information that is gathered can help revisualize outdated systems and policies that affect the quality of services and programs. Therefore, it is important for information professionals to consult different modes of evaluation and assessment because it will generate richer content that will help us make decisions about design and how to improve services and programs.
Competency Development:
Throughout my studies at SJSU in the MLIS program, evaluation was an important theme throughout many courses. A lot of my courses involved evaluating resources, programs, and services to help me understand the importance of always thinking critically about our surroundings. Looking critically at our surroundings is important because we want users to have access to resources that suit their needs based on the community we are serving. We don’t want to waste resources in areas that really aren’t appropriate for our users. The needs of users various from a community to community so it is important to understand how to use evaluations and assessments to help us figure out what is needed. Many of my assignments consisted of evaluation exercises and essays to help us look at real-world examples of resources, programs, and services that we could analyze. One of the most helpful courses, that provided me with key evaluation tools was Libr 204, Information Organizations and Management. In this course, we learned about how evaluation and assessment assisting information professionals with decisions about the organization they work for. We also learned about management strategies that will help me become an effective planner so that I can plan to use evaluation and assessments within my organization. Using evaluations and assessments throughout my courses allowed me to learn how to analyze situations and issues so that I think about possible improvements and solutions. There is always room for improvement, so it is necessary to utilize evaluation and assessment tools to help us realize areas that need adjustments.
Libr 204, Strategic Plan 1 & 2
In Libr 204, Information and Organizations, I learned how professionals tackle daily challenges using management tools and strategies. Some of the challenges that information professionals face are the constant need to evaluate the organizations they are involved with. In this course, I had the opportunity to learn how to evaluate and analysis an organization’s structure, program, and services. This allowed me to use relevant course material with current business models in a deep analysis about a public library. In this course, I worked with a team, to create a plan to help us analysis a public library. My team members and me discussed various ideas about how to collect information for our analysis and what needed to be implemented among group members. We all worked together to gather and analyse data to help create a strategic plan for the library under analysis. In Part 1, we gathered and organized our data. We implemented an environmental scan of the libraries internal and external information that would aid us in our analysis. In our environmental scan, we looked at the external information, such as, demographic and geographical information to understand the populations that lived near the library. For our internal data collection, we looked at the library’s current mission, goals, programs, and services to give us an idea about the current state of the library. We also obtained interviews from employees to help us understand the work that was being done and what was still needed from an insider perspective. We also looked at photographs of the library to understand the floor plan and its relationship to access and services in our analysis. After the collection of our data, we implemented a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis to help us organize our data and to contextualize it within this framework. For this part of the project, I was in charge of organizing the data collected from the SWOT analysis into an essay that presented our analysis. This allowed us to analyze the information collected and provide a list of recommendations in a strategic plan for the library in part 2. In this project, I learned how to create a strategic plan based on evaluations and assessments. Collecting both external and internal information helps information professionals evaluate an institution more accurately. This is vital because both internal and external factors will change, and as a result, the institution will need to progress as well. As an information professional, learning to create an evaluation plan working with others is important because these are the type of situations that are part of the work conducted by information professionals at the library. The exposure and experience will prepare me to work with others and help me understand what is needed to conduct good evaluations and assessments.
Libr 204, Good to Great Essay
In Libr 202, I also had the opportunity to evaluate an organization based on my perspective and course concepts. I wrote an essay evaluating a non-profit organization based on their resources and services. In this essay, I evaluated this organization based on Bill Collin’s concepts in his book Good to Great (2005). This book looks at factors that make a non-profit organization “Great” that is useful for making comparisons. The book provides examples of what makes organizations better and explains that all organizations have the potential to be “Great” if they utilize available resources. I specifically looked at the Hedgehog concept to evaluate a non-profit organization within my community. The hedgehog concept looks at what an organization is best at, what drives their economic engine and harness their passions to discover an organization’s strengths and focus. This allowed me to explore a different type of evaluation method, that was based on current and progressive literature about management and strategic processes. It is important to explore different evaluation and assessment methods because they provide a unique perspective to the evaluation process. This assignment is important to my development because it provided me with literature and new concepts that helped me evaluate an organization. I learned that organizations have resources that they can harness to help improve their organizations. Since informational professionals work in the non-profit sectors, the information I learned from this assignment will be carried out in future evaluations about organizations.
LIbr 254, Instruction Session Observation Report
In Libr 254, we learned about the importance of implementing standards when we are creating instruction sessions. Standards are an important component in information literacy instruction because they provide guidelines in which we can model our courses around. Standards also provide a means by which we can establish criteria for the purposes of evaluation and assessment. For this course, I had an assignment that required that I evaluate an instruction session that I attended. I attended an instruction session that was an introduction to computers in Spanish. For this assignment, I evaluated this instruction session using information based on literature about information literacy. I also looked at information literacy standards such as AASL Learning standards, to help me evaluate this session. I specifically looked at sections of 1-4 in my evaluation. Referencing standards is a useful strategy in evaluations because it provides a basis on which one can make comparisons and assessments. For this assignment, I learned the importance of using standards as one method of evaluation and assessment. In the future, when I evaluate these types of services and programs I will look for appropriate standards to compare how an organization’s services measure up in comparison. This assignment also allowed me the opportunity to focus in on one type of service and evaluate it from the ground level of the institution. Evaluating services at this level is an important component in evaluating programs and services because it gives information professionals a chance to see how the service is functioning from the user’s level. The information gathered is an important way of understanding how users are experiencing services and programs and will help me gain insights into the needs of users. This assignment is a great reminder to evaluate services and programs from the perspective of users, and that attending live sessions should be an important part in the evaluation of services and programs. This assignment has enriched my perspective in evaluating programs and services through the perspective of users and as a result, attending sessions will be an important part of my evaluation process when feasible.
Conclusion:
The needs of users are constantly changing as society progresses into more technological avenues. Programs and services need to be nurtured and fine-tuned because they are continuously changing to adapt to the needs of users. Evaluations are a necessary tool even if critiques are uncomfortable to employ. Critiques are an important part of evaluations because critiques have the power to result in significant change. As a result information professionals should utilize the power of evaluations to help make improvements to programs and services. Evaluations and assessments are an important part of the work that information professionals conduct, so it is important for me to learn about different types of evaluation and assessments. Learning about different types of evaluations and assessments will help me make better decisions about the types of evaluations needed for organizations. This will help me grow as an information professional as I discover the needs of users through evaluations and assessments.
Resources:
Collins, J. (2005). Good to great and social sectors: A monograph to accompany good to great. Jim Collins: Boulder, Colorado.
Evans, G. E., & Ward, P. L. (2007). Management Basics for Information Professionals (2nd ed.). New York: Neal-Schuman Publishers.
Hernon, P., Nitecki, D. A., & Altman, E. (1999). Service Quality and Customer Satisfaction: An Assessment and Future Directions. Journal Of Academic Librarianship, 25(1).
Nitecki, D. A. (1996). Changing the concept and measure of service quality in academic libraries. Journal Of Academic Librarianship, 22(3).
Parasuraman, A., Valarie A. Zeithaml, and Leonard L. Berry. 1985. “A Conceptual Model of Service Quality and Its Implications for Future Research.” Journal of Marketing 49 (Fall): 41-50.
Assignments