Many feedback questionnaires utilize plain old-fashioned paper and pencil or one-on-one interviews; however, online versions are increasingly being created and becoming more widely used. Whichever is used, it is important to have some open-ended questions, in order to obtain the widest feedback possible: there may be some crucial things an evaluator would want to say, that your questionnaire didn't even consider.

If questionnaires are used, the process generally goes something like this: a series of questionnaires are distributed, including one which evaluators can use to assess themselves. The others in the batch are meant to be distributed to others, such as the person's boss, the people he or she supervises, colleagues, and those who could be construed to be customers: in our case, that would certainly include library patrons, and could also be liaison faculty if you're in an academic library, leaders of local community groups if you're a public librarian, teachers in your school if you're a media specialist, vendors if you're an Acquisitions Librarian, etc., etc.

The completed questionnaires are generally returned to a central location for processing-preferably to an office completely outside your institution, to ensure maximum confidentiality and objectivity--and a report is prepared. Finally, the person who has been evaluated reviews his/her results. Since most people have a hard time taking even constructive criticism well, this review should be done under the guidance of a trained facilitator who can help to mitigate the emotional effects, help the person to analyze the criticism, and decide what kinds of behavioral change might be called for.

If you don't want to go to the expense and trouble of creating or buying standardized questionnaires from one of the many companies that supply these, an interview process can also be used. Lepsinger & Lucia recommend that if this path is chosen, the process should include the following steps:

• (1) The feedback recipient is informed about why the information is going to be collected, and how the information will be used
   

• (2) The feedback recipient is allowed to help determine what questions will be asked and who will be interviewed
   

• (3) The interviewer schedules and conducts the one-on-one interviews
   

• (4) The interviewer prepares a summary report which includes the key themes and patterns, plus some representative (but anonymous) quotes about the person's behavior
   

• (5) The feedback recipient and the interviewer meet to review the findings and decide on steps to be taken in response to the data
   

• (6) The feedback recipient creates a personal development plan which specifies activities, target dates, and progress review points

They point out also, that even if questionnaires are used, interviews are often needed in order to clarify the information received in order that the person evaluated can fully understand the criticism and take the right steps to improve his/her behavior. Some experts even insist that multiple means of collecting information is the only way to get truly helpful information. Obviously, however, the more methods used, the more complicated, expensive and time-consuming is the process. Administrators must decide for their own institutions what the balance between time and money spent, versus perceived benefits, will be.

Should you design your own questionnaire, or use a commercially-prepared one?

There are pros and cons. Some of the reasons why you might be better off not creating your own questionnaire:

(1) the questionnaires produced by commercial companies have generally been prepared by professionals who specialize in this kind of work;

(2) commercial questionnaires have usually been through comprehensive validation testing.... in the litigious world of today, a dissatisfied employee is inclined to sue, and thus a questionnaire for which validity can be proved may be a protection for you;

(3) it is expensive and time-consuming to create your own questionnaire customized to your library, since commercially-available ones can usually be found which will measure most of the areas which you think are important;

(4) some commercial questionnaires use normative scores, thus enabling the person evaluated to learn not just how he or she is rated by others, but also how his or her performance compares to that of many other people.

On the other hand, why should you make your own questionnaire?

(1) No matter how many one-size-fits-all questionnaires you look at, you may still not be able to find one which includes some competencies which are critical to your particular organization

(2) norm groups used by commercially-prepared questionnaires are almost certainly not going to be very relevant to library employees

(3) While some library jobs are comparable to customer-service and managerial positions in corporations, others are very unique to our field and thus a proprietary questionnaire may not get at the kinds of things you want to measure. (Ward 68-71)

If you do decide to design your own questionnaire, it is very important to define your objectives, which may include any or all of the following:

--(1) a desire to achieve cultural change within the organization, such as from a results-oriented environment to an orientation on results through people

-(2) the encouragement of self-development among individuals

-(3) management improvement

-(4) defining staff development needs

Quite a bit of preparatory work is required if you want to end up with a well-designed, effective assessment instrument-because it is necessary to decide what activities the successful employee in each type of job must be able to perform, before you can measure them. An extensive job analysis for each position is usually required-and there are no shortage of books on that topic, as you probably know.

Although some questionnaires are completely open-ended, and allow the rater to say whatever they want, most are constructed of items to be rated on a scale. Each item should rate a particular behavior, but this is not as easy as it sounds. Ward gives the example of rating a person's ability to communicate. Communicating includes competencies such as listening, writing, speaking and reacting. Broken down into more detail, listening may include using appropriate body language (such as nodding), asking questions, summarizing what you've said, and letting you finish without interrupting. Going to an even greater level of specificity, "appropriate body language" could include facing you while you're talking, looking you in the eye, keeping still, leaning forward, nodding, and otherwise signifying agreement with and acknowledgment of what you are saying! (Ward 79). Thus, it's easy to see how a questionnaire can get out of hand, and become formidably long and troublesome for participants.

Richard Rubin strongly suggests that there be substantial staff involvement in the creation of your instrument, that it be easy to use and able to be completed quickly, and that the person doing the questionnaire be shielded from possible identification by his/her supervisor. Actually, all the experts I consulted in order to prepare this presentation stress the necessity for both the respondents and the participants to have a sense of safety-an awareness on the part of respondents that their anonymity is completely protected-and for those being evaluated, that the data provided to them is confidential, and that it will not be used against them in a hurtful way.

Peter Ward's valuable list of tips on item-writing include the following:

-(1) begin each sentence with a verb and avoid passive voice (Example: "Approaches difficult work issues with enthusiasm")

-(2) keep items short, with sentences between 5 and 10 words long (Example: the sentence, "Shows discretion" is not very specific, but "Maintains confidentiality when dealing with sensitive issues" is both easier to understand and to evaluate.

-(3) describe only one behavior at a time. This is important, because if the person has to assign one number rating to the question, if more than one thought is expressed, it might be impossible to answer. (Example: "Give personal performance feedback to staff in a timely, sensitive manner" covers 2 ideas: timely and sensitive... the person being evaluated may do one of those very well, but not both.)

-(4) be careful about using the words AND and OR, because they usually indicate the presence of 2 thoughts. (Example: if the question is "speaks clearly and politely" -what do you answer if the person is clear but not polite, or vice versa.)

-(5) use everyday language, understandable to anyone and appropriate to both the person evaluated and the respondents

-(6) minimize the use of judgmental expressions, such as well, excellent, effective, and so on. In the sentence, "Makes effective formal presentations," what does effective really mean? Clear? Convincing? Interesting? Not too long? Keep in mind that the person being evaluated will have to ACT on this feedback, so use adjectives which add clarity and meaning

--(7) when appropriate, use questions which address the reader directly, such as "Allows you to speak without interrupting." This kind of question encourages the respondent to focus on their own relationship with the evaluee, not what they may have observed in his or her other employee interactions.

-(8) be sure all questions truly relate to the competency or value you are trying to measure-and remember at all times, that questions should be asking whether the person does or does not do something, not whether he or she possesses some personal characteristic or whatever (Caregroup 2).

-(9) use the words "demonstrates" or "shows" sparingly. This is usually taking the easy way out, instead of thinking through and precisely defining the behavior you feel is important and are trying to describe. Instead of saying, "Demonstrates effective communication techniques," you might say instead, "Presents ideas in a persuasive manner." -if that's what you really care about.

Although I noted at the outset that there are apparently few libraries utilizing some form or peer, multi-source, or 360-degree performance appraisal, I was able to identify a few, partly through scouring Library Literature, and also through the assistance of a colleague, Ms. Sonya Shepherd at Georgia Southern University. Although there won't be time to go over each one in detail as we'd originally planned, I do want to highlight a few of them, before I relinquish the podium to Frances and Dianne. In nearly every case, a copy of their form is included in your handout.

To begin, Gay Perkins reviewed the use of upward evaluation in libraries in her 1995 article in Library Adminstration & Management-and then turned to her own institution, Western Kentucky, which began a pilot project back in 1989, partly using the University of Texas at Austin system as their model. Western Kentucky's process was designated from the outset as confidential and limited to personal development only, because it was feared that some supervisees might take vindictive advantage of an anonymous review of their supervisors. After a lengthy questionnaire development period, the finished product was given to all supervisees-who were asked to fill it out voluntarily--in 1993. Responses went directly to the University's Academic Computing & Research Services unit, to help insure complete confidentiality.

Overall reaction was inconclusive; although some positive comments about the process were submitted, including the potential fairness of such a system and the benefit to supervisors of learning what their employees really think, there were a significant number of negative remarks as well. These included a perception that nothing would change as long as the upper-level administration was not allowed to see the ratings, that supervisees with a negative assessment of their supervisor would be reluctant to be honest, and that there was no anonymity in situations where the supervisor only had one supervisee. Time prevents me from going into more detail, but I highly recommend this article (the complete citation is included in the bibliography you received) if you are seriously considering such a system.

In order to help get staff to buy into the concept, the plan was developed and administered by the library Management Committee, which consisted of the College Librarian, the three Sub-Librarians, the Library's Staff Training & Development Officer, and the librarians' union representative, who is also one of the Department Librarians. Their recommendations were presented to the staff for comment and revision before the plan was finalized. It was decided that all staff with a minimum of one year's service in the Imperial College Libraries, who were on not less than half-time appointments, would be eligible to serve as appraisers-and all eligible persons were presumed to be available as appraisers unless they specifically requested exemption. Furthermore, everyone is appraised by someone who is on the same or a higher grade. Compulsory training for appraisers was provided prior to implementation.

The first time this plan was tried (1990), half of the eligible staff declined to be appraisers, thus leaving 13 staff members with the task of evaluating 30 people. By the second time around, however-which was 1993-confidence in the system had grown, and 2/3 of the eligible employees agreed to be appraisers, which spread the workload more equitably.

Those who are to be evaluated are asked to nominate as potential appraisers three people on the appropriate grade, from the list of willing appraisers-with mutual appraisal being disallowed. The College Librarian makes the final decision, and in every case was able to match each person with an appraiser of his/her choice. Because the evaluations are done by staff with differing levels of experience, the planning group decided to guide the interviews by using forms with specific questions and assessment ratings, to ensure that all aspect s of the job would be considered consistently and uniformly. Written comments clarify and amplify the grades, and the overall appraisal discussion is summarized by the appraiser, including comments by the appraisee, the College Librarian, and Head of Department if appropriate. An action plan identifying training and development needs, as well as appropriate action to be taken, is completed by the evaluator with the agreement of the person being evaluated.