1. Involve department heads in development of the instrument.Get their suggestions for content and wording, and their buy-in on the validity of the procedure. These are the people who will be accountable for the results, so they should have input into the process. Whether it's skiing, a spa, culinary arts, or meeting arrangements, they are the experts on defining the dimensions of guest expectations. 2. Use graphic design that matches the quality and style of the property and its advertising.While not a promotional piece in the usual sense, the form should have an appearance consistent with other branded materials used for guest advertising and information. Incorporate the logo and possibly other distinctive visual effects. Machine-readable forms are not pleasing to look at, and – no matter what the letter says – they tell the recipient that he or she is just a number. 3. Balance specificity of content vs. ease of self-administration in the items rated.Under Dining, it makes sense to have separate ratings for Food and Service. Perhaps we even cut these by meal – Breakfast, Luncheon, Dinner. But don't go too far. Under Food, someone will ask us to get scores for Menu Selections, Temperature, Preparation, Seasonings, Freshness, and maybe even others. Yes, they're all valid – but this demands too much of the respondent. You have to draw the line. 4. Use appropriate rating scales.A scale that respondents will quickly and intuitively relate to, and one that is easy to interpret. With labels such as Excellent, Good, Fair, and Poor. Numerical scales without words have too much subjectivity associated with the interior values. On a 10-point scale, does a "6" represent the academic "F" (60), or is it a shade above average because it's more than 5? In contrast, what part of "Good" (above Fair and below Excellent) don't you understand? 5. Maintain a primary or exclusive focus on guest evaluation.Clients often suggest including a response option for future bookings. Check this box if you would like to receive special offers, or Are you planning a seminar or conference later this year? When former guests see the survey request as marketing, their willingness to provide feedback is diminished and we damage our credibility. Questions about trip planning, other destinations, and media habits, which may help us with marketing strategies, should still be limited and of obvious relevance. 6. Avoid personal questions that are not necessary.Many survey writers automatically ask for household income, education level, race, or occupation. These are often seen as intrusive and irrelevant for a public accommodation. If it's personal and nothing more than "nice to know," leave it out. If you do ask for household income, make it optional. 7. Choose the right question.Where do you live? will produce answers, but most will be worthless. "In the suburbs," "about 50 miles from here," or "in Springfield" (almost every state has a Springfield!). If we want city and state, ask for them. Anticipate what you will and will not get from different types of questions. For example, ZIP Code tells us city, county, state, urbanization, DMA (media market), distance from the resort in miles, residential affluence, and more. 8. Precode answers whenever possible.This spares us coding large volumes of unstructured responses which we often find – after the work has been done – come down to four or five factors that we could have easily anticipated. (OK, we don't know their relative incidences, but we let the respondents tell us those.) There is nothing easier for respondents to do than check a box. Moreover, open-ended responses often remain completely untabulated because the survey designer underestimated the amount of effort required. 9. Use random sampling.The ability of a survey to represent an entire population with only a small error margin is based on the principle of random sampling. Every effort must be made to draw the sample from an exhaustive list of guests within the test period and to select those who are to receive forms in an unbiased manner. One popular restaurant and hotel guide contains prominently featured ratings that are based entirely on self-selected judges, many of whom are travel professionals. Guests returning forms left in rooms are another type of self-selected sample. When necessary, random samples are stratified to insure adequate numbers of guests in key segments that would be otherwise be insufficiently represented with strictly proportionate sampling. The sampling procedure should be designed by a qualified survey research professional. 10. Obtain statistically adequate numbers of responses.When designing the sample, identify the guest segments – by demographics, season, activity use, or anything else – for which you want statistically reliable results. (We generally consider 50 the minimum for statistical reliability.) Some segmentation variables – male vs. female, or first-time vs. repeat visit, for example – will break out approximately even. However, important but low-incidence groups will be under-represented in a purely proportionate sample. To insure that we have enough forms returned from such guests, we over-sample them (i.e. mail to them at rates exceeding their natural incidences) and later adjust the tabulations to restore proportionality to the total sample. 11. Number the forms.This allows you to exclude those who have already responded from any follow-up mailings. It permits validation of the survey process when handled by an outside vendor. Guest information can be appended from the house file and incorporated into the tabulations. It facilitates personal follow-up should it be needed because of something reported by the guest. Depending on the structure of the numbering system, it can serve as a code for the month of visit, room location, or guest category – leisure vs. business, for example. 12. Use a fully-personalized cover letter from the general manager.Survey mailings typically number in the hundreds or low thousands – small enough for desktop mail-merge production, individual checking, and blue ink signatures. Characteristics that subliminally alert recipients to mass production should be avoided. The salutation is "Dear Mr. Smith" – not "Dear John Smith" or "Dear Recent Guest." Abbreviations and truncations in addresses are reverse-engineered to be indistinguishable from one-of-a-kind letters. A carefully phrased message from the general manager explains the purpose and importance of the survey and thanks the guest for visiting. Live first-class postage is used out and back. 13. Mail forms to the home or office, post-visit.Placing survey forms in guest rooms is impersonal and invites a reply prior to the guest having completed the visit. Presentation at check-out is an imposition and associates the request with the desk clerk instead of the general manager. Plus, not all guests use the traditional check-out procedure. Best is a cover letter from the general manager containing the form and a postage-paid return envelope mailed to the guest shortly after departure. 14. Experiment with alternate response modes.Mail is excellent for simplicity and cost-efficiency. Telephone interviewing is intrusive, costly, and difficult to manage. Some experts feel telephone produces a less considered response than does self-administration. Cooperation rates for all types of telephone surveys have deteriorated. Two alternatives to mail that may be considered are self-administration via a web-hosted questionnaire (with the request and URL delivered via snail- or e-mail) and IVR (interactive voice response) in which respondents dial in to a pre-recorded interview and use the telephone keypad to answer. Both provide efficient data capture but at the expense of a more personal reply. 15. Strive for a high response rate.Response rate is the number of completed forms returned divided by the number mailed. While it is agreed that low response rates may limit the accuracy of survey results, there are no mathematical methods for quantifying the magnitude of this potential bias. (The margins of error popularly reported for public opinion polls assume response rates of 100 percent, which is unheard of today.) Many factors, including the quality of the cover letter, the length and appearance of the form, and guests' feelings about the resort, influence the response rate. Besides limiting the possibility of non-response bias, higher response rates mean that mailings can be smaller. In general, we consider response rates under 20 percent unacceptable; 30 percent, the industry norm; and 40+ percent, significantly above norm. 16. Cross-tabulate the results.Tabulation means counting the numbers giving each possible answer – 40% Excellent, 30% Good, 20% Fair, 10% Poor. Cross-tabulation is the same counting, but separately for subgroups of the sample. By women vs. men, or by first-time vs. repeat guests, for example. You need two things here: First, any proposed basis for cross-tabulation must be included in the questionnaire unless it is already available at the respondent level elsewhere; and, second, you need specially designed software capable of cross-tabulating questions by up to a dozen variables at a time. Excel and other spreadsheets, which have many uses, cannot perform cross-tabulations effectively. 17. Learn to interpret differences and trends.Comparison of results over time ("tracking") is one of the primary purposes of guest satisfaction surveys. Are we getting better, or worse? To be valid, comparisons must be based on identical question wording and survey procedures. Then we apply statistical formulas to the scores to find out if these are "significant" differences. If so, we still look for extraneous factors that might explain the changes – seasonality, competition, or a different guest profile, for example. While it's the job of the survey scientist to produce reliable ratings, interpretation is as much an art as a science and everyone should be involved. 18. Understand how surveys differ from other rating systems.Each of three types of evaluations – Zagat-type ratings, Mobil Guide or AAA quality scoring, and the guest survey – has its own unique characteristics. A site visit lasts one or two days, while respondents in a properly designed survey represent the entire year. Zagat and J.D. Power ratings are designed to sell magazines and give high-scoring properties copy for their advertising. Each has its place. However, the properly designed guest survey is a "best practices" management tool – scientific, comprehensive, tamper-proof, confidential, and customer-focused.