geront      Gerontologistgeront      The Gerontologist      The Gerontologist      0016-9013      1758-5341              Oxford University Press                    010048TG10.1093/geront/41.6.799                        RESEARCH ARTICLE                            E-mail Versus Conventional Postal Mail Survey of Geriatric Chiefs                                          Raziano            Donna Brady                    MD                      a                                                Jayadevappa            Ravishankar                    PhD                      b                                                Valenzula            Duaré                                c                                                Weiner            Mark                    MD                      d                                                Lavizzo-Mourey            Risa                    MD, MBA                      e                          aDepartment of Medicine, Division of Internal Medicine, University of Pennsylvania, Philadelphia        bDepartment of Medicine, Leonard Davis Institute on Health Economics, and Institute on Aging, University of Pennsylvania, Philadelphia        cSchool of Arts and Sciences, University of Pennsylvania, Philadelphia        dDepartment of Medicine, University of Pennsylvania, Philadelphia        eDivision of Geriatric Medicine and Institute on Aging, University of Pennsylvania, Philadelphia                    Donna Brady Raziano, MD, Fellow, Geriatric Medicine, Department of Medicine, University of Pennsylvania, Ralston-Penn Center, 3615 Chestnut Street, Philadelphia, PA 19104. E-mail: Donna.Raziano@uphs.upenn.edu.                    1        12        2001            41      6      799      804                        23          7          2001                          19          2          2001                            The Gerontological Society of America        2001                    Purpose: This study compared the response time, response rate, and cost of two types of survey administration techniques: e-mail/web-based versus conventional postal mail. The main aim of the survey was to collect descriptive information on the existence of Acute Care for Elders units and their characteristics by surveying geriatric division chiefs. Design and Methods: Two randomized cohorts of geriatric division chiefs were formed to receive a survey either by electronic mail (n = 57) or by conventional postal mail (n = 57). If there was no response to the initial mailing, two follow-up mailings were sent to both groups using the original modality; a third follow-up was performed using the alternative modality. For each group, response rate and response time were calculated. The average total cost was computed and compared across two groups. Results: The aggregate response rate was 58% (n = 31) for the e-mail group versus 77% (n = 44) for the postal mail group. The overall average response time was shorter in the e-mail group, 18 days compared with 33 days for the conventional postal mailing group. The cost comparison showed that average cost was $7.70 for the e-mail group, compared to $10.50 per response for the conventional mail group. Implications: It appears that although the web-based technology is gaining popularity and leads to lower cost per response, the conventional postal method of surveying continues to deliver a better response rate among the geriatric medicine division chiefs. The web-based approach holds promise given its lower costs and acceptable response rate combined with the shorter response time.                    Electronic mail        Mail surveys        Survey techniques                              hwp-legacy-fpage          799                          hwp-legacy-dochead          RESEARCH ARTICLE                            Decision Editor: Laurence G. Branch, PhD            Recently, Internet-based survey techniques have been gaining popularity for health care research and medical applications (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000; Thomas, Stamler, Lafreniere, and Dumula 2000). The Internet provides a new survey technique to the geriatric research community in targeting the geriatric populations in a more cost-effective way. Despite the growing enthusiasm for Internet-based surveys (Lakeman 1997; Schleyer and Forrest 2000), the geriatric medical community seems to have fallen behind in the effective use of this technique. The current literature indicates that when using conventional survey techniques, such as postal surveys, researchers can expect a response rate ranging from 70% to 75% (Davis 1999; Jones and Pitt 1999). On the other hand, the response rates for electronic mail surveys have been found to vary from 34% to 76% (Eley 1999; Jones and Pitt 1999). These numbers, however, should be viewed with caution because there is a paucity of substantial comparison studies, and none are noted in the geriatric research community. Additionally, the nature of the sample or the target group, the conditions under which the survey was administered, and the content of the survey can have profound implications on response rate and effectiveness of Internet-based survey techniques (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000). Table 1  displays some of the benefits and drawbacks of a postal survey versus an e-mail/web-based survey (Houston and Fiore 1998; Jones and Pitt 1999; Lakeman 1997; Schleyer and Forrest 2000; Thomas et al. 2000).    We compared the attributes of an e-mail/web-based survey to a conventional postal survey. This study was part of a larger project on Acute Care for Elders (ACE) units and the main aim of the survey was to collect descriptive information on the existence of ACE units and their characteristics. We present a comparison of the response rate, response time, and administration costs between electronic mail and conventional postal mail surveys, targeted toward the geriatrics programs and division chiefs across the country.          Methods      The study population consisted of 114 chiefs of established geriatric medical divisions and geriatric teaching programs across the United States. The names, mailing addresses, and e-mail addresses for the study population were obtained by reviewing the following sources: the American Geriatrics Society (AGS) board of directors and task force committees, American Directory of Geriatric Academic Programs (ADGAP), and Geriatric Education Centers (GEC). Two groups, each containing 57 institutions, were formed by random assignment of the study population. The institutions were numbered from 1 to 114; then, with the help of random number tables, an institution was assigned to either e-mail group (group 1) or the conventional postal mailing group (group 2). The process was repeated until each group had 57 institutions. We obtained e-mail addresses for chiefs in group 1 via the following sources: Internet institution directory sites, professional medical society directories of the AGS, American College of Physicians, The Gerontological Society of America, GEC lists, ADGAP program directories, and the Association of American Medical Colleges Graduate Medical Education directory. We were able to obtain e-mail addresses for 40 chiefs by using these methods. For the remaining 13 chiefs, whose e-mail could not be obtained by these methods, we obtained e-mail addresses from their office assistants without disclosing the study. Four individuals who did not have an e-mail address were excluded from the e-mail survey group; thus, the adjusted denominator for group 1 was 53. For group 2 (n = 57), the most recent postal mailing addresses from the sources mentioned were used.              Survey Design        The survey instrument was designed to determine the prevalence and characteristics of ACE units among academic geriatric programs. The survey (see the Table A1 1) consisted of 17 multiple-choice questions, including questions on the existence, size, length of stay, and patient characteristics of the ACE unit and demographics.        To develop the web-based version of the survey, Claris Home Page 3.0 was used to format the survey with Hyper Text Markup Language (HTML). Functional testing of the web page was successful on Netscape and Internet explorer platforms on both Macintosh and Windows-based computers. Each question was reviewed for readability and ease of downloading. Downloading took only a few seconds, and the survey took about 5 minutes to complete. List boxes, radio buttons, and check boxes were provided because none of the questions were open ended; the format was simple, with consistent positioning of check buttons that made concentrating on the questions easy. The preliminary draft of the survey was reviewed within the Division of Geriatric Medicine, University of Pennsylvania. The faculty was asked to comment on the various aspects of the survey, such as question format, layout, wording, and duration. We responded by revising some of the questions for their content and omitting one question altogether.                    Survey Administration        Respondents in the e-mail group received a letter stating the purpose of the survey and our contact information. The survey could be accessed through the e-mail letter as an attachment in an HTML format and/or as a hyper link to our survey on the World Wide Web. The Uniform Resource Locator address for the location of the survey was not stated in the letter, but an access button was created for convenience and to restrict unsolicited responses. Thus, the e-mail group could respond to the survey by completing and returning the attachment, by accessing and completing the survey on web page, or by printing out the survey and mailing back the completed survey via postal mail or fax. In total, three follow-ups were done for the nonrespondents. For those who did not respond to the first e-mail survey request, surveys were re-sent twice at 5-day intervals. The third and final follow-up for the nonrespondents was performed through conventional postal mail (the alternative method).        Respondents in the conventional postal mail group received an introductory letter similar to group 1. The chiefs received a hard copy of the survey and a self-addressed, stamped return envelope. They were also given an option to return the survey via fax. For nonrespondents in group 2, three follow-ups were done in total. They were re-sent the postal mail survey twice, allowing about 20 days between each attempt. The third and final follow-up for the nonrespondents was performed by e-mail (the alternative method). It is important to note here that the allocated time space between subsequent follow-ups is different for the two groups. We had to take into account the longer institutional processing time for conventional mail. For each group, response rate and response time were calculated. The response time was defined as the number of days between the date the survey was sent out and the date it was received. Hence, the first mailing and subsequent follow-ups were treated as discrete events.        The overall costs included labor, supplies, and postal costs. Costs of the final follow-up by the alternative method were not included. The average total cost was computed and compared across two groups. Finally, paired t-test and chi-square statistics were used to compare the average group response rate and response time.                    Results      The e-mail group had four individuals with either incorrect or no e-mail address, thus reducing the total number of participants in this group to 53. Although these individuals were sent a postal mail survey, their responses were excluded for the purpose of this study. The first attempt received maximum responses in both groups, with a 39% response rate in the e-mail group and a 63% response rate in the conventional postal mailing group. For both groups, the response rate showed a decreasing trend during the second and third follow-ups. Key findings are summarized in Table 2 .              Response Rate        There was a significant (p < .001) difference in aggregate response rate of all three attempts, with 58% for the e-mail group and 77% for the conventional mail group. For the e-mail group, the total number of responses for the three attempts was 31 out of 53. Of these, 23 responses were returned via e-mail. Eight respondents (26%) in the e-mail group preferred to print out the survey and responded either through postal mail or fax. However, after the third follow-up attempt through conventional postal mailing follow-up, the total response rate was increased to 83% (n = 44) for the e-mail group. For the conventional mail group, total number of responses for the first three mailings was 44 out of 57 (77%). The third and final follow-up attempt, using e-mail, resulted in increasing the total responses to 46 (81%).        Following the completion of the study, the e-mail nonresponders were contacted to better understand the nonresponder behavior and influencing factors. Individuals who did not respond to all three e-mail attempts (n = 6), but completed a conventional postal mail survey, were contacted personally by telephone and asked why they did not respond via e-mail. The reasons reported included a higher level of comfort with the conventional mail survey (n = 1), unavailability of e-mail accounts (n = 2), and lack of technical savvy with the Internet and attachments (n = 3). Therefore, in group 1, in addition to four for whom we could not identify an e-mail address, three more individuals had but did not use e-mail accounts. Thus, approximately 12% of the e-mail group was inaccessible electronically.                    Response Time        There was a significant difference in the response time between the two groups (p < .001). The average response time for the e-mail group was 18 days, with 30% of the responses arriving within 24 hours after request. On the other hand, for the conventional postal mailing group, the average response time was 33 days. The first response from the conventional postal mailing group was received on the 15th day. This suggests that e-mail surveys are capable of generating much quicker responses compared with conventional postal surveys. For the e-mail group, the average electronic response time for the first attempt was 3.3 days. The average electronic response time for the second attempt was 4.7 days and for the third attempt was 2.3 days. For the conventional postal group, the average response time for the first mailing was 21 days. The average response time for the second mailing was 15.5 days and for the third mailing was 21 days.                    Cost Analysis        We calculated average cost per response for both groups. The total costs included the labor costs involved in survey preparation, follow-up, survey administration, data input, stationary cost, and postage cost. We derived these costs based on the cost estimates for our institution letterhead, large and regular mailing envelopes, hard copies of the survey, and postage for both our mailing of the envelopes and prepaid return envelopes. Long-distance telephone calls and Internet connection costs were not included. The labor cost differed for each activity: The development of an Internet-based e-mail survey required a research assistant with reasonable computer skills at $40.00 per hour, whereas for all other activities (data input, postal mailing survey administration, and survey preparation) the cost was $20.00 per hour.        The average cost per response was $10.50 for conventional postal mail group and $7.70 for the e-mail group. Thus, the conventional postal mail survey costs were 27% higher than the e-mail survey. This higher cost of a conventional postal mail survey can be attributed to the extra costs of stationary, postage, administration, and follow-up. However, for longitudinal surveys and surveys of large populations, cost of survey administration and follow-ups can be reduced significantly by using e-mail/web-based survey techniques. Table 3  presents the cost analysis for both groups.        For the e-mail group, the task of survey preparation was more expensive compared to administration and follow-up costs. It required 2 hours at the rate of $40 per hour for a computer professional to set up the web-based survey. The survey administration, follow-up, and data input costs were small ($60 in total) for the e-mail group. On the other hand, the task of survey preparation accounted for only a small portion of the total cost for the conventional postal mailing group, with $17 for institutional letterheads and other stationary. In this case, survey administration, follow-up, and data input were the more expensive items. Survey administration required a total of $197 ($57 in postage, 2 hours of labor at the rate of $20 per hour, and $100 for obtaining the ADGAP address list). Survey follow-up cost was $150 for two follow-up attempts, and an additional $100 was required for data input. The costs of the fourth attempt by the alternative method were not included in the cost analysis. However, it should be noted that costs are extremely sensitive to the complexity and the length of the survey. A lengthier survey would increase the administration, follow-up, and data input costs significantly in group 2. Actual sending of the e-mail surveys did not take as long as preparing and sending the postal mail surveys, but it did take much longer to sort out problems with incorrect e-mail addresses, redirects, and miscommunication via e-mail messaging and other incidentals.                    Discussion      Although the postal mail survey had a higher response, e-mail questionnaires are a promising way to survey the division chiefs of geriatric medicine because of lower response time and costs. Currently there are many reports available that describe the methods involved in conducting an e-mail or web-based survey. However, little knowledge exists about the effectiveness of this technique among geriatric chiefs and the barriers faced in implementing these techniques.      Earlier studies have indicated the ways in which e-mail or web-based surveys could be used across different health professionals. Researchers at Temple University, Philadelphia, developed a web-based survey with 22 questions to study the use of the Internet in clinical practice. They recruited 450 dentists from an e-mail address list to receive this web-based survey. Participants were sent one e-mail, and a maximum of three e-mail follow-ups were sent to the nonresponders. E-mail response rates, strictly defined as an e-mail only, were 32.9%, 50.2%, 57.1%, and 64.4% for the chronological attempts. The addition of alternative responses, such as fax or mail, raised total response rate to 74.2%. The cost comparison indicated almost 50% reduction in total cost for e-mail surveys compared with conventional mail surveys ($1,916 compared with a hypothetical postal cost of $3,092). A minimum of 275 e-mail responses were recommended to break even on the costs of e-mail and to make it a more economical method than postal mail (Schleyer and Forrest 2000). This study differs from our study because recruited participants were established e-mail users.      The process of obtaining e-mail addresses through mailing address lists was extremely labor-intensive, despite the availability of a number of Internet search engines. This finding was less of an issue in other studies, such as the dental study described, and suggests that geriatrics is lagging other medical subspecialties in creating searchable electronic databases of its physicians. The existence of such a database would have made the electronic surveys even more cost-effective.      A study of university staff done in the United Kingdom compared the response timing and rates using electronic and paper methods. E-mail health surveys that required web-based submission were sent to 200 people, and 100 other people received traditional postal surveys. The postal mail survey received a response rate of 72%, followed by 32% with respect to e-mail and 19% for the World Wide Web. Although e-mail and web-based surveys were quicker and cheaper by half, the authors concluded that the higher response rate made postal mail more preferable (Jones and Pitt 1999). Caution must be exercised, because the overall cost is extremely sensitive to the type and scale of the target population. If the Internet usage of the target population is low, a great deal of effort would be expended trying to locate e-mail addresses for people who are not reachable electronically; among those who could be reached, response rates could be poor as a result of inadequate Internet skills.      An experiment comparing the response rates and response time for different approaches to mailing highlighted the strengths and weakness of e-mail surveys. In this study, the respondents recruited from the Internet received postal mail or e-mail surveys. The respondents were randomly assigned to one of five groups. Group 1 consisted of regular mail with no prenotification, no incentives, and no reminders (n = 202); group 2 consisted of regular mail with prenotification, incentives, and reminders (n = 107); and groups 3 (n = 60) and 4 (n = 122) were e-mail replications of groups 1 and 2. The final group (n = 172) is an international group, otherwise the same as group 4. The results showed that e-mail surveys had much lower response time (2–3 days vs 3 weeks) and higher response rates (40% in group 1 vs 45% in group 3). In the prenotified groups the rates increased to 63% and 83%, respectively, and were much less expensive compared to conventional mail surveys (Mehta and Sivadas 1995).      None of the previously reported survey research has directly compared the postal mail and the electronic web-based mail in a random selection method in the geriatric community. In addition, all previous studies utilized established e-mail address lists, implying that the potential respondents were known to have both a working e-mail address and Internet access. Open recruitment of participants as well as guaranteed Internet users as participants has commonly occurred in other published research (Eley 1999; Mehta and Sivadas 1995). Our study differs in this respect since we had no prior knowledge of the computer skill level of the participants or their level of Internet knowledge.      The higher response rate for the conventional mail group was countered by a longer response time in our study. On the other hand, the administration of the e-mail surveys resulted in faster responses and was more cost-effective albeit with a lower overall response rate. It should be noted that, from the first three attempts, we received 31 responses from the e-mail group and 44 responses from the conventional mail group. The electronic response was acceptable but still not as high as expected. The response rate of 58% for the e-mail group is less than other published results that indicate a response rate closer to 70% (Lakeman 1997; Mehta and Sivadas 1995). This discrepancy may have been due to the fact that our target population is unlike the other research populations that used existing Internet or e-mail user lists. Also, it is important to note that our response rates, response time, and related cost for the chiefs may not be generalized to other physicians. To minimize sampling bias in our small preselected population, randomization of survey recipients was used to make the two groups comparable. Despite moderate response rates, the rapid turnaround time could make e-mail the superior method for eliciting data in the future as more and more geriatric chiefs access electronic mail accounts. In tune with earlier findings, the electronic mail response rate for our survey was comparatively lower. We conclude that, even with this limitation, e-mail should be the preferred method if a prompt response is a primary goal. There may also be better control over the respondents and quality of the data with e-mail. Especially if traditional postal mailing is used as a salvage step, the e-mail/web-based survey follow-up reminders were easier and more cost-effective. The cost saving associated with electronic surveys must be weighed against the poorer expected response rate.      Practice Concepts      The Forum      Book Reviews                                                  Table 1.                                 Comparison of E-mail/Web-Based and Conventional Postal Mail Survey                                                              E-mail/Web-Based Survey                Conventional Postal Mail Survey                                            Can be cost-effective in established e-mail users.                Targeted population is more difficult to monitor for receipt and completion of survey.                                            Data are collected in electronic format, thus minimizing data input effort.                Requires more effort in terms of data input.                                            Survey administration is less costly.                Survey administration and follow-up are costly.                                            Hypertext markup language can be designed to suit preferences of the respondent.                It is more time intensive.                                            Accessibility is restricted to populations that have access to Internet or e-mail.                Wide range of population accessibility.                                            Requires some basic skills of computer.                Basic computer skills are not necessary.                                                                          Table 2.                                 Response Rate and Response Time                                                                                              E-mail/Web-Based (n = 53)Consecutive Attempts                                                                                Conventional Postal Mail (n = 57) Consecutive Attempts                                                                                            Measure                                First                Second                Third                Fourth                                First                Second                Third                Fourth                                            Total response                                21                7                3                13                                36                5                3                2                                            Average response time (in days)                                3.3                4.7                2.3                                                21                15.5                21                                                            Response rate for each attempt                                39%                22%                10%                50%                                63%                23%                19%                12%                                                                            (21/53)                (7/32)                (3/29)                (13/26)                                (36/57)                (5/21)                (3/16)                (2/13)                                            Total average response time                                18 Days                                                                                33 Days                                                                                            Mode of response                E-mail = 52%                                                                                E-mail = 3.5%                                                                                                                            Mail = 45%                                                                                Mail = 96.5%                                                                                                                            Fax = 3%                                                                                Fax = 0                                                                                                                                          Table 3.                                 Cost Analysis                                                              Types                E-mail Group (n = 53)                Postal Mail Group (n = 57)                                            Survey preparation                $80                $17                                            Survey administration cost                $120                $197                                            Survey follow-up cost                $40                $150                                            Data input cost                00                $100                                            Total cost                $240                $464                                            Average cost per response                $7.70 (n = 31)                $10.50 (n = 44)                                                                          Table A1.                                 Institute on AgingDivision of Geriatric MedicineDepartment of MedicineUniversity of PennsylvaniaNATIONAL ACE UNIT SURVEY                                                              1. Name of the hospital _____________________________________________________________________________                                                                                                                            2. Address _________________________________________________________________________ Zip: _________                                                                                                                            3. Location of the hospital                                                                                                                                            □ Rural                □ Urban                □ Suburban                                                                            4. Is your hospital (check all that apply)?                                                                                                                                            □ For Profit                □ Non Profit                □ Community Hospital                                                                                            □ Community Teaching Hospital                                □ University based Teaching                                                                                            □ Non-teaching                                                                                                            5. Do you have an Acute Care for Elderly (ACE) unit?                                                                                                                                            □ Yes                □ No                                                                                                              If answer to question 5 is NO, you may STOP here but please return the survey                                                                                                                                            6. Year ACE unit was established: _ _ _ _                                                                                                                            7. Is your ACE unit limited to community dwelling (e.g., non-nursing home) patients only?                                                                                                                                            □ Yes                □ No                                                                                            8. What is the average daily census on the ACE unit?                                                                                                                                            □ <10                □ 10-15                □ 16-20                                                                                            □ 21-25                □ >25                                                                                            9. Do you have telemetry beds within your ACE unit?                                                                                                                                            □ Yes                □ No                                                                                            10. Average length of stay in days ________________                                                                                                                            11. Patients average age group                                                                                                                                            □ 60-65                □ 66-70                                                                                                            □ 71-75                □ 76-80                □ >80                                                                            12. Are patients from the following settings routinely admitted to your ACE unit (Check all that apply)?                                                                                                                                            □ Nursing Home                □ Surgical Floor                                                                                                            □ Emergency Room □ Non-ICU medical unit □ Living at home                                                                                                            13. Please check the top three diagnoses in your ACE unit                                                                                                                                                                            1                2                3                                            Congestive Heart Failure                                                ____                ____                ____                                            Pneumonia & Respiratory infection                                                ____                ____                ____                                            Kidney & Urinary Tract Infection                                                ____                ____                ____                                            Metabolic disorders                                                ____                ____                ____                                            Mental status change                                                ____                ____                ____                                            Gastrointestinal hemorrhage                                                ____                ____                ____                                            Sepsis                                                ____                ____                ____                                            Cerebrovascular events                                                ____                ____                ____                                            Other ________________________________________                                                ____                ____                ____                                            14. Are the following health care professionals allowed to admit patients to your ACE unit (Check all that apply)?                                                                                                                                            □ Attending in Geriatric Medicine                                                □ Attending in Internal Medicine                                                                            □ Attending in Family Practice                                                □ Attending in Surgery                                                                            □ Attending in Psychiatry                                                □ Attending in Rehabilitation Medicine                                                                            □ Community Physicians                                                                                                            15. What is the average nurse to patient ratio in your ACE unit?                                                                                                                                            □ 1:4                □ 1:5                □ 1:6                                                                                            □ 1:7                □ 1:8                □ 1:9                □ 1:10                                                            16. Which of the following does your ACE unit have? (Check all that apply)                                                                                                                                            □ Pharmacist                                                □ Physical Therapist                                                                            □ Occupational therapist                                                □ Social worker                                                                            □ Geriatric psychiatrist                                                □ Speech pathologist                                                                            □ Nutrition/dietitian specialist                                                                                                            17. Would you be interested in participating in a randomized control trial to evaluate the outcomes of patients admitted to ACE units?                                                                                                                                            □ Yes                □ No                                                                                                              THANK YOU                                                                                                                                                                    Support for this study was provided by an American Federation on Aging research geriatric fellowship.                      Davis R. N., 1999. Web-based administration of a personality questionnaire: Comparison with traditional methods. Behavior Research Methods Instruments:31-572.                     Eley S., 1999. Nutrition research using electronic mail. British Journal of Nutrition 81:413-416.                     Houston J. D., Fiore D. C., 1998. Online medical surveys: Using the Internet as a research tool. MD Computing 15:116-120.                     Jones R., Pitt N., 1999. Health surveys in the workplace: Comparison of postal, email and World Wide Web methods. Occupational Medicine (Oxford) 49:556-558.                     Lakeman R., 1997. Using the Internet for data collection in nursing research. Computers in Nursing 15:269-275.                     Mehta R., Sivadas E., 1995. Comparing response rates and response content in mail versus electronic mail surveys. Journal of the Market Research Society 37:429-439.                     Schleyer T. K. L., Forrest J. L., 2000. Methods for the design and administration of web-based surveys. Journal of the American Medical Information Association 7:416-425.                     Thomas B., Stamler L. L., Lafreniere K., Dumula R., 2000. The Internet: An effective tool for nursing research with women. Computers in Nursing 18:13-18.