HEALTH EDUCATION RESEARCH

Vol.25 no.3 2010
Pages 464–477
Advance Access publication 15 January 2010

Health literacy among adults: a study from Turkey

H. Ozdemir, Z. Alper, Y. Uncu and N. Bilgel*
Department of Family Medicine, Uludag University Faculty of Medicine, 16059 Bursa, Turkey
*Correspondence to: N. Bilgel. E-mail: nazan@uludag.edu.tr
Received on May 22, 2009; revised for publication November 5, 2009; accepted on December 7, 2009

Abstract

Introduction

Patients’ health literacy is increasingly recognized as a critical factor affecting health
communication and outcomes. We performed
this study to assess the levels of health literacy
by using Rapid Estimate of Adult Literacy in
Medicine (REALM) and Newest Vital Sign
(NVS) instruments. Patients (n 5 456) at
a family medicine clinic completed in-person
interviews, REALM and NVS tests which
were translated into the Turkish language by
translation-back translation process. Additional
questions regarding demographic characteristics were also collected. The mean scores (mean
% standard error) for REALM and NVS were
60.29 % 0.32 and 2.60 % 0.08, respectively.
The REALM test scores showed that 2.7% had
inadequate (less than or equal to 6th grade),
38.6% marginal (7th to 8th grade) and 58.7%
(greater than or equal to 9th grade) adequate
health literacy. The NVS test score revealed
a proportion of 28.1% had adequate health literacy. Educational attainment was the most important demographic characteristic found to be
related to the health literacy. Reading and vocabulary skills were better than numerical
capabilities. Female, primary school educated
and poor economic condition participants and
those who were older had the lowest scores in
both the tests.

Health literacy is a concept that is both new and old.
The term of health literacy has been used in health
literature for at least 30 years [1]. There are various
definitions of health literacy in the literature, all
with similarities. One of them is: ‘The degree to
which the individuals have the capacity to obtain,
process, and understand basic health information
and services needed to make appropriate health
decisions is called health literacy’ [2–4]. This short
and succinct approach follows the definition of literacy as used in the 1992 National Adult Literacy
Survey in the United States, which defined literacy
as ‘using printed and written information to function in society’ [5]. The authors defined functional
health literacy as the ability to apply reading and
numerical skills in a health care setting. These skills
include the ability to
(i)

Read consent forms, medicine labels, inserts
and other health care information;
(ii) Understand written and oral information given
by health care personnel;
(iii) Act upon necessary procedures and directions
such as medication and appointment schedules.
In the World Health Organization (WHO) [6]
health promotion glossary, health literacy is defined
as ‘the cognitive and social skills which determine
the motivation ability of individuals to gain access
to understand and use information in ways which
promote and maintain good health’. According to
the WHO, health literacy implies the achievement of a level of knowledge, personal skills and

Ó The Author 2010. Published by Oxford University Press. All rights reserved.
For permissions, please email: journals.permissions@oxfordjournals.org

doi:10.1093/her/cyp068

Health literacy
confidence to take action to improve personal and
community health by changing personal lifestyles
and living conditions. Thus, health literacy means
more than being able to read pamphlets and make
appointments. According to the Committee on
Health Literacy of the Institute of Medicine:
‘Health literacy is of concern to everyone involved
in health promotion and protection, disease prevention and early screening, health care maintenance,
and policy making. Health literacy skills are needed
for dialogue and discussion, reading health information, interpreting charts, making decisions about
participating in research studies, using medical
tools for personal or family health care, calculating
timing or dosage of medicine, or voting on health or
environment issues’ [4]. By improving people’s access to health information, and their capacity to use
it effectively, health literacy is critical to empowerment. Health literacy is itself dependent upon more
general levels of literacy. Poor literacy can affect
people’s health directly by limiting their personal,
social and cultural development, as well as hindering the development of health literacy itself.
In the United States, this term is used particularly
to describe and explain the relationship between the
patient’s literacy levels and their ability to comply
with prescriptions, appointment cards, drug labels
and directions for health care [7].
These definitions present health literacy as a set
of individual capacities that allow the person to
acquire and use new information. Literacy may improve with educational programs or decline with
aging or cognitive impairment [7]. Some researchers argue that if health literacy is the ability to function in the health care environment, it must be
related to characteristics of both the individual
and the health care system. From this perspective,
health literacy is a dynamic state and may vary
depending upon the health problem, the health care
provider and the health system [8]. Some other
researchers accept health knowledge as a part of
health literacy. The Institute of Medicine expert
panel divided the domain of health literacy into four
categories [4]. They are listed below:
(i)

Cultural and conceptual knowledge

(ii)

Oral literacy, including speaking and listening
skills
(iii) Print literacy, including writing and reading
skills
(iv) Numeracy

As is seen from the literature, there is no common
definition for the term health literacy, and this fact
raises a problem for measuring it because no one
refers to the same underlying construct. If health
literacy is accepted as only an individual’s capacity,
measuring a person’s reading ability and vocabulary are appropriate. On the other hand, if health
literacy is accepted as a relationship between an
individual’s communication capacities, the health
care system and society, measuring only the individual capacities may not be adequate. If knowledge is a part of health literacy, this too must be
measured.
However, today, health literacy is something that
has a broader meaning and researchers are still
debating on this issue and searching for new, appropriate, reliable and practical instruments for
measuring and classifying health literacy in an
objective manner.
Although health literacy is a complex and multifaceted construct, researchers have developed several instruments to assess health literacy [9–12].
Existing health literacy assessment tools and their
relevance to examine health literacy onto the four
categories of the Institute of Medicine’s classification are listed below [4]:
(i)

Test of Functional Health Literacy in Adults
(TOFHLA)—print literacy and numeracy
(ii) Rapid Estimate of Adult Literacy in Medicine
(REALM)—print literacy
(iii) Health Activities Literacy Scale (HALS)—
print literacy and numeracy
(iv) Newest Vital Sign (NVS)—print literacy and
numeracy

TOFHLA is a two-part test available in English
and Spanish [9]. The first part provides participants
465

H. Ozdemir et al.
with medical information, instructions on a prescription label or instructions for a diagnostic procedure.
Participants review the scenarios and then answer
the questions. In the second part of the test, participants read the given text passages about medical
topics with selected words deleted and replaced
with blank spaces. Participants must fill in the blank
spaces using words from a multiple-choice list.
TOFHLA scores range from 0 to 100 with higher
scores indicating better literacy. Scores <60 represent inadequate literacy. A short version of
TOFHLA (S-TOFHLA) is also available. TOFHLA
is the instrument most often used for the assessment
of health literacy and has good psychometric
properties, but the length of time for administration
(18–22 min for full and 7–10 min for short version)
precludes its use in busy primary care settings.
The REALM can be administered quickly
(<5 min) and is available in English and Spanish
[10]. It is a word recognition and pronunciation test
composed of 66 medical terms arranged in order of
pronunciation difficulty, starting with simple onesyllable words (e.g. pill and eye) and ending with
multi-syllable words (e.g. antibiotics and potassium). While the participant reads down the list,
the examiner scores the number of words that are
pronounced correctly. For every correct word, one
point is given. The sum of the points correspond to
four categories of grade equivalent reading levels:
0–18 being less than or equal to 3rd grade; 19–44
being equal to 4th to 6th grade; 45–60 being equal
to 7th to 8th grade and 61–66 being greater than or
equal to 9th grade. Although the REALM and the
TOFHLA are useful tools for the assessment of
health literacy both in clinical and in community
settings neither the REALM nor the TOFHLA is
eligible for capturing the full complexity of the
construct of health literacy. Both of them are measures of basic print literacy using health-related
terms, some degree health-related texts, and
TOFHLA also includes a measure of numeracy.
However, health literacy includes more than word
recognition, text comprehension and numeracy
skills. A full set of skills and knowledge associated
with health literacy, tasks that are not limited to the
health care system and comprise a broad spectrum
466

of activities in a variety of contexts that are defined
by the Institute of Medicine could not be assessed
by measures of basic print literacy. Therefore, the
results of REALM and TOFHLA must be interpreted cautiously [4].
The HALS test includes prose, quantitative and
document items in five health-related areas [11]:
health promotion, health protection, disease prevention, health care and maintenance and systems
navigation. The full length of HALS yields a score
from 0 to 500 in a five-point Likert scale and takes
;1 hour to complete. Despite its potential value for
assessing health literacy in a broader term, the
length of the HALS will prohibit its use in most
research studies.
The NVS is a quick screening tool capable of
reliably testing individuals for low health literacy
[12]. It is available in English and Spanish. Participants extract information from an ice cream nutritional label and then are required to answer six
questions interpreting the information from the
label. For each correct answer, one point is given,
and the sum of the points indicates the level of
health literacy. A score of <4 indicates limited
health literacy. The quantitative questions on the
NVS require both reading comprehension and
mathematical ability, and these abilities are directly
correlated with understanding and managing basic
health information.
There are several studies in Western literature that
assessed health literacy by using REALM and NVS.
An observational study involving face-to-face interviews with 1796 veterans who received primary care
services at one of four large VA medical centers in
the United States revealed that 4.2% of the participants had inadequate (less than or equal to 6th grade),
17.0% marginal (7th to 8th grade) and 78.4%
adequate (greater than or equal to 9th grade) health
literacy regarding REALM test scores [13]. Another
survey among 992 adults aged 18–45 years showed
5% had a level of Grade 3 or below, 6.6% in the 4th
through 6th grade range, 16.3% in the 7th through
8th grade range and 72.1% at the level of Grade 9 or
higher [14]. In a study for measuring the preventability of hospital admission among 400 veterans, the
health literacy levels were found to 6.8% in the

Health literacy
Grade 3 or below, 17.7% in the 4th through 6th grade
range, 33.2% in the 7th through 8th grade range and
42.3% at the level of Grade 9 or higher [15]. A crosssectional study involved 98 adults who identified
themselves as the primary caregiver of preschool
children found health literacy levels as follows:
0–3rd grade = 9.2%, 4–6th grade = 26.5%, 7–8th
grade = 32.7% and >9th grade = 31.6%) [16].
None of the existing measures of health literacy
examines oral literacy and cultural and conceptual
knowledge. Current assessment tools cannot differentiate among lack of background knowledge in
health-related domains, lack of language and types
of materials familiarity with cultural differences in
approach to health and health care [4].
In their daily lives, adults are likely to face a broad
range of health literacy tasks from reading an article
about preventive health practices to selecting and
buying an over-the-counter medication. As parents,
they must manage their children’s health care. Older
adults must make decisions about prescription drug
benefits. All these activities require the ability to
read and understand written and printed information. Many studies have revealed that low health
literacy is associated with poor health communication, health outcomes, increased hospitalization
rates, less frequent screening for early detectable
diseases such as cancer and disproportionately high
morbidity and mortality rates [17–25].
From this perspective, health literacy is the ability
to function in the health care environment; health
literacy must apply not only to the individual characteristics but also to the health care system. It would
therefore be useful to summarize the health delivery
system and challenges to health faced in Turkey. The
Ministry of Health (MoH) is the main government
body responsible for policy making in the health
sector and implementing national health strategies
through programs and direct provision of health
services [26, 27]. MoH is the unique provider of
primary health care through an extensive network
of health facilities (;5700 health centers and
26 000 health posts) [26]. Primary health care is
universal and free of charge for everyone. With
regard to the universal preventive care services, great
achievements have been made in controlling com-

municable diseases and reducing infant and maternal
deaths. Besides routine health educational activities,
which are part of the primary health care system,
many public health educational programs have
been implemented through several campaigns,
which were organized with the collaboration of international organizations. In summary, primary preventive health services in Turkey are well organized,
accessible to everyone and effective. The public sector accounts for 92% of the hospital capacity in
Turkey. MoH and the universities are the major
public providers of inpatient hospital care [27].
In the year 2003, MoH launched the Health Transformation Program, which is aimed at developing
universal health insurance coverage, implementation
of family medicine at the primary level and giving
more autonomy to hospitals [27]. Patients’ rights,
informing patients at every stage of their treatment
and obtaining informed consent from patients, all
these are new issues that have gained importance
within the Health Transformation Program. Traditionally in Turkey, physicians are accepted as persons to whom the patients should obey; however,
with the changes in the health system, with improvements in social, economic and cultural areas and
with globalization, this traditional view has been
changing, and patients now want to be part of the
decisions regarding their health. Furthermore, within
the health system patients become obligated to read,
understand, fill out and sign many forms that they
were not used to completing. Therefore, health literacy has become an important issue that was not previously in the agenda of health care providers.
Assessment of health literacy is a neglected area in
Turkey, and research concerning this matter is very
rare. Most of the published literature depends on
knowledge about special health issues and health literacy is accepted as the equivalent of general literacy
level. Our study may be the first one that has used the
specific health literacy measures for the assessment.

Materials and methods
This cross-sectional study was conducted in a suburban area of a metropolitan city in Turkey.
467

H. Ozdemir et al.
According to the last census, the total population of
this area is ;41 373 (19 695 female).
The Family Medicine Clinic where the study
was conducted serves as a primary health care unit
and as a training facility for medical students and
provides a wide range of protective and therapeutic primary health care services free of charge.
Participants of the study were chosen among the
579 adult patients who had visited the clinic from
1 February 2008 to 1 April 2008. Patients who
were illiterate (could not read and write) (N = 57),
those with previously known cognitive impairments
(N = 23) and who did not want to participate
(N = 43) were excluded. All the participants gave
their consent for participation, and assessment of
health literacy was performed after the examination
in a different room with respect to the participant’s
privacy and anonymity.

Study instruments
We used two different tests for assessment of health
literacy: REALM and NVS. These tests were not
available in Turkish; therefore, first we performed
the translation and back translation process. A panel
of two English teachers, one native Turkish and
a native British with a knowledge of the Turkish
language, translated both instruments from English
into Turkish and then reversed the process and
formed the Turkish version of the instrument. We
used the same 66 medical terms in the original
REALM instrument, but they were rearranged in
order of pronunciation difficulty in the Turkish language, starting with simple one-syllable words (e.g.
pill, dose, eye and flu) and ending with multisyllabic words (e.g. osteoporosis, antibiotics, potassium, obesity and depression). Both the study
tests were administered by the same person (author
H.O.) after giving the necessary directions. The
directions for the REALM test were as follows: ‘I
want to learn what medical words you are familiar
with. You should look at this list of words, beginning here with the first word on the list. Say all the
words you know. If you come to a word you don’t
know or you cannot read, skip it and try the next
one.’ Words pronounced correctly and without any
468

deletions or additions to the beginning or ending of
the word were counted. Dictionary pronunciation
was accepted as the scoring standard if a doubt occurred. The scoring was performed by the examiner
at the same time of the reading process. After the
REALM test, the NVS test was administered. A
laminated copy of the nutrition label of an ice cream
container was given to the patient, and he or she
was asked to read it carefully. Then, a series of six
questions were asked about it. During this process,
patients were allowed to retain the laminated copy
of the label so they could refer to it while answering
the questions. The questions were asked orally, and
the responses recorded on a special score sheet that
contained the correct answers. The number of correct answers gave the health literacy level. We used
the same scoring grades of the original REALM and
NVS as follows: REALM: 0–18, less than or
equal to 3rd grade; 19–44, equal to 4th to 6th grade;
45–60, equal to 7th to 8th grade and 61–66, greater
than or equal to 9th grade. NVS: 0–1 suggests
high likelihood (>50%) of limited literacy; 2–3
indicates the possibility of limited literacy and
4–6 indicates adequate literacy.
We also collected some demographic data of the
participants such as age, gender, educational attainment, occupation and economic situation. Regarding health literacy, three additional questions were
asked. These were:
(i)
(a)
(b)
(c)
(d)
(ii)
(a)
(b)
(c)
(iii)
(a)

If the health care provider gives me forms to
fill out .
I always read them and fill out by myself.
I ask for my accompanying person’s help to
read and fill them out.
I ask for the health personnel’s help to read
and fill them out.
Other . .
I understand the printed material which is
given to me by the health care provider .
Always
Sometimes
Never
If the health care provider gives you some
printed material .
I read it.

Health literacy
(b) I ask for someone’s help to read it.
(c) I put it into trash without reading.
(d) Other . .

Data analysis
We used mean, standard deviation (SD), standard
error (SE) of the means and variance analysis to
summarize the participants’ demographic characteristics and their performance on the tests. Reliability
was assessed in terms of internal consistency (Cronbach a). We calculated the correlation (Pearson’s r)
between scores on REALM and NVS. Sensitivities,
specificities, likelihood ratios and cutoff points of the
tests were calculated by obtaining two dummy variables from the responses given to the statement: ‘I
understand the printed material given to me by the
health care provider’ and accepting the ‘always’
response as adequate and ‘sometimes or never’
responses as inadequate health literacy.

Results
Sociodemographic characteristics of the
study group
A total of 456 patients participated in the study. The
mean age of the participants was 36.21 6 12.61
years (mean 6 SD). Female participants consisted
60.5% of the study group. Most of the participants
were primary school educated. Economic situation
was reported as middle by 57.9% of the participants.
Types of occupation vary among male participants
but most of the female participants (76.8%) were
housewives. Table I shows the demographic characteristics and scores of the participants on the
Turkish language versions of REALM and NVS.

The REALM test
The mean score (mean 6 SE) for REALM
was 60.29 6 0.32 and the mean completing
time of the REALM test was 2.59 6 0.05 min.
Eight terms of the REALM test were pronounced correctly by all the participants and they
were: eye, prescription, nutrition, nausea, medica-

Table I. Demographic characteristics and test scores
Characteristics and scores
Age (years)
Mean (SD)
Range
Gender, n (%)
Male
Female
Education, n (%)
Primary school
Secondary school
High school
University
Reported economic situation, n (%)
Good
Middle
Poor
Occupation, n (%)
Housewife
Labor worker
Self-employed
Student
Salesman
Employee
Retired
Civil servant
Test scores (mean 6 SEM)
REALM
NVS
Test scores (median)
REALM
NVS
Test scores (minimum–maximum values)
REALM
NVS

36.2 6 12.6
17–72
180 (39.5)
276 (60.5)
188 (41.2)
84 (18.4)
132 (28.9)
52 (11.4)
180 (39.5)
264 (57.9)
12 (2.6)
216 (47.4)
48 (10.5)
48 (10.5)
48 (10.5)
44 (9.7)
32 (7.0)
16 (3.5)
4 (0.9)
60.2982 6 0.32
2.6053 6 0.08
63
2
18–66
0–6

tion, occupation, sexually and obesity. The first
10 terms that were missed and the percent of participants who had not pronounced correctly were as
follows: menstrual (49.1%), osteoporosis (39.5%),
appendix (36.8%), arthritis (36.8%), inflammatory
(27.2%), incest (26.3%), syphilis (23.7%), caffeine
(22.8%), potassium (20.2%) and rectal (20.2%).

The NVS test
The mean score (mean 6 SE) for NVS was
2.60 6 0.08 and the mean completing time of the
NVS test was 6.28 6 1.27 min. The internal
consistency of the NVS was good (Cronbach
469

H. Ozdemir et al.
a = 0.70) as was the criterion validity (r = 0.52;
P < 0.01). The most correctly answered question
of the NVS test was the fifth question, which concerns the allergy to peanuts, and 73.7% of participants gave the right answer. The following sixth
question was connected to the previous question
and answered correctly by 71.1%. The third question, which was about the intake of saturated fat,
was answered correctly only by 8.8% of the participants, and finally, the first and second questions of
the NVS test were correctly answered by roughly
one-third of the participants.
Distribution of the study group regarding their
health literacy level is shown in Table II.

Comparison of the REALM and NVS scores
According to the REALM test score, 58.7% of the
study group had adequate health literacy, whereas
the NVS test score revealed a proportion of 28.1%.
Therefore, we can say that the medical word recognition and pronunciation capacity of the study
group was better than the numerical and reasoning
skills. Figure 1 shows the distribution and the scatter plot of REALM and NVS scores.

Test scores according to gender, age groups,
education and economic situation are shown in
Table III.
We found a statistically significant relationship
between age, educational attainment, economic
condition and gender and test scores of both
REALM and NVS tests. Females, participants
with primary education, participants who reported
poor economic situations and older participants
had lower scores for both the tests. The gap among
male and female participants in terms of both test
scores was due to the gap between educational
attainment; hence, we found statistically significant differences among males and females according to their educational attainment (Pearson
v2 = 44.420; df = 3; P < 0.05); female participants were less educated. Similar differences were
found in terms of age and educational attainment
(Pearson v2 = 110.467; df = 9; P < 0.05), and we
accepted age and sex as cofounding factors.

Responses to additional statements
The responses regarding three statements that are
thought to be related to health literacy and their

Table II. Distribution of participants by REALM and NVS scores
NVS score

Total

0–1 (high likelihood
of limited literacy)

REALM score

0–18; Grade <3

19–44; Grade 4–6

45–60; Grade 7–8

61–66; Grade >9

Total

470

N
%
%
N
%
%
N
%
%
N
%
%
N
%

within REALM score
within NVS score
within REALM score
within NVS score
within REALM score
within NVS score
within REALM score
within NVS score

2–3 (possibility of
limited literacy)

4–6 (adequate literacy)

4
100.0
4.2
8
100.0
8.3
52
29.5
54.2
32
11.9
33.3
96
21.0

—
—
—
—
—
—
96
54.6
41.4
136
50.7
58.6
232
50.9

—
—
—
—
—
—
28
15.9
21.9
100
37.4
78.1
128
28.1

4
100.0
0.9
8
100.0
1.8
176
100.0
38.6
268
100.0
58.7
456
100.0

Health literacy
160

80

140
60

100

Frequency

Frequency

120

80
60
40

40

20

20
0

0

,00

1,00

2,00

3,00

4,00

5,00

6,00

18,00

48,00

NVS Scores

55,00

60,00

65,00

REALM Scores

70

Total score on REALM Turkish

60

50

40

30

20

10
-1

0

1

2

3

4

5

6

7

Number correct on NVS Turkish

Fig. 1. Distribution of the scores and scatter plot of REALM and NVS.

relationship with the scores of both tests are shown
in Table IV.
Participants who reported that they always read
and filled out the forms by themselves, that always
understand the printed material and that they read
the given printed material got the highest scores in
both tests.

Receiver operating characteristic analysis
Area under the receiver operating characteristic
curve for predicting adequate health literacy was
0.74 [95% confidence interval (CI), 0.65–0.75;
P < 0.001] for the REALM Turkish version and
0.67 (95% CI, 0.63–0.73; P < 0.001) for the
NVS Turkish version. The cutoff points for the
471

H. Ozdemir et al.
Table III. Test scores by education, gender, economic situation and age groups
N

ANOVA

Lower

Upper

F

Significance

180
276
180
276

61.711
59.376
3.000
2.347

5.023
7.601
1.743
1.570

0.374
0.457
0.129
0.094

60.972
58.476
2.743
2.161

62.450
60.277
3.256
2.534

13.208

0.000

17.207

0.000

Education
Primary
Secondary
High
University

188
84
132
52

56.021
61.619
63.515
65.461

8.279
3.350
2.545
1.092

0.603
0.365
0.221
0.151

54.830
60.891
63.076
65.157

57.212
62.346
63.953
65.765

65.129

0.000

Primary
Secondary
High
University

188
84
132
52

1.595
2.333
3.636
4.076

1.382
1.175
1.458
1.152

0.100
0.128
0.126
0.159

1.396
2.078
3.385
3.756

1.794
2.588
3.887
4.397

83.008

0.000

Economic situation
Good
180
Middle
264
Poor
12

63.022
58.712
54.333

3.743
7.791
2.994

0.278
0.479
0.864

62.471
57.767
52.430

63.572
59.656
56.236

29.595

0.000

Good
Middle
Poor

180
264
12

3.422
2.075
2.000

1.644
1.452
1.705

0.122
0.089
0.492

3.180
1.899
0.916

3.664
2.251
3.083

42.014

0.000

Age groups (years)
15–24
84
25–34
136
35–44
124
45+
112

62.190
62.235
59.161
57.785

3.129
4.382
8.948
7.392

0.341
0.375
0.803
0.698

61.511
61.492
57.570
56.401

62.869
62.978
60.751
59.170

13.088

0.000

15–24
25–34
35–44
45+

3.809
2.705
2.032
2.214

1.540
1.205
1.775
1.662

0.168
0.103
0.159
0.157

3.475
2.501
1.716
1.903

4.143
2.910
2.347
2.525

25.071

0.000

REALM

NVS

REALM

Gender
Male
Female
Male
Female

NVS

95% CI for mean

REALM

SE

NVS

SD

REALM

Mean

NVS

TEST

84
136
124
112

ANOVA, analysis of variance.

REALM and NVS were found as 59.0 and 2.0,
respectively (Table V and Fig. 2).

Discussion
In Turkey, health literacy is seen as equal to general
literacy, and most of the studies used the general
literacy level of their participants to make explanations. Researches whose object is to specifically
measure the levels of health literacy are rare [28,
29]. Some other studies tend to measure specific
472

knowledge about specific health conditions and
addressed knowledge as health literacy [30–38].
Therefore, we were not able to compare our results
with the findings of other Turkish studies.
According to the REALM scores, 41.3% of our
study group had limited health literacy. A review
regarding health literacy said that there are ;37
studies that used only the REALM test for assessing health literacy levels and a rate of low literacy
was found to be 22%, where age, educational
level, ethnicity, geographic location and income
were found to be associated with health literacy

Health literacy
Table IV. Answers to some statements and test scores
Test

Statement 1: If the health care provider gives me forms to fill out

Mean

SD

REALM

I always read and fill out by myself
My accompany will read and fill out
Health personnel will read and fill out

368
36
24

61.760
56.222
42.000

3.912
5.319
12.880

I always read and fill out by myself
My accompany will read and fill out
Health personnel will read and fill out

368
36
24

2.858
1.222
0.666

1.538
1.045
0.963

Lower

Upper

F

Significance

0.203
0.886
2.629

61.359
54.422
36.560

62.162
58.021
47.439

124.524

0.000

0.080
0.174
0.196

2.701
0.868
0.260

3.016
1.575
1.073

25.394

0.000

SE

REALM

N

ANOVA

Statement 2: I understand the printed material which is given to me by the health care provider
95% CI
Answers
N
Mean
SD
SE
Lower
Upper
Always
200 62.780
3.344
0.236 62.313 63.246
Sometimes
228 58.736
8.108
0.536 57.678 59.795
Never
28 55.285
7.091 10.340 52.536 58.035

NVS

Answers

NVS

95% CI

Always
Sometimes
Never

Test

200
228
28

3.160
2.263
1.428

1.531
1.620
1.708

0.108
0.107
0.322

2.946
2.051
0.765

3.373
2.474
2.091

ANOVA
F
30.487

25.190

Significance
0.000

0.000

REALM

Answers
I read it
I ask someone’s help to read it
I put it into trash without reading

N
380
40
36

Mean
61.821
47.300
58.666

SD
3.706
12.412
7.649

SE
0.190
1.962
1.274

95% CI
Lower
61.447
43.330
56.078

Upper
62.195
51.269
61.254

NVS

Statement 3: If health care provider gives me some printed material
Test

I read it
I ask someone’s help to read it
I put it into trash without reading

380
40
36

2.873
0.600
2.000

1.504
0.928
2.242

0.077
0.146
0.373

2.721
0.303
1.241

3.025
0.896
2.758

[39]. We also found that female, primary school
educated, >45 years of age and economically poor
participants had the lowest REALM scores. The
low scores of REALM test were in fact due to
low educational attainment among females and
older participants.
The second test that we have used for assessing
health literacy was the NVS test. Participants who
score >4 on the NVS had literacy level of greater
than or equal to 7th grade when measured with the
REALM test. On the other hand, 84.1% of participants who got a health literacy level of 7th to 8th
grade on the REALM test scored <4 on the NVS
test and were assessed as those with limited literacy.
Similarly, 62.6% of participants with an adequate
health literacy level according to the REALM test
were classified as those with limited literacy

ANOVA
F
132.086

42.733

Significance
0.000

0.000

according to the NVS test (see Table II and
Fig. 1). In a study among 250 English- and 250
Spanish-speaking participants, the mean (SEM)
scores for the NVS test were found to be 3.4 (1.9)
and 1.6 (1.5), respectively [12]. Another study
among 271 participants showed that 19.9% of the
study group had likely low, 26.2% possibly low and
53.9% adequate health literacy levels [40]. In our
study, 21.0% of participants had likely low, 50.9%
possibly low and 28.1% adequate health literacy
levels. Female, primary school educated and poor
economic condition participants and those with
older ages had the lowest NVS scores, which was
similar to the REALM scores.
Our study group got better scores on the REALM
test when compared with the NVS test. This may be
due to the complex math abilities the NVS test
473

H. Ozdemir et al.

Fig. 2. ROC curve for NVS and REALM.

Table V. Cutoff points and coordinates of the ROC curve for
NVS and REALM
Cutoff point
NVS
>0
>1
>2a
>3
>4
>5
>6
REALM
>54
>55
>56
>57
>58
>59a
>60
>61
>62
>63
>64
>65
>66

Sensitivity

Specificity

+LR

ÀLR

93.41
90.11
68.13
34.07
19.78
9.89
0.00

22.02
28.44
66.06
76.15
84.40
100.00
100.00

1.20
1.26
2.01
1.43
1.27

0.30
0.35
0.48
0.87
0.95
0.90
1.00

96.77
96.77
91.40
90.32
83.87
79.57
75.27
68.82
66.67
50.54
48.39
23.66
0.00

22.64
23.58
28.30
35.85
38.68
59.43
60.38
62.26
67.92
79.25
88.68
93.40
100.00

1.25
1.27
1.27
1.41
1.37
1.96
1.90
1.82
2.08
2.43
4.27
3.58

0.14
0.14
0.30
0.27
0.42
0.34
0.41
0.50
0.49
0.62
0.58
0.82
1.00

Bold indicates sensitivity, specificity +LR and -LR of the cutoff
points, +LR, positive likelihood ratio; ÀLR, negative likelihood
ratio.
a
Cutoff points of the Turkish versions of the NVS and REALM.

474

requires, but it could also be due to the participants
who are not familiar with reading nutritional labels.
This fact must be kept in mind when prescriptions
are given. Reliance on the drug labels should be
avoided because it is highly possible that patients
will not be able to understand their content. On the
other hand, the reasoning capability of our study
group was relatively better than their math skills
because the last two questions of the NVS test,
which measure reasoning, were correctly answered
nearly by the three-fourths of the participants. The
educational level found to be the most important
factor related to the health literacy.
If we include in the health literacy concept the
information and decision-making skills that we
make for health in our daily lives, the measurement
of health literacy with the existing assessment tools
would not be appropriate because these tools only
assess basic reading and writing skills to understand
and follow simple health messages, which is the
functional concept of health literacy. The functional
concept of health literacy lacks the much deeper
meaning and purpose of literacy, which is what it
is literacy enables us to do [41]. Health literacy
becomes more informed by the fields of psychology,
sociology, cognition and cultural studies, and it
is clear that to describe the health literacy abilities
of people, addressing understanding and behavioral change is needed. Obtaining, processing and

Health literacy
understanding basic health information, connecting
this information with appropriate health decisions
and making decisions that are consistent with promoting or maintaining good health should be the
definition of health literacy [42]. Therefore, the assessment tools should be able to distinguish between
the possession of information, the understanding of
it and the inclination and ability to act on it consistent with promoting health. None of the existing
health literacy assessment tools is capable of doing
this. Measuring the ability of reading, understanding, reasoning and numeracy is the first step in
assessing health literacy. Hence, without these basic
skills other dimensions of health literacy could not
be realized. Furthermore, the rapid changes in the
medical environment, quality and ethical issues, human rights, confidentiality and the necessity of keeping records make these basic skills more important
and issues worthy of measuring.
In conclusion, this may be the first study in
Turkey that measures health literacy by using specific health literacy measures developed and validated in Western countries. Because of the nature of
the measurement tools, we were only able to
measure the print literacy and to some extent numerical skills. However, these basic literacy skills
should be gained in childhood through a proper
formal education and should not be a matter of
testing in adulthood but because of many reasons
this could not be achieved. Our findings should be
taken into account in the changing health care
environment of Turkey, an environment that has
become more dependent on printed forms and
materials. Health care providers should know not
to rely only on printed forms. To communicate with
patients, to fully inform them and to obtain patient
feedback in order to control patients’ understanding
and reasoning seem to be essential actions. Traditional or ‘old-fashioned’ health communication is
needed even in this globalized, standardized, computerized and automated medical world.
There are several limitations to our study methods
that should be considered when interpreting the
results. First, our study was conducted in a localized
geographic area and, therefore, the results cannot
necessarily be generalized to other locales. Secondly,

we were not able to validate the Turkish version of
the instruments we used because there was not any
reliable test in Turkish, which is still being used for
this purpose. A third limitation is that we did not
measure whether the communication style of health
providers changed or improved when they found
a patient to have limited literacy skills. Further research is needed to determine if this occurs and if
such changes result in improved outcomes for
patients. A fourth limitation is that patients with
a long-term patient–provider relationship may be
more willing to undergo literacy assessments than
patients seeing a provider for the first time. We did
not, however, measure whether the participants in
this study were making first visits or if they have
a long-term relationship. Finally, we did not collect
data about the reason for the patients’ visit. All these
factors can influence patients’ satisfaction and their
willingness to undergo literacy assessment.
Finally, we want to point out the need for
addressing the broader definition of health literacy
and hope that this initial study will contribute to
putting health literacy on the agenda in Turkey.
Acknowledgements
The authors would like to thank Scribendi, Inc. for
their valuable contribution in editing the language
of this manuscript and Associate Professor Nuran
Bayram, PhD, for her assistance in performing the
receiver operating characteristic analysis.

Conflict of interest statement
None declared.
References
1. Ad Hoc Committee on Health Literacy for the American
Council on Scientific Affairs, American Medical Association Health Literacy: Report of the Council on Scientific
Affairs. JAMA 1999; 281: 552–7.
2. Ratzan SC, Parker RM. Introduction. In: Selden CR, Zorn
M, Ratzan SC, Parker RM (eds). National Library of Medicine Current Bibliographies in Medicine: Health Literacy.
Bethesda, MD: National Institutes of Health, US Department of Health and Human Services, 2000.

475

H. Ozdemir et al.
3. HHS (US Department of Health and Human Services).
Healthy People 2010: Understanding and Improving
Health. Washington, DC: US Department of Health and
Human Services, 2000.
4. Institute of Medicine. Health Literacy: A Prescription to
End Confusion. Washington, DC: Institute of Medicine,
Board on Neuroscience and Behavioral Health, Committee
on Health Literacy, 2004, 31–55.
5. Rudd R, Kirsch IJA, Yamamato K. Literacy and Health in
America. Center for Global Assessment. Policy Information
Center. Research and Development Educational Testing
Service. 2004. Available at: http://www.ets.org/Media/Re
search/pdf/PICHEATH.pdf. Accessed: 25 January 2009.
6. WHO. Health Promotion Glossary (WHO/HPR/98.1).
Geneva: WHO, 1998. Available at: http://www.who.int/
healthpromotion/about/HPR%20Glossary%201998.pdf.
Accessed: 13 January 2009.
7. Baker DW. The meaning and the measure of health literacy.
J Gen Intern Med 2006; 21: 878–83.
8. Baker DW, Gazmararian JA, Sudano J et al. The association
between age and health literacy among elderly persons.
J Gerontol B Psychol Sci Soc Sci 2000; 55: S368–74.
9. Parker RM, Baker DW, Williams MV et al. The test of
functional health literacy in adults: a new instrument for
measuring patients literacy skills. J Gen Intern Med 1995;
10: 537–41.
10. Davis TC, Long S, Jackson R et al. Rapid estimate of adult
literacy in medicine: a shortened screening instrument. Fam
Med 1993; 25: 391–5.
11. Kutner M, Greenberg E, Jin Y et al. The Health Literacy of
America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006–483). Washington, DC:
US Department of Education, National Center for Education
Statistics, 2006. Available at: http://nces.ed.gov/pubs2006/
2006483.pdf.
12. Weiss BD, Mays MZ, Martz W et al. Quick assessment of
literacy in primary care: the newest vital sign. Ann Fam Med
2005; 3: 514–22.
13. Chew LD, Griffin JM, Partin MR et al. Validation of
screening questions for limited health literacy in a large
VA outpatient population. J Gen Intern Med 2008; 23:
561–6.
14. Al-Tayyib AA, Rogers SM, Gribble JN et al. Effect of low
medical literacy on health survey measurements. Am J Pub
Health 2002; 92: 1478–81.
15. Arozullah A, Lee SYD, Khan T et al. The roles of low
literacy and social support in predicting the preventability
of hospital admission. J Gen Int Med 2006; 21: 140–5.
16. Bennett IM, Robbins S, Haecker T. Screening for low literacy among adult caregivers of pediatric patients. Fam Med
2003; 35: 585–90.
17. Baker DW, Parker RM, Williams MV et al. The relationship of patient reading ability to self-reported health and
use of health services. Am J Pub Health 1997; 87:
1027–30.
18. Baker DW, Parker RM, Williams MV et al. Health literacy
and the risk of hospitalization. J Gen Intern Med 1998; 13:
791–8.
19. Gazmararian JA, Baker DW, Williams MV. Health literacy
among medicare enrolles in a managed care organization.
JAMA 1999; 281: 545–51.

476

20. Gordon MM, Hampson R, Capell HA et al. Illiteracy in
rheumatoid arthritis patients as determined by the Rapid
Estimate of Adult Literacy in Medicine (REALM) score.
Rheumatology (Oxford) 2002; 41: 750–4.
21. Williams MV, Davis T, Parker RM et al. The role of health
literacy in patient-physician communication. Fam Med
2002; 34: 383–9.
22. Baker DW, Gazmararian JA, Williams MV et al. Functional
health literacy and the risk of hospital admission among
medicare managed care enrollees. Am J Public Health
2002; 92: 1278–83.
23. Gazmararian JA, Williams MV, Peel J et al. Health literacy
and knowledge of chronic disease. Patient Educ Couns
2003; 51: 267–75.
24. Baker DW, Gazmararian JA, Williams MV et al. Health
literacy and use of outpatient physician services by medicare
managed care enrollees. J Gen Intern Med 2004; 19:
215–20.
25. Safer RS, Kenan J. Health literacy: the gap between
physicians and patients. Am Fam Physician 2005; 72:
463–8.
26. The Ministry of Health of Turkey. Turkey Health Report.
Ankara: MoH, RSHC, School of Public Health, Publication
No: SB-HM-2004/01, 2004. Available at: http://www.sag
lik.gov.tr. Accessed: 26 October 2009.
27. The Ministry of Health of Turkey. Turkey Health
Transformation Program. Ankara, 2007. Available at:
http://www.saglik.gov.tr. Accessed: 26 October 2009.
28. Pette M, Borde T, David M. Turkish and German patients’
current knowledge of diagnosis and therapy at the beginning
and at the end of the in-patient stay. J Turkish German
Gynecol Assoc 2004; 5: 330–7.
29. Topbas M, Yaris F, Can G. Do the elderly have adequate
x
information about the drugs they use? Results of a study
conducted in a primary health care in Trabzon. Ege J Med
2003; 42: 85–90.
30. Ayranci U. AIDS knowledge and attitudes in a Turkish
population: an epidemiological study. BMC Public Health
2005; 13: 95.
31. Yilmaz H, Gurel S, Ozdemir O. Turkish patients with osteoarthritis: their awareness of the side effects of NSAIDs.
Turk J Gastroenterol 2005; 16: 89–92.
32. Turhan E, Inandi Y, Inandi T. Risk perception, knowledge and social distance of Turkish high school students
about HIV/AIDS. J Public Health (Oxford) 2006; 28:
137–8.
33. Biri A, Korucuoglu U, Ilhan M et al. Turkish women’s level
of knowledge on an attitude toward sexual health. Maturitas
2007; 58: 236–40.
34. Uncu Y, Alper Z, Ozdemir H et al. The perception of menopause and hormone therapy among women in Turkey. Climacteric 2007; 10: 63–71.
35. Pirincci AF, Oguzoncul E. Knowledge and attitude of married Turkish men regarding family planning. Eur J Contracept Reprod Health Care 2008; 13: 97–102.
36. Gemalmaz A, Oge A. Knowledge and awareness about osteoporosis and its related factors among rural Turkish
women. Clin Rheumatol 2008; 27: 732–8.
37. Aydemir N. Developing two different measures for assessing knowledge of and attitudes toward epilepsy for the
Turkish population. Epilepsy Behav 2008; 12: 84–9.

Health literacy
38. Baykal C, Al A, Ugur MG et al. Knowledge and interest of
Turkish women about cervical cancer and HPV vaccine. Eur
J Gyneacol Oncol 2008; 29: 76–9.
39. Paasche-Orlow MK, Parker RM, Gazmararian JA et al. The
prevalence of limited health literacy. J Gen Intern Med
2005; 20: 175–84.
40. Ryan JG, Leguen F, Weiss BD et al. Will patients agree to
have their literacy skills assessed in clinical practice? Health
Educ Res 2008; 23: 603–11.

41. Peerson A, Saunders M. Health literacy revisited: what do
we mean and why does it matter? Health Promot Int 2009;
24: 285–95.
42. Nutbeam D. The evolving concept of health literacy. Soc Sci
Med 2008; 67: 2072–8.
Received on May 22, 2009; revised on November 5, 2009;
accepted on December 7, 2009

477