Occupational Medicine 2008;58:370–372
Published online 20 May 2008 doi:10.1093/occmed/kqn053

SHORT REPORT

...............................................................................................................................................................................................

Effectiveness of e-learning in continuing medical
education for occupational physicians
Nathalie I. R. Hugenholtz, Einar M. de Croon, Paul B. Smits, Frank J. H. van Dijk and
Karen Nieuwenhuijsen
...................................................................................................................................................................................

Background Within a clinical context e-learning is comparable to traditional approaches of continuing medical
education (CME). However, the occupational health context differs and until now the effect of
postgraduate e-learning among occupational physicians (OPs) has not been evaluated.
...................................................................................................................................................................................

Aim

To evaluate the effect of e-learning on knowledge on mental health issues as compared to lecturebased learning in a CME programme for OPs.

...................................................................................................................................................................................

Methods

Within the context of a postgraduate meeting for 74 OPs, a randomized controlled trial was conducted. Test assessments of knowledge were made before and immediately after an educational
session with either e-learning or lecture-based learning.

...................................................................................................................................................................................

Results

In both groups, a significant gain in knowledge on mental health care was found (P , 0.05).
However, there was no significant difference between the two educational approaches.

...................................................................................................................................................................................

Conclusion

The effect of e-learning on OPs’ mental health care knowledge is comparable to a lecture-based
approach. Therefore, e-learning can be beneficial for the CME of OPs.

...................................................................................................................................................................................

Key words

Continuing medical education; e-learning; mental health; occupational medicine; randomized
controlled trial.

...................................................................................................................................................................................

Introduction
In contrast to what is commonly believed, elderly physicians and physicians with many years of experience generally possess less factual medical knowledge [1].
Continuing medical education (CME), therefore, is widely
acknowledged as an indispensable part of the working life
of physicians. With the introduction of the Internet,
e-learning, i.e. the use of Internet technologies that enhance knowledge and performance, can be integrated into
CME programmes. Compared to conventional learning,
e-learning has the advantage that participants can choose
the place and time of education themselves. Within a clinical context, the effect of Internet-based CME programmes is comparable to traditional approaches of
CME [2,3]. Although there are some initiatives among
undergraduate occupational physicians (OPs) in using
e-learning, until now the effect of postgraduate e-learning
among OPs has not been evaluated [4].
Coronel Institute of Occupational Health, Academic Medical Center, University
of Amsterdam, The Netherlands.
Correspondence to: Nathalie I. R. Hugenholtz, Coronel Institute of Occupational
Health, Academic Medical Center, University of Amsterdam, PO Box 22700,
1100 DE Amsterdam, The Netherlands. Tel: 131 20 566 5325; fax 131 20 697
7161; e-mail: n.i.hugenholtz@amc.uva.nl

The occupational health context differs from the clinical context or primary health care because OPs must,
next to medical issues, consider the working situations
of their patients, management priorities and legislation.
Another barrier for e-learning in CME in occupational
health care could be that OPs are not frequently using the
Internet for obtaining information [5]. The aim of this
study, therefore, was to evaluate the effect of e-learning
on knowledge gain as compared to regular lecture-based
learning in a CME programme on mental health care
for OPs.

Methods
The study was a randomized controlled trial and was
conducted in December 2006 during a meeting on mental health care within a series of four postgraduate meetings for OPs. OPs were randomly assigned to four
different groups in order of arrival at the meeting, by
means of a four-block randomization system. The OPs
were unaware that two teaching approaches were compared during the meeting. Two groups received lecturebased teaching while the other two groups received
individual e-learning in a classroom, both with the same

Ó 2008 The Author(s).
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/
2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

N. I. R. HUGENHOLTZ ET AL.: EFFECTIVENESS OF E-LEARNING IN CME 371

content. The duration of both lessons was 30 min. Immediately before and immediately after the lesson, the
OPs completed a knowledge test.
Since employees with mental health problems constitute
a large part of the working population that visits the OP,
we developed the website http://mentalhealthandwork.com
(psychischenwerk.nl), covering three main topics: diagnosis,
prognosis and treatment. The website includes an e-learning
module which was designed as a self-directed teaching tool
for occupational health care professionals. The module provides participants with information, invites to search for information on the website and asks to solve cases or answer
multiple-choice questions using this information.
Each knowledge test consisted of 30 true/false questions and three open-ended ones. Two experts in mental
health care and one expert in occupational health care
education developed a pool of 66 questions. Two
versions of the test, version X and Y, each containing
different questions from the pool were generated. The
use of the knowledge tests was counterbalanced:
one e-learning group completed test X before finishing
the e-learning module and test Y after finishing.
The other e-learning group completed test Y before the
e-learning module and test X after finishing it. The same
procedure was used in the two lecture-based learning
groups. For each correct answer, the score was ‘1’ and
for each incorrect answer the score was ‘0’. Sum scores
were converted to a percentage of the possible total score.
Differences in baseline characteristics were tested with
t-tests for continuous variables and chi-square tests for
categorical variables. To determine whether ‘test version’
needed to be included as a covariate in the analyses, score
differences at baseline between version X and Y were
tested using a t-test. The effect of both learning approaches on knowledge was evaluated by comparing the
change in knowledge between the two groups. Analyses
were performed using the general linear model for repeated
measures. Next, subgroup analyses were conducted within
both learning approaches to investigate age and experience
as an OP as potential predictors for change in knowledge.
Data were analysed using SPSS version 13.0.

four groups differed at baseline, they differed not significantly within each teaching approach as well as between
the two teaching approaches. Both learning approaches
significantly enhanced OPs’ knowledge on mental health
care issues. The mean score for the e-learning approach
was 52.1 (SD 8.4) at baseline and 65.1 (SD 9.6) at posttest (P , 0.05). For the lecture-based approach, the
mean score was 52.3 (SD 9.0) at baseline and 64.3
(SD 9.0) at post-test (P , 0.05). The improvement in
knowledge did not differ significantly between these
groups. The potential predictors’ age and experience as
an OP showed no significant relation to knowledge scores
over time within the two learning approaches.

Results

Discussion

In total, 74 OPs attended the meeting. Randomization
assigned half of them to the e-learning and half to the
lecture-based groups. Two OPs in the lecture-based
group were excluded because they arrived too late. The
baseline characteristics of the OPs are described in
Table 1. Only the years of experience as an OP differed
significantly between both groups.
As the baseline scores of both knowledge tests X and Y
(mean 52.9, SD 9.2 and mean 51.5, SD 8.1, respectively)
did not differ significantly, the variable test version was
not included in further analyses. In Table 2, the scores of
the four groups are shown. Although the scores of the

The results of this study show that for OPs, e-learning is
just as effective in enhancing knowledge as lecture-based
learning. This is especially of interest since the study was
conducted in a group of OPs with a high mean age and
assumed fewer computer skills. Earlier, a lack of computer
skills has been identified as a major barrier preventing
doctors to use computer-based learning methods, rather
than a lack of preference for new technologies [6]. Since
computer skills are part of the current vocational training
and education, e-learning may play an even more important role in the CME of OPs in the near future. We did
not include the participants’ preference for, or satisfaction

Table 1. Baseline characteristics of OPs
Characteristics

E-learning
group (n 5 37)

Lecture-based
learning group
(n 5 35)

Age in years, mean (SD)
Women, n (%)
OP years of experience,
mean (SD)*
Acquaintance with the
website, n (%)
Previously used the
website, n (%)

50 (4.3)
15 (41)
18 (5.6)

49 (6.9)
18 (49)
14 (6.5)

17 (46)

13 (35)

11 (30)

6 (16)

*P # 0.05 t-test.

Table 2. Total scores of the two e-learning groups and the two
lecture-based learning groups at baseline and at post-test
Teaching
approach

E-learning Group
Group
Lecture
Group
based
Group

Total score, Test
Total score, Test
baseline
version post-test
version
(SD)
(SD)
A 50.9 (8.3) X
B 53.2 (8.6) Y
C 55.0 (10.0) X

64.9 (9.2) Y
65.3 (10.2) X
63.8 (7.3) Y

D 49.7 (7.4)

64.9 (10.5) X

Y

372 OCCUPATIONAL MEDICINE

with, the tested learning approaches. Nevertheless, the
meeting itself was rated 7.1 (0–10), which was in line with
the rating of the other three meetings that year.
This study is the first to demonstrate that e-learning
can be useful for CME in the occupational health care
setting. It will enable OPs to choose the place and time to
educate themselves. Recently, Fordis et al. [7] found that
web-based CME can lead to behaviour change as well as
sustained knowledge gains that are superior to traditional
approaches. A next step within occupational health care
may be to study the effectiveness of different forms of
e-learning and also the persistence of obtained knowledge
over time. Even more important is to study the impact of
e-learning on professional practice.

Funding
Funding to pay the Open Access publication charges for
this article was provided by the OA (Open Access) Fund
of the University of Amsterdam.

Key points
• Within the complex setting of occupational health
care, postgraduate e-learning is just as effective in
enhancing knowledge as lecture-based learning.
• Since the lack of computer skills of OPs will decrease, e-learning may play an even more important role in the CME of OPs in the near future.
• Although our e-learning module enhanced the
knowledge of OPs, the effectiveness of different
forms of e-learning, persistence of obtained
knowledge over time and the impact of e-learning
on professional practice should be further explored.

Acknowledgements
The researchers would like to thank the Netherlands School of
Public and Occupational Health and especially Jac van der
Klink, Suzanne Lagerveld, Kitty Meulenbeld, Erik Noordik
and Frans Vlek for organizing the CME meeting.

Conflicts of interest
None declared.

References
1. Choudhry NK, Fletcher RH, Soumerai SB. Systematic
review: the relationship between clinical experience and
quality of health care. Ann Intern Med 2005;142:260–273.
2. Wutoh R, Boren SA, Balas EA. eLearning: a review of
Internet-based continuing medical education. J Contin Educ
Health Prof 2004;24:20–30.
3. Curran VR, Fleet L. A review of evaluation outcomes of
web-based continuing medical education. Med Educ
2005;39:561–567.
4. Kolb S, Reichert J, Hege I et al. European dissemination of
a web- and case-based learning system for occupational
medicine: NetWoRM Europe. Int Arch Occup Environ Health
2007;80:553–557.
5. Schaafsma F, Hulshof C, van Dijk F, Verbeek J. Information
demands of occupational health physicians and their attitude
towards evidence-based medicine. Scand J Work Environ
Health 2004;30:327–330.
6. Mamary EM, Charles P. On-site to on-line: barriers to the
use of computers for continuing education. J Contin Educ
Health Prof 2000;20:171–175.
7. Fordis M, King JE, Ballantyne CM et al. Comparison of the
instructional efficacy of Internet-based CME with live interactive CME workshops: a randomized controlled trial. J Am
Med Assoc 2005;294:1043–1051.