Phase 3-Evaluation of a research study (Due by WEEK 9)
In phase 3 we will examine a research study titled:
Translating evidence to practice in the health professions: a randomized trial of Twitter vs Facebook
As future advance practice clinicians, how are you going to translate your evidence into the health field?
Your assignment for phase three is to read the attached article and analyze it based on its merit by answering the following questions:
1- Has the author formulated an appropriate research question based on the problem/issue?
2- Is the research question clearly defined in terms of its cope and relevance?
3- What is the authors orientation towards the research problem/issue- is it critical analysis or interpretation based?
4- How does this article contribute to your understanding of possible research modalities and methods?
5- What are the strengths, limitation of the study?
6- Which Essentials do you see represented in this article and why?
Phase 3-Evaluation of a research study (Due by WEEK 9)
In phase 3 we will examine a research study titled:
Translating evidence to practice in the health professions: a
randomized trial of Twitter vs Facebook
As future advance practice clinicians, how are you going to translate
your evidence into the health field?
Your assignment for phase three is to read the attached article and
analyze it based on its merit by answering the following questions:
123456-
Has the author formulated an appropriate research question based
on the problem/issue?
Is the research question clearly defined in terms of its cope and
relevance?
What is the authors orientation towards the research
problem/issue- is it critical analysis or interpretation based?
How does this article contribute to your understanding of possible
research modalities and methods?
What are the strengths, limitation of the study?
Which Essentials do you see represented in this article and why?
This paper should follow APA rules for grammar and citations. It should
be 5- 6 pages in length.
Cohesiveness
12.5%
Ties together
information
from all sources.
Paper flows
from one issue
to the next with
no headings.
Authors writing
demonstrates an
understanding of
the relationship
among material
obtained from all
sources
Mostly, it ties
together
information
from all sources.
Paper flows with
only some
disjointedness.
The authors
writing
demonstrates an
understanding of
the relationship
among material
obtained from all
sources.
Spelling and
Grammar
12.5%
Minimal spelling Some spelling
and/or grammar and or grammar
mistakes
mistakes.
Sources
12.5%
Over 5 current
sources, of
which at least 3
are peer-review
journal articles
or scholarly
books.
Sources include
both general
background
sources and
specialized
sources.
Special-interest
sources and
popular
literature and
acknowledged as
such if they are
cited.
All web sites
utilized are
authoritative.
5 current
sources, of
which at least 2
are peer-review
journal articles
or scholarly
books.
All web sites
utilized are
authoritative.
Sometimes ties
together
information
from all sources.
Paper does not
flow.
Disjointedness is
apparent.
The authors
writing does not
demonstrate an
understanding of
the relationship
among material
obtained from all
sources.
Noticeable
spelling and
grammar
mistakes.
Fewer than 5
current sources
or fewer than 2
of 5 are peerreviewed journal
articles or
scholarly books.
All web sites
utilized are
credible.
It does not tie
together
information.
Paper does not
flow and appears
to be created
from disparate
issues.
Headings are
necessary to link
concepts.
Writing does not
demonstrate
understanding
any relationship
An unacceptable
number of
spelling and/or
grammar
mistakes
Fewer than 5
current sources
or fewer than 2
of 5 are peerreviewed journal
articles or
scholarly books.
Not all web sites
utilized are
credible, and/or
sources are not
current.
Citations
12.5%
Cites all data
obtained from
other sources.
APA citation
style is used in
both text and
bibliography
Cites most data
obtained from
other sources.
APA citation
style is used in
both text and
bibliography.
Cites some data
obtained from
other sources.
Citation style is
inconsistent or
incorrect.
Does not cite
sources.
Journal of the American Medical Informatics Association, 24(2), 2017, 403408
doi: 10.1093/jamia/ocw085
Advance Access Publication Date: 29 June 2016
Brief Communication
Brief Communication
Translating evidence to practice in the health professions:
a randomized trial of Twitter vs Facebook
Jacqueline Tunnecliff,1 John Weiner,2 James E Gaida,3 Jennifer L Keating,1
Prue Morgan,1 Dragan Ilic,2 Lyn Clearihan,4 David Davies,5 Sivalal Sadasivan,6
Patitapaban Mohanty,7 Shankar Ganesh,7 John Reynolds,2 and Stephen Maloney1
1
Department of Physiotherapy, Monash University, Frankston, Australia, 2Department of Epidemiology and Preventive Medicine,
Monash University, Melbourne, Australia, 3Discipline of Physiotherapy and University of Canberra Research Institute for Sport
and Exercise (UCRISE), University of Canberra, Canberra, Australia, 4School of Primary Health, Monash University, Melbourne,
Australia, 5Warwick Medical School, University of Warwick, Coventry, United Kingdom, 6JC School of Medicine & Health Sciences, Monash University Malaysia and 7Swami Vivekanand National Institute of Rehabilitation Training and Research, Odisha,
India
Correspondence to Associate Professor Stephen Maloney, Department of Physiotherapy, Monash University, PO Box
527, Frankston, 3199, Victoria, Australia; [email protected]
Received 7 October 2015; Revised 21 April 2016; Accepted 30 April 2016
ABSTRACT
Objective: Our objective was to compare the change in research informed knowledge of health professionals
and their intended practice following exposure to research information delivered by either Twitter or Facebook.
Methods: This open label comparative design study randomized health professional clinicians to receive
practice points on tendinopathy management via Twitter or Facebook. Evaluated outcomes included knowledge change and self-reported changes to clinical practice.
Results: Four hundred and ninety-four participants were randomized to 1 of 2 groups and 317 responders analyzed. Both groups demonstrated improvements in knowledge and reported changes to clinical practice. There
was no statistical difference between groups for the outcomes of knowledge change (P ¼ .728), changes to clinical practice (P ¼ .11) or the increased use of research information (P ¼ .89). Practice points were shared more by
the Twitter group (P < .001); attrition was lower in the Facebook group (P < .001).
Conclusion: Research information delivered by either Twitter or Facebook can improve clinician knowledge and
promote behavior change. No differences in these outcomes were observed between the Twitter and Facebook
groups. Brief social media posts are as effective as longer posts for improving knowledge and promoting
behavior change. Twitter may be more useful in publicizing information and Facebook for encouraging course
completion.
Key words: social media, evidence-based practice, communication, education, professional, computer-assisted instruction
INTRODUCTION
A significant gap remains between research generated healthcare
knowledge and clinical practice.13 Social media can rapidly link
researchers and clinicians from diverse geographical regions,
disciplines, and areas of practice; making it an ideal medium for
knowledge exchange and education. Approximately 25% of
health professionals currently use social media for obtaining
research information.4
Social media has been defined as a collection of web-based
technologies that share a user-focused approach to design and functionality, where users can actively participate in content creation
C The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved.
V
For Permissions, please email: [email protected]
403
404
Journal of the American Medical Informatics Association, 2017, Vol. 24, No. 2
and editing through open collaboration between members of communities of practice.5 The use of social media in education may
lead to positive learning experiences,5,6 increases in knowledge and
skills,710 and changes to the clinical practices of health professionals.10,11 However, there is a need for studies to evaluate the relative
effectiveness of different social media based applications.12
Two of the largest social media applications are Facebook (1.49
billion monthly active users) and Twitter (316 million monthly
active users).13,14 Both sites promote user interaction and allow
posting of text, videos, and weblinks; however, Twitter limits posts
to 140 characters. Neither site charges access costs. The popularity
and features of these sites indicate their potential application in
communicating research information and, therefore, were chosen
for investigation in this study.
OBJECTIVE
The primary objective of this study was to determine if research
information delivered by Twitter or Facebook would result in
greater changes in research informed knowledge and practices of
health professionals. The secondary aim was to compare participant
behavior and engagement with the two mediums.
METHODS
Design
An open label randomized comparative design was used, with a
mixed methods approach to data collection and analysis. The Monash University Human Research Ethics committee (CF 14/1372
2014000640) approved the study.
Participants
Health professional clinicians of any discipline (e.g., medicine, physiotherapy, podiatry), geographical location, or level of expertise
(including undergraduate students), were eligible to participate.
Recruitment occurred via an email invitation distributed to clinical
affiliates and departments of Monash University, Faculty of Medicine, Nursing and Health Sciences, Australia; Monash University
Malaysia; Swami Vivekanand National Institute of Rehabilitation
Training and Research, India; and the University of Southern California. Professional associations representing professions registered
with the Australian Health Practitioner Regulation Agency15 were
also invited to distribute the invitation to participate via email or
their own social media sites.
Intervention
A short course, consisting of the same 8 practice points or key
educational messages of 140 characters or less, on topics related to
tendon management were delivered to each group via posts on Twitter16 and Facebook17 web pages. Each practice point was linked to
supplementary information in the form of peer-reviewed journal
articles or podcasts by clinical experts. The course was designed by
educational, clinical, and research experts, and was identical except
that the Facebook posts contained the practice point plus an additional 26 short written statements (12 sentences) that highlighted
key concepts from the supplementary information. The practice
points were delivered evenly over a 2 week period, to both groups at
the same time points. The pages were not restricted access.
Procedure
Clinicians consented to participate by providing contact details
through an online survey. Those who provided a valid email address
were enrolled. Participants were stratified by role (student, clinician,
or other) and randomized to receive the practice points via Twitter
or Facebook. Participants received video and written instructions on
obtaining a social media account and accessing the practice points
from their allocated site. The instructions also encouraged interaction on the allocated site. Participants were sent three reminder
emails at each data collection point to minimize attrition. The study
was conducted between August and October 2014.
Outcomes
Data was obtained via an anonymous online survey completed 1
week before (baseline assessment) and after (post-intervention
assessment) the short course. A password was used to match preand post-course data. Demographic details, information on tendon
management experience, and current use of social media were
obtained.
Outcomes were determined based on the Kirkpatrick hierarchical levels of evaluation 13.18 Participation and engagement data
was also collected. A data collection summary can be found in
Appendix 1.
Kirkpatrick Level 1: Participant Reactions
The Social Media Use and Perception Instrument (SMUPI), a questionnaire of 10 items with high internal consistency,19 measured
attitudes towards using social media in continuing professional
development.
Kirkpatrick Level 2: Knowledge
Sixteen multiple choice questions assessed knowledge (AE
responses) (Appendix 2). One question correlated with each
practice point and one correlated with information from each
piece of supplementary information. The questions in both assessments were identical, but question and response order were randomized to minimize score improvements based on pattern recognition.
Participants were not given assessment answers until the conclusion
of the study. Self-rated measures of tendon management confidence
and knowledge were also obtained.
Kirkpatrick Level 3: Behavior Change
Participants were asked has the education you have received via
social media during this trial changed the way you practice, or
intend to practice, with musculoskeletal clients? and has the education you have received during this trial increased your use of
research evidence within your clinical practice?
Participation was evaluated via the number of participants who
connected with the social media pages and completed the assessments. Data on interaction was obtained through participant selfreport and from the number of times posts were approved of
(liked or favorite), shared or commented on.
Analysis
Mixed linear models were used to analyze the repeated measurements (pre- and post-exposure to the intervention) on the participants. The restricted maximum likelihood method (REML), as
implemented in the GenStat statistical package,20 was used to fit the
models, calculate predicted means and test, using F-tests, the main
effects of group (Twitter vs Facebook) and time (pre vs post) as well
Journal of the American Medical Informatics Association, 2017, Vol. 24, No. 2
Expressed interest in par cipa ng (n=500)
Table 1. Participant demographics and participant characteristics
Twitter
N (%)a
Excluded (n=6)
n=1 complaint about process
n=5 no email address provided
Randomized (n=494)
Allocated to Twi er (n=247)
Allocated to Facebook (n=247)
Withdrew (n=1)
Reason unknown
Comple on of baseline Assessment
(n=128)
Comple on of baseline Assessment
(n=144)
Comple on of post interven on
assessment (n=86)
Comple on of post interven on
assessment (n=132)
Figure 1. Consort flow chart showing attrition of study participants.
as their 2-way interaction. Pairwise least significant difference tests of
the group-by-time means were based on these analyses and conducted
at the 5% significance level. Diagnostic plots of residuals were
checked for assumptions on which these methods are based. Analyses
of the 5-point Likert scale responses also used the restricted maximum
likelihood method as is customary with large datasets.21 The analyses
of binary response outcomes, measured post intervention, were based
on logistic regression models, also fitted using GenStat. Discrete count
data from Twitter and Facebook sites were analyzed using a variancestabilizing transformation in an analysis of variance.
RESULTS
Five hundred clinicians consented to participate. Five were excluded
due to an invalid email address, and one participant asked to be
removed. Four hundred and ninety-four participants were randomized.
The attrition rates from randomization to baseline assessment were
48.2% for the Twitter group and 41.7% for the Facebook group; the
difference was not significant [v2 (1, n ¼ 494) ¼ 2.09, P ¼ .148]. Attrition from baseline assessment to post intervention assessment was
32.8% for the Twitter group and 8.3% for the Facebook group; this
difference was significant [v2 (1, n ¼ 494) ¼ 17.37, P < .001]. Three
hundred and seventeen responses were analyzed (140 Twitter, 177
Facebook). There were 99 baseline assessments, 45 post intervention
assessments, and 173 matched baseline and post intervention assessments. A consort flow-chart is available in Figure 1.
Demographics
Demographic data and data on tendon management experience and
social media use was obtained from the baseline assessment and is
presented in Table 1.
Kirkpatrick levels 1, 2 and 3
Following the intervention, (the short course consisting of practice
points) there were statistically significant increases in SMUPI score,
self-rated confidence, self-rated knowledge and multiple choice
assessment score; but no statistically significant differences between
the groups in their changes over time. Participants in both groups
reported a change in practice/intended practice and increased use of
research in practice/intended practice as a result of the intervention
but there was no statistically significant difference between the
groups. This is shown in Table 2.
405
Baseline demographic data sets
Area of practice
Physiotherapy/physical therapy
Medicine
Osteopathy
Podiatry
Other
Not stated
Role
Undergraduate Student
Postgraduate Clinical Trainee
Clinician
Other
Not stated
Age
Under 18
1824
2534
3544
4554
5564
65þ
Sex
Male
Female
Not stated
Country
Australia
India
Malaysia
UK
USA
Other
Not stated
Tendon management experience
Provide health care to clients with
tendon disorders once a week or more
Social Media experience
Use Twitter
Use Facebook
Facebook
N (%)a
128
144
95 (74.2)
18 (14.1)
2 (1.6)
7 (5.5)
4 (3.1)
2 (1.6)
98 (68.1)
19 (13.2)
3 (2.1)
11 (7.6)
11 (7.6)
2 (1.4)
33 (25.8)
9 (7.0)
78 (60.9)
8 (6.3)
0 (0.0)
36 (25.0)
13 (9.0)
78 (54.2)
17 (11.8)
0 (0.0)
0 (0.0)
28 (21.9)
59 (46.1)
31 (24.2)
8 (6.3)
2 (1.6)
0 (0.0)
0 (0.0)
39 (27.1)
64 (44.4)
28 (19.4)
8 (5.6)
4 (2.8)
1 (0.7)
79 (61.7)
47 (36.7)
2 (1.6)
71 (49.3)
71 (49.3)
2 (1.4)
48 (37.5)
14 (10.9)
5 (3.9)
29 (22.7)
12 (9.4)
19 (14.8)
1 (0.8)
59 (41.0)
14 (9.7)
6 (4.2)
23 (16.0)
17 (11.8)
24 (16.7)
1 (0.7)
61 (47.7)
62 (43.1)
75 (58.6)
106 (82.8)
66 (45.8)
130 (90.3)
a
Percent of group (Twitter or Facebook) that provided baseline data.
The Twitter page developed 428 followers and the Facebook
page received 155 likes. An estimated 10.0% (8/80) of the Twitter
group and 7.8% (9/115) of the Facebook group reported interacting
online. The difference between groups was not significant [v2 (1,
n ¼ 195) ¼ 0.28, P ¼ 0.597)]. An estimated 42.6% (20/47) of the Twitter group and 34.8% (24/69) of the Facebook group reported lack of
time as a reason for lack of interaction on the social media sites.
Statistically significant differences were found between groups
for number of times information was shared (mean shares per post
Twitter 10.40, Facebook 0.20, SED 3.030, P < .001) and approved
of (liked/favourite) (mean Twitter 14.00, Facebook 8.00, SED
1.414, P ¼ .005).
DISCUSSION
This study has demonstrated that research information delivered by
either Twitter or Facebook can improve clinician knowledge and
406
Journal of the American Medical Informatics Association, 2017, Vol. 24, No. 2
Table 2. Kirkpatrick level 13 outcomes
Baseline measures
Predicted Mean (n)
Post-Intervention measures
Predicted Mean (n)
Kirkpatrick level 1 outcomes
SMUPIa
Twitter
40.34 (126)
41.85 (86)
Facebook
39.53 (143)
40.86 (127)
Difference (SED)b
0.81 (0.82)
0.99 (0.91)
P-value
.326
.277
Kirkpatrick level 2 outcomes
Self-rated confidence in tendon managementc
Twitter
3.380 (128)
3.784 (86)
Facebook
3.216 (143)
3.644 (131)
0.164 (0.106)
0.141 (0.116)
Difference (SED)b
P-value
.124
.227
Tendon management self-rated knowledgec
Twitter
3.181 (127)
3.727 (86)
Facebook
3.027 (143)
3.570 (131)
0.154 (0.102)
0.157 (0.112)
Difference (SED)b
P-value
.135
.163
Multiple choice assessment total score (max score 16)
Twitter
7.649 (123)
10.308 (80)
Facebook
6.599 (136)
9.435 (118)
1.050 (0.469)
0.874 (0.521)
Difference (SED)b
P-value
.026
.095
Assessment score for questions that addressed the practice points (max score 8)
Twitter
4.155 (123)
5.523 (80)
Facebook
3.789 (136)
5.431 (118)
0.366 (0.259)
0.093 (0.293)
Difference (SED)b
P-value
.159
.752
Assessment score for questions addressing the supplementary information (max score 8)
Twitter
3.485 (123)
4.819 (80)
Facebook
2.848 (136)
4.025 (118)
0.637 (0.255)
0.793 (0.211)
Difference (SED)b
P-value
.013*
.006*
Number reporting change (n)
% of group (95% CI)
Kirkpatrick level 3 outcomes
Reported change in practice due to intervention
Twitter
59 (77)
Facebook
77 (117)
Reported increased use of research in practice
Twitter
55 (78)
Facebook
80 (115)
Difference (SED)b
1.51 (0.66)
1.33 (0.58)
P-value
.024
.022
.841d
0.404 (0.083)
0.428 (0.072)
Purchase answer to see full
attachment
Recent Comments