Posts Tagged ‘evaluation’
A comparative evaluation of the effect of internet-based CME delivery format on satisfaction, knowledge and confidence
Background:Internet-based instruction in continuing medical education (CME) has been associated with favorable outcomes. However, more direct comparative studies of different Internet-based interventions, instructional methods, presentation formats, and approaches to implementation are needed. The purpose of this study was to conduct a comparative evaluation of two Internet-based CME delivery formats and the effect on satisfaction, knowledge and confidence outcomes.Methods:Evaluative outcomes of two differing formats of an Internet-based CME course with identical subject matter were compared. A Scheduled Group Learning format involved case-based asynchronous discussions with peers and a facilitator over a scheduled 3-week delivery period. An eCME On Demand format did not include facilitated discussion and was not based on a schedule; participants could start and finish at any time. A retrospective, pre-post evaluation study design comparing identical satisfaction, knowledge and confidence outcome measures was conducted.Results:Participants in the Scheduled Group Learning format reported significantly higher mean satisfaction ratings in some areas, performed significantly higher on a post-knowledge assessment and reported significantly higher post-confidence scores than participants in the eCME On Demand format that was not scheduled and did not include facilitated discussion activity.Conclusions:The findings support the instructional benefits of a scheduled delivery format and facilitated asynchronous discussion in Internet-based CME.
Process evaluation of the Intervention with Microfinance for AIDS and Gender Equity (IMAGE) in rural South Africa
The Intervention with Microfinance for AIDS and Gender Equity (IMAGE) combines microfinance, gender/HIV training and community mobilization (CM) in South Africa. A trial found reduced intimate partner violence among clients but less evidence for impact on sexual behaviour among clients’ households or communities. This process evaluation examined how feasible IMAGE was to deliver and how accessible and acceptable it was to intended beneficiaries during a trial and subsequent scale-up. Data came from attendance registers, financial records, observations, structured questionnaires (378) and focus group discussions and interviews (128) with clients and staff. Gender/HIV training and CM were managed initially by an academic unit (‘linked’ model) and later by the microfinance institution (MFI) (‘parallel’ model). Microfinance and gender/HIV training were feasible to deliver and accessible and acceptable to most clients. Though participation in CM was high for some clients, others experienced barriers to collective action, a finding which may help explain lack of intervention effects among household/community members. Delivery was feasible in the short term but both models were considered unsustainable in the longer term. A linked model involving a MFI and a non-academic partner agency may be more sustainable and is being tried. Feasible models for delivering microfinance and health promotion require further investigation.
Pharmacology as a foreign language: A preliminary evaluation of podcasting as a supplementary learning tool for non-medical prescribing students
Background:Nurses and other health professionals in the U.K. can gain similar prescribing rights to doctors by undertaking a non-medical prescribing course. Non-medical prescribing students must have a thorough understanding of the pharmacology of prescribing to ensure safe practice. Pharmacology education at this level is complicated by the variation in students’ prior subject knowledge of, and anxiety about, the subject. The recent advances in technology, particularly the potential for mobile learning, provide increased opportunities for students to familiarise themselves with lecture materials and hence promote understanding. The objective of this study was therefore to evaluate both the subjective (student perception) and objective (student use and exam results) usefulness of podcasts of pharmacology lectures which were provided as an extra learning tool to two cohorts (n=69) of non-medical prescribing students.Methods:The podcasts were made available to students through the virtual learning environment WebCT. Use of podcasts by two successive cohorts of nurse prescribing students (n=69) was tracked through WebCT. Survey data, which was collected from 44 of these students, investigated patterns of/reasons for podcast use and perceived usefulness of podcasts as a learning tool. Of these 69 students, 64 completed the pharmacology exam. In order to examine any impact of podcasts on student knowledge, their exam results were compared with those of two historical cohorts who did not have access to podcasts (n=70).Results:WebCT tracking showed that 91% of students accessed at least one podcast. 93% of students used the podcasts to revisit a lecture, 85% used podcasts for revision, and 61% used the podcasts when they had a specific question. Only 22% used the podcasts because they had missed a pharmacology session. Most students (81%) generally listened to the entire podcast rather than specific sections and most (73%) used them while referring to their lecture handouts. The majority of students found the podcasts helpful as a learning tool, as a revision aid and in promoting their understanding of the subject. Evaluation of the range of marks obtained, mode mark and mean mark suggested improved knowledge in students with access to podcasts compared to historical cohorts of students who did not have access to pharmacology podcasts.Conclusions:The results of this study suggest that non-medical prescribing students utilised podcasts of pharmacology lectures, and have found the availability of these podcasts helpful for their learning. Exam results indicate that the availability of podcasts was also associated with improved exam performance.
Process evaluation of a school-based weight gain prevention program: the Dutch Obesity Intervention in Teenagers (DOiT)
Health promotion programs benefit from an accompanying process evaluation since it can provide more insight in the strengths and weaknesses of a program. A process evaluation was conducted to assess the reach, implementation, satisfaction and maintenance of a school-based program aimed at the prevention of excessive weight gain among Dutch adolescents [Dutch Obesity Intervention in Teenagers (DOiT)]. Our process evaluation involved data collections by means of questionnaires among students, teachers, school board and site staff. The results indicated immense difficulties in the recruitment phase and therefore a low reach at school level. However, among adolescents of the schools that participated, the reach was high (84%). Furthermore, the classroom intervention was implemented successfully based on the number of lessons taught. Most teachers rated the DOiT-intervention positively; students rated the intervention 6.6 on a scale of 1–10. The majority of the teachers planned to implement the DOiT-intervention program in the future, as they perceived DOiT feasible for pre-vocational education students.
Evaluation of an interactive, case-based review session in teaching medical microbiology
Background:Oklahoma State University-Center for Health Sciences (OSU-CHS) has replaced its microbiology wet laboratory with a variety of tutorials including a case-based interactive session called Microbial Jeopardy!. The question remains whether the time spent by students and faculty in the interactive case-based tutorial is worthwhile? This study was designed to address this question by analyzing both student performance data and assessing students’ perceptions regarding the tutorial.Methods:Both quantitative and qualitative data were used in the current study. Part One of the study involved assessing student performance using archival records of seven case-based exam questions used in the 2004, 2005, 2006, and 2007 OSU-CHS Medical Microbiology course. Two sample t-tests for proportions were used to test for significant differences related to tutorial usage. Part Two used both quantitative and qualitative means to assess student’s perceptions of the Microbial Jeopardy! session. First, a retrospective survey was administered to students who were enrolled in Medical Microbiology in 2006 or 2007. Second, responses to open-ended items from the 2008 course evaluations were reviewed for comments regarding the Microbial Jeopardy! session.Results:Both student performance and student perception data support continued use of the tutorials. Quantitative and qualitative data converge to suggest that students like and learn from the interactive, case-based session.Conclusions:The case-based tutorial appears to improve student performance on case-based exam questions. Additionally, students perceived the tutorial as helpful in preparing for exam questions and reviewing the course material. The time commitment for use of the case-based tutorial is justified.
Evaluation of the Medical Student Research Programme in Norwegian medical schools. A survey of students and supervisors
Background:The Medical Student Research Programme is a national education and grant scheme for medical students who wish to carry out research in parallel with their studies. The purpose of the programme is to increase recruitment of people with a standard medical degree to medical research. The Research Programme was established in 2002 and underwent a thorough evaluation during the spring of 2007. The evaluation should investigate if the programme had fulfilled its objectives of increased recruitment to medical research, in addition to the students’ and supervisors’ satisfaction of the programme, and unwanted differences between the universities.Methods:Data was collected from students, supervisors and administrative staff via web-based questionnaires. Information about admission, implementation, results achieved and satisfaction was analysed and compared between the four Norwegian medical schools. In addition, the position of the scheme in relation to the national Quality Reform of Higher Education was analysed.Results:At the end of 2006, the Medical Student Research Programme had recruited 265 medical students to research. These consisted of 214 active students, 35 who had completed their studies and only 17 who had dropped out. Both students and supervisors were generally very satisfied with the scheme, including the curriculum, the results achieved and the administrative service. The majority of students wanted to continue their research towards a PhD and, of those who had completed the Medical Student Research Programme, practically all had published one or several scientific papers. The survey showed only small differences between the four medical schools, despite their choice of somewhat different solutions in terms of administration and organisation. The Medical Student Research Programme satisfies the majority of the demands of the Quality Reform, however as an integrated research programme aimed at a PhD it presupposes access to PhD courses before the completion of medical studies, as well as the ability to include undergraduate scientific work in a PhD thesis.Conclusions:The Medical Student Research Programme has led to an increase in the recruitment of graduated physicians to medical research in Norway. It will only be possible to evaluate whether this in turn will result in a larger number of PhDs in 3-5 years; this will also depend on the access to grants and fellowships.
Evaluation of HIV/AIDS secondary school peer education in rural Nigeria
In this study, we assessed whether peer education is an effective method of HIV/AIDS awareness, in terms of knowledge, misconception and behavior, among adolescents in the rural area of Nigeria. A comparative case series (n = 250), cross-sectional structured survey (n = 135) and focus group discussions (n = 80) were undertaken among adolescents. In both the case series and structured survey, a questionnaire was used which addresses the following issues: socio-demography, knowledge on transmission and prevention of HIV/AIDS, accessibility to different sources of HIV/AIDS information, stigmatization and sexual behavior. Binary logistic regression was applied to compare responses from the peer-educated and not peer-educated populations. The model was adjusted for confounders. We demonstrated increased knowledge and decreased misconception and sexual risk behavior in adolescents receiving peer education as compared to adolescents not receiving peer education. These differences are apparent both over time (2005–2007) and cross-sectional (2007). In conclusion, peer education in rural areas can be effective in HIV/AIDS prevention. Knowledge and behavior can be influenced positively.
An evaluation of the performance in the UK Royal College of Anaesthetists primary examination by UK medical school and gender.
Background:There has been comparatively little consideration of the impact that the changes to undergraduate curricula might have on postgraduate academic performance. This study compares the performance of graduates by UK medical school and gender in the Multiple Choice Question (MCQ) section of the first part of the Fellowship of the Royal College of Anaesthetists (FRCA) examination.Methods:Data from each sitting of the MCQ section of the primary FRCA examination from June 1999 to May 2008 were analysed for performance by medical school and gender.Results:There were 4983 attempts at the MCQ part of the examination by 3303 graduates from the 19 United Kingdom medical schools. Using the standardised overall mark minus the pass mark, graduates from five medical schools performed significantly better than the mean for the group and five schools performed significantly worse than the mean for the group. Males performed significantly better than females in all aspects of the MCQ – physiology, mean difference = 3.0% (95% CI 2.3, 3.7), p < 0.001; pharmacology, mean difference = 1.7% (95% CI 1.0, 2.3), p < 0.001; physics with clinical measurement, mean difference = 3.5% (95% CI 2.8, 4.1), p < 0.001; overall mark, mean difference = 2.7% (95% CI 2.1, 3.3), p < 0.001; and standardised overall mark minus the pass mark, mean difference = 2.5% (95% CI 1.9, 3.1), p < 0.001. Graduates from three medical schools that have undergone the change from Traditional to Problem Based Learning curricula did not show any change in performance in any aspects of the MCQ pre and post curriculum change.Conclusions:Graduates from each of the medical schools in the UK do show differences in performance in the MCQ section of the primary FRCA, but significant curriculum change does not lead to deterioration in post graduate examination performance. Whilst females now outnumber males taking the MCQ, they are not performing as well as the males.
Training simulated patients: evaluation of a training approach using self-assessment and peer/tutor feedback to improve performance
Background:Most medical schools use simulated patients (SPs) for teaching. In this context the authenticity of role play and quality of feedback provided by SPs is of paramount importance. The available literature on SP training mostly addresses instructor led training where the SPs are given direction on their roles. This study focuses on the use of peer and self evaluation as a tool to train SPs.Methods:SPs at the medical school participated in a staff development and training programme which included a) self-assessment of their performance while observing video-tapes of their role play using a structured guide and b) peer group assessment of their performance under tutor guidance. The pre and post training performance in relation to authenticity of role play and quality of feedback was blindly assessed by students and tutors using a validated instrument and the scores were compared. A focus group discussion and a questionnaire assessed acceptability of the training programme by the SPs.Results:The post-training performance assessment scores were significantly higher (p<0.05) than the pre-training scores. The degree of improvement in the quality of feedback provided to students was more when compared to the improvement of role play. The acceptability of the training by the SPs was very satisfactory scoring an average of 7.6 out of 10. The majority of the SPs requested the new method of training to be included in their current training programme as a regular feature.Conclusions:Use of structured self-reflective and peer-interactive, practice based methods of SP training is recommended to improve SP performance. More studies on these methods of training may further refine SP training and lead to improvement of SP performance which in turn may positively impact medical education.
PanGenetics Receives Approval For Clinical Evaluation Of Anti-NGF Antibody In Patients With Chronic Pain
PanGenetics B.V. announced that the Competent Authority of the Netherlands has approved the Clinical Trials Application for a first-in-man study with antibody PG110. This humanized antibody is a member of the class of Nerve Growth Factor (NGF) inhibitors which represent a promising novel approach in the treatment of chronic pain. PG110 will be evaluated in patients suffering from pain caused by osteoarthritis (OA).
Rx Drug Compare
Translator
By N2H