Radiological Medical Physics: Assessment Administrative Draft v0.1 2 August 2004 Assessment of a Registrar in Radiological Medical Physics Aim To determine whether a registrar has satisfactorily completed the training and education program the Accreditation Panel shall assess the level of knowledge and competencies acquired during the program. Accreditation in Radiological Medical Physics Upon receiving a satisfactory assessment in the training and education program from the Accreditation Panel the registrar will be accredited in Radiological Medical physics. Assessment Processes The Accreditation Panel shall assess the registrar’s performance on the following: * Obtaining an accredited MSc (Medical Physics) or previously agreed alternative. * Continuous assessment o Log Book, Portfolios and o Assessment by the Clinical Supervisor and the External Assessor ? Clinical Supervisor’s reports ? External Assessors reports o Final Report * Completing ACPSEM Radiology Medical Physics Accreditations exam: consisting of a: o Practical examination o Written exam o Oral examination * Overall assessment by the Accreditation Panel: o Reviewing Log Book, Portfolios and Final Report o Reviewing all assessment reports o Assessment of research/dissertations/projects outcomes ? Either one peer reviewed publication as first author and 1 peer reviewed publications as co-author or 3 peer reviewed publications as co-author (I would suggest that evidence of research would be desirable but not compulsory in the TEAP – the accredited medical physicist is not being given the “nod” because they are a published researcher) ? Presented their work to at least 2 meetings (at least 1 at a national or international level). Post graduate degree in Medical Physics to at least an MSc level Ideally the registrar will have completed ACPSEM accredited MSC degree in Medical Physics. When candidates enter the programme the accreditation committee may recognise prior learning as fulfilling all or part of the educational objectives of the MSc. Alternatively the registrar may wish to undertake a different degree (e.g. a PhD) as part of the programme. In both these circumstances the accreditation committee will specify objectives for the registrar which would indicate that he or she has reached a level equivalent to an MSc degree in Medical Physics and will require a written statement from the university that the registrar has achieved that level. Progress Review A progress review shall be held between the Clinical Supervisor, the External Assessor and the registrar at least annually. Although not desirable, the review may be by teleconference on approval by the Chairman of the Accreditation Panel. The registrar progress review shall consider: ? The clinical and academic progress made by the registrar ? The Log Book and the Portfolios under development ? The six monthly reports made by the Clinical Supervisor ? Any feedback and/or recommendations between the three parties on the registrar’s performance or the training program itself. A report of the Progress Review and recommendations shall be forwarded to the Accreditation Panel by the External Assessor. A copy shall also be provided to the registrar with the proviso that confidential recommendations to the Department may be removed provided that the Accreditation Panel are advised of the sections removed. Portfolios Three pPortfolios shall be complied and submitted based on the progressive completion of the following training Modules. There is considerable flexibility in the order in which the modules can be completed. * RMP Module 1 - Basic Radiation Instrumentation, Measurement and Practice * RMP Module 4 - General Radiography equipment * RMP Module 5 - Film, screens and Chemistry * RMP Module 2 - Patient Dosimetry * RMP Module 3 - Imaging Theory and Quality Assessment * RMP Module 6 - Fluoroscopy and DSA * RMP Module 7 - Mammography * RMP Module 8 - CT * RMP Module 9 - PACS and image display systems * RMP Module 10 - Digital Radiography * RMP Module 11 - MRI * RMP Module 12 - Ultrasound * RMP Module 13 - Radiation Protection * Acquaintanceship discipline e.g. Radiation Oncology or Nuclear Medicine In the event that a registrar does not have a 3 month acquaintanceship period in a different discipline a portfolio will be developed that demonstrates her/his learning during that time. The first portfolio is to comprise of RMP Modules 1,4,5 and 2 other modules as agreed with the Clinical Supervisor (recommend that one of the additional modules be either 6,7,8,10). This should be completed by the end of the third year. The second portfolio is to comprise of any other 4 modules as agreed with the Clinical Supervisor (It is recommended that the Patient Dosimetry and remaining x-ray modalities are completed for the second portfolio i.e RMP Modules 2,6,7,8). This should be completed by the end of the fourth year. The final portfolio is to comprise of the remainder of Modules. This should be completed mid way through the fifth year. Each Portfolio provides the evidence which demonstrates the registrar has acquired the required knowledge and competencies in the individual study area. It shall include: * A Table of Contents * The evidence which demonstrates the registrar has acquired the relevant knowledge and competencies (detailed evidence will be in the Log Book). * Copies of reports and documentation described in each module On completing each portfolio: * A copy shall be forwarded to the External Assessor * The Clinical Supervisor shall provide a confidential assessment to the accreditation panel on the knowledge and competencies gained in the Portfolio. * The External Assessor shall review the Portfolio and if necessary have it reviewed by an examiner with specialist knowledge in the study area. * There shall be a discussion between the registrar, the Clinical Supervisor and the External Assessor on the Portfolio. * The External Assessor shall make a report and a recommendation of the registrar’s performance (“pass”, “not of required standard”) on this portfolio, including a brief record of the discussions held with the registrar and supervisor. * When the recommendation is “pass”, the report and recommendation is forwarded to the Accreditation Panel along with a copy of the portfolio. * When the recommendation is “not of required standard” the registrar may: o Request a review by the Accreditation Panel o Revise the portfolio and resubmit for assessment Final Report The registrar shall compile a Final Report summarising the evidence which demonstrates the registrar has acquired the required knowledge and competencies and are ready to assume a position of responsibility in the field. It shall include: * A Table of Contents * A brief curriculum vitae * The registrar’s Training Plan * The registrar’s job description * The department structure diagram and the registrar’s place in it * A list of Modules submitted in each portfolio * A copy of the External Assessors report for each portfolio submitted * A list of courses, meetings, seminars attended and a brief critique of each and its value * A brief description of research and development projects * A brief description of protocols and procedures the registrar has helped to develop and write and the registrar’s role in the development * A brief description of internal reports the registrar have helped develop and the registrar’s role in the development * Published papers * Sources must be cited The Final Report is to be completed by mid way through the fifth year and must be submitted to the Accreditation panel prior to the practical examination. On completing the Final Report: * The Clinical Supervisor shall provide a confidential assessment to the External Assessor on the overall knowledge and competencies acquired by the registrar and the registrar’s general attitude and performance. Included in the report will be an assessment of the capability of the registrar to work unsupervised and to take responsibility. * The External Assessor shall review the Final Report. * There shall be a discussion between the registrar, the Clinical Supervisor and the External Assessor on the Final Report (to what end? Is there a record of the discussion? Etc) * The External Assessor shall make a report and a recommendation of the registrar’s performance (“pass”, “not of required standard”) * When the recommendation is “pass”, the report and recommendation is forwarded to the Accreditation Panel. * When the recommendation is “not of required standard” the registrar may: o Request a review by the Accreditation Panel o Revise the report and resubmit for assessment. (External Examination Accreditation Exam for Radiology Physics - BJL) Written Exams All registrars will be required to sit a 3.5 hour written examination covering the major areas of Radiological Physics in their final year of the training program. The arrangements for this examination shall be organised at the discretion of the Accreditation Panel with appropriate supervision organised for the applicant to carry out the exam locally. The written examination shall comprise two sections: (a) 200 general multiple choice questions to be completed over 2 hours. (b) Six short answer questions, the candidate to choose three, to be completed over 1.5 hours. The candidate shall answer one short answer question on each of the general areas of radiation safety, the basic science of imaging and advanced medical imaging technology (CR/DR, CT, DSA, MRI, Mammography and Ultrasound). Some example questions are given in the Candidates Kit[1]. There shall be a short break between the MCQ and short answer exam, the duration of which shall be at the discretion of the exam supervisor. It is expected that the candidate achieve a minimum grade of 75% in the multiple choice section of the written examination. A candidate who does not achieve at least 65% will not be allowed to continue with the practical and oral examination. A candidate who achieves greater than 65% but less than the expected 75% may, at the discretion of the Accreditation Panel, be required to re-sit an MCQ examination. The pass mark for the short answer examination is 65%. In instances when the candidate has not quite achieved this level but has performed exceptionally well in the MCQ examination he or she may be allowed to proceed with the practical and oral examination but only at the discretion of the Accreditation Panel. When a registrar fails a written exam, the registrar may repeat that written exam at the next opportunity. Practical Examination (Who conducts the practical exam, how many examiners, where?) It is compulsory that the candidate pass the practical examination. The practical examination set by the Chief Examiner of the Radiological Physics Accreditation Panel shall comprise two tasks of approximately 30 minutes per task. The candidate shall be advised of the nature of the tasks at least 15 minutes before the first task is to be attempted. The progress of the work during the practical examination shall be the responsibility of the Chief Examiner, Radiological Physics Accreditation Panel, or their designated alternate, carrying out the accreditation. Typical examinable tasks are given in the Candidates Kit[1]. Questions may be asked by the examiners during the practical examination but these interruptions will be limited to points of immediate and pertinent concern at the time. General questions on the practical examination shall be asked at the end of the practical examination and during the general oral and these may relate to issues such as why testing is done in a particular way or the related physics behind the measurement process. The candidate shall not be expected to have a working knowledge of the equipment with which they will be asked to use to carry out the practical examination. A member of the staff from the department where the examination is being held shall operate the equipment for the candidate if required. If time is limited, the Chief Examiner of the Accreditation Panel, or their designated alternate, shall ask the candidate to delete some aspects of the procedure in which they are being examined. This shall be at the discretion of the Chief Examiner. (The meaning of the highlighted bit is unclear – is delete the right word?) The result of the practical examination shall not be disclosed to the candidate at this stage. General Oral Examination (Who conducts the oral exam, how many examiners, where, when? With respect to when, there is another oral in the Final assessment – is this the same (there are time discrepancies)?) The oral examination shall involve the candidate answering specific questions during a period of approximately 30 minutes. This oral is to ensure that the examiners are able to complete their assessment on whether the candidate may be accredited. This may involve questions being asked on: (a) The written examination, and/or (b) The two cases examined during the practical, and/or (c) Portfolios (d) Clinical supervisor’s and External Assessor’s reports (e) Details submitted in the curriculum vitae, reports and publications, and/or (f) Other areas related to radiological imaging equipment commissioning, and quality assurance, and radiation safety. At the end of the oral exam the examiners shall vote on whether the candidate has successfully completed the accreditation. The examiners shall prepare a report and forward it to the Chief Examiner, Radiological Physics Accreditation Panel who shall then forward it to the Chairperson of the Education Committee. The registrar shall carry out a practical examined by the external assessor, clinical coordinator and an examiner from the Accreditation Panel approximately six months prior to undertaking the final assessment. The registrar must pass this practical assessment to attend the final assessment. (Is this the same practical as above, or in addition to? It would seem unnecessary to have 2 major practicals.) Final Assessment (Who does the final assessment – who does the review, the oral, and assesses the presentation? Plus comments as above re duplication of the oral) The final assessment of the entire program shall consist of: * A review of the registrar’s assessment to date including the yearly progress reviews, theory exams, portfolios and final report. * A general oral on all aspects including the portfolios and the final report (1.5 hours). * A 20 minute presentation by the registrar on a research topic of the registrar’s choosing plus 10 minutes for questions (0.5 hours). The outcome of the final assessment may be: * A pass * A fail. In this case the registrar may attempt the final assessment again in a minimum of six months time. Overall Assessment The registrar must obtain a satisfactory pass in each of the following elements of the assessment process: (what is a satisfactory pass?? –criteria have been stated for the written, but for the rest??) * Each Portfolio * Each written exam * The practical exam (Which practical, and does it include the Oral?) * The final report * The final assessment On obtaining a satisfactory accreditation assessment a recommendation shall be made to the ACPSEM Council that the registrar has satisfied the requirements of the training program in Radiological Medical Physics and should be accredited in Radiological Medical Physics. Assessment Criteria There are two parts to the assessment criteria. Firstly, to determine that the candidate can satisfactorily complete aspects of radiological medical physics work that is commonly encountered in clinical practice; secondly, to satisfy the Panel that the candidate can work safely without supervision. Being able to assess that an individual can work safely without supervision is more difficult than assessing practical competency. It requires an assessment of the behavioural characteristics that a candidate exhibits in undertaking the practical exam questions and an assessment of the candidate’s scientific ability, in addition to assessment of their knowledge and skills[2]. In particular: 1. Broad understanding of a task or issue 2. Theoretical aspects relating to a task or issue 3. Practical aspects of undertaking a task or issue (including proficiency in completing task) 4. Interpreting the data (including a knowledge of relevant standards or expected results) and its applicability to other aspects such as planning or treatment image quality, radiation safety, etc?? 5. Demonstrate a clear understanding of the processes involved 6. Demonstrate the ability to clearly & logically explain the work as it is undertaken and by answering to be able to answer questions on the same 7. Satisfy the Accreditation Panel that the candidate would satisfy the following and that the Accreditation Panel would be confident that the candidate could safely work without supervision1:- i) Identify & define a problem and formulate strategies for solving it ii) Approach work analytically and thoroughly iii) Critically evaluates processes and outcomes iv) Takes due care with their work. v) Applies quality management strategies to their work vi) Interpret novel or non-standard data vii) Make value judgements in unfamiliar situations viii) Self assured & "thinks well on their feet" ix) Communicate scientific information clearly and accurately to others x) Recognise fault situations and take suitable corrective action xi) Appreciate the limitations of one's knowledge & ability. xii) Take responsibility for own work Appeals Providing Council’s policy on examination appeals is satisfied, the candidate may appeal against a decision of “not of required standard” made at any time by the External Assessor or by an adverse final result issued by the Accreditation Panel [3]. References. 1. ACPSEM Requirements for Accreditation in Radiology Physics: Candidates' Kit 2002 1 Based in part on the IPEM's definition of a professional physicist. 2. Insitute of Physics and Engineering in Medicine. Programme of Advanced Training and Registration 2003 3. ACPSEM Appeals Process for ACPSEM Accreditation 2004 1 Based in part on the IPEM’s definition of a professional physicist.