Examination Development Administration & Reporting

Introduction

The American Board of Physician Specialties (ABPS) develops examinations for each of the medical specialties represented by the ABPS Boards of Certification. The examination development process is collaborative, involving ABPS Diplomates, as well as American Board of Medical Specialties (ABMS) and the American Osteopathic Association Bureau of Osteopathic Specialist (AOABOS) Diplomates, who serve as the subject matter experts.

For each specialty, ABPS develops a written certification examination consisting of multiple-choice questions. For several specialties, ABPS also develops oral examinations that provide candidates an opportunity to demonstrate their knowledge and skill in diagnosing and treating patients. The oral examinations permit the candidate to probe for information, develop differential diagnoses, and request laboratory tests or diagnostic examinations.

ABPS also develops multiple-choice recertification examinations. ABPS Diplomates are required to recertify every eight years.

Practice Analysis

All ABPS examinations are based on practice analyses. Meetings with subject matter experts, e.g., ABPS, ABMS and AOABOS Diplomates, lead to the identification of technical knowledge, skills, and abilities essential to the performance of a certified physician. These are then validated through review by a representative sample of professionals in the field, usually certified by the ABMS, the American Osteopathic Association (AOA), or ABPS. Through this process, ABPS determines the importance, criticality, and frequency of the identified knowledge, skills, and abilities required in practice. This information serves as the cornerstone of the content and process validity of the examinations, and ensures that every question on ABPS examinations represents an assessment of knowledge, skills, or abilities clinically relevant to the practice of that medical specialty.

Test Specifications (Examination Blueprints)

The development of blueprints requires the collaboration of professionals with psychometric expertise and those with the medical knowledge. The psychometrician challenges the physicians to consider what skills, knowledge, and abilities can be assessed within the specialty. This approach allows the collaborating team to identify any knowledge, skills, and abilities identified in the practice analysis but not yet addressed by the exam blueprint.

The test specifications include details regarding the weighting of each content category. For example, the test blueprint of each written examination indicates the approximate number of questions for each category and sub-category based upon the data from the practice analysis. The number of topics to be covered, and the number of questions needed to provide reliable diagnostic information in each content category, contribute to decisions regarding the number of questions that will appear on an examination.

The majority of written certification examinations are comprised of 200 items, with the longest examination containing 350. Recertification examinations consist of 100 items. Several of the specialties require candidates for initial certification to successfully complete an oral examination once they have passed the written examination.

Item Development

Using the examination blueprints as a guide, a variety of questions are developed for the examinations. Questions are developed to meet content and task categories. The initial items are designed by subject matter experts, typically ABPS Diplomates, who have particular knowledge or expertise relevant to the examination being developed. Items are reviewed by an editor, who revises the items with attention to item construction rules, readability, grammar, and potential bias. Once the item has been through an editorial review, it is then submitted to a committee of ABPS Diplomates of the same specialty for final editing and approval.

Examination Forms Development

ABPS staff configures each form of each specialties’ examination to meet the needs of the examination blueprint. Subject matter experts then review the draft examination to identify items that are too similar to appear on the same examination form, as well as those that might provide information that leads to the answer of another item. Additionally, an editor reviews the examination form for consistency of style, embedded directions, arrangement of items, and inclusion of all necessary components for a complete examination.

Administration of the Examinations

Examinations are administered twice a year under standardized conditions. Although written examination items and examinations undergo continuing review, the accuracy of an item can be impacted at any time due to changes in accepted medical practice. In order to identify such concerns, candidates have the opportunity to comment on examination items that may be inaccurate or out of date while they are taking the examination. Candidates are advised that their comments will be reviewed before ABPS performs the final scoring and reporting of results.

For oral examinations, specific scripts following a standardized outline are developed for the examiners. ABPS trains the examiners regarding appropriate administration procedures. As with the written examinations, the content and scoring of the oral examinations undergo reviews prior to each administration to verify accuracy and relevance.

Establishing Passing Scores

ABPS uses variations of the Angoff method to establish passing scores on the written and oral examinations.

Review of Comments and Item Analyses

After administration of the written examinations, the candidates’ responses are initially scored and item statistics are generated. The results of the item analysis and a compilation of candidates’ comments are provided to the Examination Committee for review. The candidates’ responses are rescored based upon the recommendations of the reviewers. The results are used to develop the score reports for candidates and examination analysis reports.

Scoring and Reporting

All responses by candidates on the written examinations administered via computer-based testing are submitted and scored electronically. Follow-up audits of scoring procedures and examination results may be conducted. If an audit indicates an error in scoring resulting in a change in the results of a candidate, ABPS will rescore and provide the candidate the revised results.

Recertification Examinations

The written examinations, required for recertification every eight years, are drawn from the same bank of items as the certification examinations. These examinations are administered in two hour sessions and contain 100 multiple-choice questions. The recertification examinations are assessing skills and knowledge applicable to acceptable medical practices in the specialty. These examinations do not address cutting-edge changes in medicine, but rather address established practices that remain current.

Realizing the need for competent physicians to be abreast of the latest developments in medicine, recertifying physicians are required to complete self-assessments in their specialty annually. Physicians may select self-assessments to address their needs and are encouraged to select areas that address current day issues, medical research, and possible changes in medical practice.

Save the Date
House of Delegates & Annual Scientific Meeting
Innovation & Overcoming Challenges
June 10-15, 2022
Patient Care Is Our Priority

Medical organizations throughout North America understand that our rigorous certification standards prove that ABPS Diplomates are capable of delivering the best patient care possible.

Far too often, medicine is led by less than 5% of non-practicing physicians taking away and replacing the voice of the 95% of physicians practicing and placing patient safety and care first on the front lines every day. The American Board of Physician Specialties has raised the standards in physician board certification not only in the quality of their boards of certification, but in hearing and allowing for the voice of those active physicians caring directly for patients. Having been a part of the ABPS over the last 28 years has allowed me to grow as a woman leader in a field often wrought with challenges. It helped me and others raise the bar of the standards of care in my specialty, Emergency Medicine, through their Board Certification in Emergency Medicine (BCEM). ABPS also helped raise the standards of care for 21st century medicine through their certifications in other specialties, particularly in Integrative Medicine & Disaster Medicine. Having physician voices heard matters to medicine and is essential in the betterment of patient safety and care.

Sarah E. Gilbert, MD, FAAEP
Sarah E. Gilbert, MD, FAAEP
Emergency Medicine
On October 18, 2007, President George W. Bush released Homeland Security Presidential Directive 21 (HSPD-21), calling on our nation, among other initiatives, to “collectively support and facilitate the establishment of a discipline of disaster health”. It is a great testament to the wisdom and foresight of the American Board of Physician Specialties that it immediately set to work and created, within the short span of only one year, an educational blueprint and set of certification examinations, both written and oral, for a new subspecialty of disaster medicine—and it is why I chose to be part this vital initiative and this wonderful organization. This is but one of the many innovative programs initiated by the American Board of Physician Specialties over the years, and why I am proud to support its work on behalf of our nation’s public health.

Art Cooper, MD
Disaster Medicine
When the American Board of Physician Specialties offered to host the American Board of Integrative Medicine, ABPS became a landmark organization working to move medicine into the twenty first century. Certifying physicians who have completed rigorous academic training in Integrative Medicine ensures that the field of Integrative Medicine will continue to develop academically, clinically, and professionally. The leadership of ABPS continues to impress me - they are diligent in constantly innovating to provide certifications for physicians who want to advance their careers and their areas of expertise. I am honored to be a part of this organization.

Ann Marie Chiasson, MD
Integrative Medicine
There are many ways board certification advances a physician career. ABPS Board examination verifies your accuracy, precision, and reflects your mastery of your residency training verifying your expertise. ABPS Board certification demonstrates your level of expertise beyond your practice experience, primary education degrees, and training which are necessary for insurance reimbursement and practice privilege requirements. Attaining your ABPS Board Certification will clarify your purpose, secure your practice growth, and expand into leadership positions. Board certification can serve as an indication of a physician’s commitment to medicine, beyond the minimal standards and competency of training, their measurement to quality of care, and attaining an award for excellence.

Chris Kunis MD
Internal Medicine
When I think historically, advancement in medicine and patient safety and care has been driven by the diversity of people and scientific thought. That’s what I found at the ABPS and more. For over 60 years that is just who we are. I found a physician certifying body that provides a choice and voice to all physicians ensuring that patients are always placed first.

Jerry Allison, MD
Emergency Medicine
When I decided to pursue a full time role as a physician executive it was important to me to obtain additional professional training, education and work experience. Board certification through the ABPS in Administrative Medicine is validation of my efforts and a demonstration of dedication to professional development. We need more physicians to become full time health care executives, knowing there is a board certification option in Administrative Medicine encourages physicians to take the leap from full time clinical practice to healthcare organizational leadership.

Richard Paula, MD
Administrative Medicine
The American Board of Physician Specialties has provided me with the opportunity to demonstrate mastery of internal medicine through board certification. As a hospitalist, board certification is an expected credential, and hospitals recognize the American Board of Physician Specialties (ABPS) as one of the three standard credentialling bodies for Internal Medicine. Additionally, the ABPS has helped me develop leadership skills as a Board member and Committee Chairperson. ABPS has also helped me sharpen critical thinking skills as a test question developer and reviewer. The Allopathic (MD) and Osteopathic (DO) physicians in the ABPS are lifelong learners and frequently pursue multiple board certifications. I enjoy the camaraderie of my peers in ABPS.

Loren Jay Chassels, DO
Internal Medicine