Methods are published before results, so the results can be judged against a design that did not move. Data analysis and methods: Kurt E. Sather (lead), API Automations, with Puja D. Patel, MD, Maimonides Medical Center.
Questions and hypotheses
The questions and hypotheses were written down before any data were pulled, and every hypothesis is reported whatever the result.
- How often are device-related reports filed for gynecologic robotic surgery, how has that changed since 2010, and how severe are the outcomes, by product code and by the guidance's own hazard categories?
- What share describe a conversion, an aborted procedure or an emergency undocking, and how do outcomes differ in those reports?
- Do reports differ between the guidance's umbrella and covered procedures?
- Who comments on comparable FDA device-guidance dockets?
- How do adverse events in registered robotic gynecologic trials compare, in kind, with the surveillance reports?
- How has minimally invasive gynecologic procedure volume grown?
| Hypothesis | Test | |
|---|---|---|
| H1 | Software or interface faults are the most common device problem | The largest share, with a 95% interval clear of the next category's |
| H2 | Emergency-removal reports are rare, under 5%, but over-represented among deaths and transfusions | One-sided Fisher exact test |
| H3 | Bedside-team and instrument-exchange events are under-reported by operating-room staff relative to manufacturers | Chi-square test, or Fisher exact for small counts |
| H4 | Manufacturers and trade groups out-comment clinicians and societies on comparable dockets | Counts pooled over five comparison dockets |
| H5 | Outcomes in covered procedures are not predictable from umbrella procedures | Two-sided Fisher exact test |
The five comparison dockets are power morcellators (FDA-2014-D-1804), surgical staplers (FDA-2019-D-1262), remanufacturing (FDA-2018-N-3741), cybersecurity (FDA-2021-D-1158) and device software functions (FDA-2024-D-4488). Where a docket holds more than the draft guidance, only comments on the draft are counted.
Four rules
- Inputs are named. Every figure names each source, the endpoint or series used, the access date and the filters applied. A snapshot manifest records the URL, access time, checksum and row count of every pull.
- Periods are aligned before combining. The analysis window is the newest period every input shares, never the newest of each.
- A missing input withholds the figure. Nothing is imputed and nothing defaults. A table that needs an absent input is not written, and the withholding is stated.
- History is recomputed, not appended. Every run rebuilds every table from the dated snapshot, because the source systems restate.
Sources
| Input | Source | Use |
|---|---|---|
| Medical device reports, product codes NAY, QNM, SAB, SAQ, plus a brand backstop for mis-coded reports | openFDA /device/event | The core corpus |
| Classification, 510(k) clearances, recalls, enforcement | openFDA | Clearance lineage; recalls tied to hazard categories |
| Public dashboard counts | FDA Adverse Event Monitoring System | Reconciliation against openFDA after the 2026 migration |
| Docket documents and public comments, this docket and five comparison dockets | regulations.gov API v4 | Who comments on device guidance |
| Registered robotic gynecologic trials with posted results | ClinicalTrials.gov API v2 | Adverse events in trials, compared in kind |
| Medicare Part B provider and service utilization, gynecologic minimally invasive procedure codes, 2013–2024 | data.cms.gov | Volume trend (robotic is not distinguishable from laparoscopic in Part B) |
| General payments from robotic-device manufacturers | CMS Open Payments | Industry relationships in the specialty, and the authors' own records |
| Manufacturer-reported US gynecology procedure volume | SEC EDGAR open data: every Form 10-K since FY2010 is pulled into the snapshot and the count is read from the filing text by a published rule, with the sentence recorded beside the figure | The main denominator |
| National estimates of robotic-assisted procedures | AHRQ HCUP published reports and HCUPnet summary tables | Context for how common robotic gynecologic surgery is; not used as a denominator |
| Published literature and conference abstracts | PubMed, Ovid MEDLINE and CINAHL Complete; meeting abstracts from the AAGL, SGS and SLS sources; citation chasing through Europe PMC, OpenAlex and Crossref; open-access methods text from PubMed Central | The systematic search for prior art |
Prior art
A systematic search of PubMed, Ovid MEDLINE and CINAHL Complete runs before the analysis, in two branches: robotic surgery with device-report terms across specialties, and robotic surgery with gynecologic procedures and safety terms. Meeting abstracts are searched in the gynecologic surgery societies' own sources (AAGL, SGS, SLS), year by year, and forward and backward citations are chased from the device-report studies already known, through Europe PMC, OpenAlex and Crossref. Embase and Scopus are not licensed at the clinical lead's institution; they are searched only if a collaborator with authorized access can run them, and otherwise the report says they were not searched. The strategy is peer-reviewed by a medical librarian. The search is reported to the PRISMA-S standard, and every reference resolves to a DOI or PubMed ID before it is cited.
Cohort
A report enters the cohort if it carries one of the four product codes or matches the brand backstop, and if its narrative names a gynecologic procedure. Reports are de-duplicated on the report key and then on manufacturer, event date, device lot or serial, and a normalized narrative hash, so manufacturer follow-ups collapse to the initial event. Reports marked insufficient information are excluded from the primary analysis and counted as a data-quality measure; a sensitivity analysis repeats every table with them included. The window runs from 2010 to the snapshot date.
Coding schema
Every hazard category is annotated with the guidance section and line numbers it maps to, so each count corresponds to a specific ask in the comment:
- emergency removal or undocking (IV.A, IV.D(2)(b), IV.R)
- emergency and safety stops (IV.D(2)(c))
- unintended motion; motion scaling or tremor; haptic or force feedback (IV.D(1)(b), IV.E)
- software prompt or fault (IV.E, IV.J)
- power or connectivity loss (IV.I)
- instrument exchange or port switching, including bedside-team factors (IV.H, IV.I)
- arm collision (IV.I)
- instrument fracture or fragment; energy arcing or thermal; visualization loss
Procedure outcome (completed robotically, converted to laparoscopy, converted to open, aborted, rescheduled, unknown), patient outcome (none, injury by type, transfusion, reoperation, death), umbrella versus covered procedure, reporter type and a team-factor flag complete each record. Umbrella and covered procedures are the ones the guidance's Appendix A names for gynecology (Table 9): radical total laparoscopic hysterectomy, total laparoscopic hysterectomy and sacrocolpopexy are the umbrella procedures, and the procedures listed beside them are covered. Myomectomy, which the appendix lists on its own (Table 10), is coded but sits outside the umbrella-against-covered test. Rules are deterministic and published, and a match does not count where the narrative denies it ("no fragments fell into the patient"), so the coding is reproducible from the snapshot. A stratified sample of 300 reports is coded by hand, blind to the rules' codes, and a second sample of 200 confirms the result. Cohen's kappa must reach 0.7 on every coded field before any hypothesis is tested, and disagreements are adjudicated by the clinical lead, Dr. Puja D. Patel.
Analysis
Descriptive counts and shares by year, product code, hazard category and outcome, with exact Clopper–Pearson confidence intervals; annual trend by Poisson regression, with and without the manufacturer procedure-volume offset, and with quasi-Poisson intervals when the counts are overdispersed; reporter-type comparisons; and Fisher exact tests of whether death or transfusion differs between umbrella and covered procedures and between emergency-removal reports and the rest, with Holm's adjustment across the hypothesis tests. Sensitivity analyses repeat the hazard table on every robotic device report, not only those that name a gynecologic procedure (about half of such reports name none), count malfunctions found before a case began separately, and re-run the trend allowing for known changes in reporting practice, including summary reports that each stand for many malfunctions. No cross-platform rate comparisons: the surveillance data carry no per-system denominator. Every hypothesis test is repeated with the insufficient-information reports kept. Reporting follows the RECORD extension of STROBE. The analysis plan is drafted for pre-registration on OSF and is registered before the data are pulled; the registration will be linked here.
Limitations, stated first
Passive surveillance, voluntary for users and mandatory for manufacturers; narratives often manufacturer-authored; no denominator; duplicates and follow-ups; procedure identification from narrative text, which about half of robotic device reports do not give; since August 2018, summary reports that each stand for many malfunctions; Medicare data skew older than the robotic gynecologic population; Open Payments identifies relationships, not influence.
Code and data
The extraction and analysis package, the coding schema, the audit sample template and the derived tables are released with the preprint under an MIT license for code and CC BY 4.0 for data.