This study presents a series of analyses to characterize the potential benefits and limitations of using an artificial intelligence model for ranking blastocyst stage embryos.

Objective: To perform a series of analyses that characterize an AI model for ranking blastocyst stage embryos. The primary objective was to evaluate the benefit of the model for predicting clinical pregnancy, while the secondary objective was to identify limitations that may impact clinical use.

Design: Retrospective.

Setting: Consortium of 11 assisted reproductive technology centers in the U.S.

Patients: Static images of 5,923 transferred blastocysts and 2,614 non-transferred aneuploid blastocysts.

Intervention: None.

Main Outcome Measure: Prediction of clinical pregnancy (fetal heart beat).

Results: The AUC of the AI model ranged from 0.6 to 0.7 and outperformed manual morphology grading overall and on a per-site basis. A bootstrapped study predicted improved pregnancy rates between +5% to +12% per site using AI compared to manual grading when using the ICSI microscope. One site that used a low-magnification stereo zoom microscope did not show predicted improvement with the AI. Visualization techniques and attribution algorithms revealed that the features learned by the AI model largely overlap with the features of manual grading systems. Two sources of bias were identified and mitigated relating to the type of microscope and presence of embryo holding micropipettes. Analysis of AI scores in relation to pregnancy rates showed that score differences of 0.1 (10%) or greater correspond with improved pregnancy rates, while score differences of less than 0.1 may not be clinically meaningful.

Conclusion
This study demonstrates the potential of AI for ranking blastocyst stage embryos and highlights potential limitations related to image quality, bias, and granularity of scores.