StillSafe

Problem & Motivation

Stillbirth, defined as the loss of a baby at or after 20 weeks of pregnancy, is a devastating outcome for families worldwide.

Globally, approximately 2 million stillbirths occur each year—equivalent to one every 16 seconds. In the United States, over 21,000 stillbirths are reported annually, meaning 1 in every 175 births results in stillbirth (CDC).

The loss of a baby is only part of the profound toll stillbirth takes on families. The financial burden is significant, with costs estimated between $7,000 and $9,000 per stillbirth. However, the emotional toll is immeasurable. Parents who experience stillbirth face significantly higher risks of depression, anxiety, and post-traumatic stress disorder compared to those who have live births (Westby et al., 2021).

While the underlying causes of stillbirth are often known, they are not always identifiable early in pregnancy, limiting opportunities for timely intervention. A large proportion of stillbirths occur in pregnancies that appear otherwise healthy, making early identification of risk factors a critical challenge (NHS). However, research suggests that many stillbirths could be prevented with early detection of risk factors, timely life-saving interventions, and access to high-quality healthcare (You et al., 2020).

If current trends persist, an estimated 20 million additional stillbirths will occur over the next decade. Addressing this challenge requires innovative solutions that enable earlier identification of at-risk pregnancies and empower healthcare providers and families to take preventative action.

Solution and Impact

That’s where StillSafe comes in.

Our mission is to empower pregnant individuals and their families by providing a data-driven tool that predicts the risk of stillbirth early in pregnancy. Using cutting-edge machine learning models and accessible technology, StillSafe enables timely identification of high-risk pregnancies, facilitating early interventions and offering critical support to those who need it most. By leveraging this tool, we aim to make healthcare more equitable by providing personalized risk assessments to a broad audience, ultimately improving maternal and fetal outcomes while reducing the emotional and financial burdens associated with stillbirth.

We believe that the development of a robust predictive model for stillbirth risk has the potential to transform prenatal care. By equipping families with accurate and actionable information, StillSafe can play a pivotal role in significantly reducing the global stillbirth rate and ensuring better health outcomes for mothers and their babies.

Data Source & Data Science Approach

All data used in this project was sourced from the Centers for Disease Control and Prevention (CDC) and is publicly available on the CDC website. The dataset includes both live (successful) birth records and fetal death records.

Data Preparation

For the fetal death dataset, which includes records of stillbirths, miscarriages, and spontaneous abortions, we filtered the data to align with the definition of stillbirth: pregnancies that ended at or after 20 weeks of gestation. Specifically, we used the "Obstetric Estimate Tabulation Flag" to isolate records of pregnancies meeting this criterion. This preprocessing step ensured our dataset was focused on our desired problem space—predicting the risk of stillbirth.

We sourced data from 2019 to 2022 because, starting in 2019, the CDC mandated the inclusion of the "race" feature, which is a critical variable for identifying high-risk pregnancies. Additionally, the live birth data was already in a clean and usable format, whereas the fetal death data required extensive preprocessing. To address this, we developed custom parsing functions to standardize the fetal death files and output them in a format compatible with the live birth data.

To reduce bias in the model, we balanced the dataset to include an equal number of live births and stillbirths. Balancing ensures that the model does not favor the majority class (live births) at the expense of accurately identifying the minority class (stillbirths). Sparse and incomplete rows, common in health data, were also removed to improve data quality and reliability.

The final dataset contained 136,216 pregnancy records from 2019 to 2022, evenly split between live births and stillbirths. Each record in the dataset represents a pregnancy and includes a target variable indicating whether the outcome was a live birth or stillbirth.

Data Science Approach

Our goal is to classify a pregnancy as high or low risk for stillbirth using a set of 25 features, including demographic, medical, and pregnancy-related attributes. We explored and tested a variety of classification models to determine the best approach, including:

• Traditional Machine Learning Models: Logistic Regression, Bernoulli Naive Bayes, k-Nearest Neighbors (KNN), Random Forest, Decision Trees, and AdaBoost.
• Gradient Boosting Models: LightGBM, XGBoost, and CatBoost.
• Neural Networks: Convolutional Neural Networks (CNNs).
• Ensemble Methods: Various ensemble approaches to combine the strengths of multiple models.

We performed hyperparameter tuning, such as GridSearch and RandomizedSearch, on the top-performing models to optimize their performance. The final pipeline integrates the best-performing model, enabling accurate predictions of stillbirth risk. This tool will empower healthcare providers and families to take timely action, potentially preventing adverse outcomes.

Evaluation

Identifying the best predictive model for stillbirth risk involved rigorous evaluation using multiple metrics and iterative testing across various algorithms. Our goal was to balance accuracy, which reflects overall performance, and recall, which minimizes the risk of missing high-risk pregnancies—a critical factor for enabling timely interventions.

Initial Baselines

We began by testing traditional machine learning models, including Logistic Regression, k-Nearest Neighbors (KNN), and Naive Bayes. After preprocessing the data using StandardScaler, Logistic Regression achieved the highest accuracy (63%), while KNN exhibited the best recall (73%). These results provided a solid foundation and identified areas for further improvement.

Exploring Advanced Models

Recognizing the limitations of baseline models, we investigated advanced algorithms. Gradient boosting models—LightGBM, XGBoost, and CatBoost—outperformed the baselines by capturing complex, non-linear relationships in our dataset of over 100,000 rows. Fine-tuning via grid search and random search yielded significant improvements in both accuracy and recall.

However, ensemble methods that combined gradient boosting models yielded only marginal gains in accuracy (77.28%) while decreasing recall by 5%. The trade-off was not justifiable, leading us to favor the standalone CatBoost model, which achieved 77.17% accuracy and 81.33% recall.

Final Model Selection

While CatBoost showed promise, it underperformed during deployment. The fine-tuned XGBoost model emerged as the top-performing classifier, achieving 79.70% accuracy and 85.86% recall. By leveraging AWS's built-in hyperparameter tuning, XGBoost consistently outperformed other models, making it the most reliable choice for deployment.

Model Insights

The XGBoost model was deployed using a 70-20-10 train-validation-test split to ensure robust evaluation and generalizability. While ensemble methods offered minor accuracy improvements (0.1%), their significant recall reduction underscored the importance of prioritizing metrics aligned with the problem's critical needs.

By enabling early detection of high-risk pregnancies, our predictive model forms the cornerstone of the StillSafe tool. This empowers families and healthcare providers to take proactive measures, improving maternal and fetal outcomes.

Key Learnings & Impact

Through the development of StillSafe, we have gained critical insights into the complexities of healthcare data and the importance of tailoring machine learning solutions to address real-world challenges.

1. Understanding Healthcare Challenges: Working closely with subject matter experts has highlighted the nuanced hurdles faced by healthcare providers and families, including disparities in access to care, variability in data quality, and the critical need for timely interventions. These insights have informed our approach to building a tool that is both accurate and accessible.
2. Data Complexity and Ethical Considerations: Balancing the technical challenges of working with incomplete and imbalanced datasets against the ethical responsibility of delivering reliable predictions was a key learning. We recognized the importance of transparency in explaining risk factors and ensuring the tool builds trust with users.

By leveraging machine learning in this space, we aim not only to reduce the global stillbirth rate but also to inspire broader adoption of advanced analytics in prenatal care. This project has the potential to catalyze a virtuous cycle of technological innovation, improved maternal health outcomes, and enhanced healthcare equity worldwide

Acknowledgements

The StillSafe team extends our deepest gratitude to the subject matter experts who generously shared their time, expertise, and insights with us throughout the course of this project. Their guidance in understanding the complexities of healthcare data and their thoughtful responses to our numerous questions were invaluable in shaping the direction and impact of our work.

We would like to especially thank:

• Dr. Glenn Grossman (OBGYN)
• Monica Wojcik and Micaela Mateo Smith (Stillbirth Working Group of Council)
• Megan Aucutt (Healthy Birth Day / Count the Kicks)
• Natasha Williams (NICHD Subject Matter Expert)
• Elizabeth Gregory (CDC National Center for Health Statistics)
• Sarah Lopez, Susannah Leisher, & Nathan Blue (University of Utah Health, Stillbirth Center of Excellence)

This project would not have been possible without their contributions, insights, and willingness to help us navigate such a multifaceted and sensitive domain. Their support has been instrumental in furthering our mission to make a meaningful impact in prenatal care.

References

Centers for Disease Control and Prevention. (n.d.). Data and statistics on Stillbirth. Centers for Disease Control and Prevention. https://www.cdc.gov/stillbirth/data-research/index.html

NHS. (n.d.). Stillbirth - Causes - NHS. NHS choices. https://www.nhs.uk/conditions/stillbirth/causes/

Westby, C. L., Erlandsen, A. R., Nilsen, S. A., Visted, E., & Thimm, J. C. (2021). Depression, anxiety, PTSD, and OCD after stillbirth: A systematic review. BMC Pregnancy and Childbirth, 21(1). https://doi.org/10.1186/s12884-021-04254-x

You, D., Hug, L., Mishra, A., Blencowe, H., & Moran, A. (2020). A Neglected Tragedy: The Global Burden of Stillbirths. United Nations Children’s Fund.