An NLP Model Finetuned for Aviation Safety Reports
Nobody understands the safety risks and events occurring in your operation better than the frontline operational workforce. Voluntary Safety Reporting Programs (VSRPs) allow aviation organizations to identify, monitor, and address emerging safety issues based on direct input from frontline operational personnel. Reviewing and analyzing the large volume of safety reports organizations receive can be time-consuming and resource intensive.
Fort Hill Group is applying cutting-edge natural language processing (NLP) techniques to drastically enhance the efficiency, accuracy, and time required to label safety factors of interest within aviation safety reports. This technological innovation offers the potential for near real-time safety insights that can contribute to a more responsive and adaptive National Airspace System (NAS).
AVIAN-S Model
Identifies safety event factors described in narrative safety reports with a high degree of accuracy
Utilizes the AirTracs hierarchical framework
Custom built multi-classification hierarchical model utilizing an internally fine-tuned BERT-based embedding layer and custom developed text classification layer.
Extensive Training Data Set
Built using a comprehensive set of 60,000+ manually labelled factors identified by subject matter experts in actual aviation safety reports.
AVIAN-S Model
Example Report Classification
This AVIation Analytic Neural network for Safety events (AVIAN-S) model incorporates ML and NLP to automate the identification and labeling of human factors (HF) issues within VSRP reports. Our team has already developed and trained a preliminary model utilizing our in-house data sets created by subject matter experts over years of manually analyzing VSRP data. The training data includes over 80,000 manually labeled factors across over 24,000 narrative-based safety reports. This data set provides the foundation for interpreting safety reports given the domain-specific language and shorthand typical to narrative-based safety reports. With further testing, development, and validation, Avian-S will be capable of identifying patterns, trends, and correlations exponentially faster than the typical safety event analysis process. These time and efficiency savings will pave the way for significant improvements in aviation safety.
AVIAN-S Publications
Sawyer, M., Berry, K., Bynum, E., Hinson, R.J., & Kinsella, A. (2024). AVIAN-S: A Natural Language Processing Model for Analyzing Safety Event Reports. In Proceedings for the International Symposium on Human Factors and Ergonomics in Health Care, Chicago, IL.
Sawyer, M., Berry, K., Bynum, E., Hinson, R.J., & Kinsella, A. (2024). Using Natural Language Processing to Identify Mental Health Indicators in Aviation Voluntary Safety Reports. In Proceedings for the National Training Aircraft Symposium, Daytona Beach, FL.
Kinsella, A., Bynum, E., Hinson, R.J., Berry, K., & Sawyer M. (2023). Project AVIAN: Implications of Utilizing the Novel AVIAN-S Machine Learning Model in Analyzing Aviation Safety Event Reports. In Proceedings for the Annual Meeting for the Human Factors and Ergonomics Society, Washington, DC.
Hinson, R.J., Bynum, E., Kinsella, A., Berry, K., & Sawyer M. (2023). Project AVIAN-S: Development of a Natural Language Processing Model for Analyzing Aviation Safety Event Reports. In Proceedings for the Applied Human Factors and Ergonomics Conference, San Francisco, CA.
Hinson, R.J., Bynum, E., Kinsella, A., Berry, K., & Sawyer M. (2023). A natural language processing model for analyzing aviation safety event reports: a subset of results. In Proceedings for the International Symposium on Aviation Psychology, Rochester, NY.
