Gene Mutation AI / NLP Deep Learning
Leveraging NLP for Enhanced Analysis of Genetic Mutations and Disease Correlations.
problem
Our client is a healthcare startup focused on developing software tailored for personalized diagnostics and treatment, aiming to leverage advanced technologies to enhance medical processes. Despite the vast amount of information available on the Internet, conventional search engines struggle with complex queries, especially in the medical domain. Navigating through the PubMed library, in particular, poses challenges due to the sheer volume of data. The client identified a need for a system capable of analyzing scientific abstracts from PubMed, specifically focusing on establishing connections between diseases and genetic mutations to aid in diagnosis.
problem
Our client is a healthcare startup focused on developing software tailored for personalized diagnostics and treatment, aiming to leverage advanced technologies to enhance medical processes. Despite the vast amount of information available on the Internet, conventional search engines struggle with complex queries, especially in the medical domain. Navigating through the PubMed library, in particular, poses challenges due to the sheer volume of data. The client identified a need for a system capable of analyzing scientific abstracts from PubMed, specifically focusing on establishing connections between diseases and genetic mutations to aid in diagnosis.
solution
To address the challenge, our team devised an NLP-based solution capable of parsing scientific abstracts from PubMed and conducting content analysis to identify the impact of genetic mutations on diseases. Through API requests, users can input disease and mutation data, along with MESH and OMIM codes, to obtain lists detailing the relationship between mutations and diseases. This system provides invaluable insights for medical professionals during patient examinations and diagnosis processes. The technical scope of the project was predominantly covered by using Python, leveraging the BioBERT model for identifying disease-mutation connections, and the SpaCy model for ethnicity recognition. All abstracts were stored within the ElasticSearch database, and the solution was deployed on Amazon Web Services (AWS) for scalability and reliability.
solution
To address the challenge, our team devised an NLP-based solution capable of parsing scientific abstracts from PubMed and conducting content analysis to identify the impact of genetic mutations on diseases. Through API requests, users can input disease and mutation data, along with MESH and OMIM codes, to obtain lists detailing the relationship between mutations and diseases. This system provides invaluable insights for medical professionals during patient examinations and diagnosis processes. The technical scope of the project was predominantly covered by using Python, leveraging the BioBERT model for identifying disease-mutation connections, and the SpaCy model for ethnicity recognition. All abstracts were stored within the ElasticSearch database, and the solution was deployed on Amazon Web Services (AWS) for scalability and reliability.
results
Initially, we developed an NLP abstract processing model based on the BioBERT model, pre-trained in biomedical language. Training our model involved utilizing a structured dataset comprising 6000 training and 2000 test mutation-disease compounds extracted from nearly 600 abstracts. Each paper in the dataset could contain multiple mutation-disease connections. Additionally, to ensure comprehensive understanding, we incorporated the SpaCy model to identify and extract named entities, particularly focusing on demographic information like ethnicity. Following the development phase, we established an infrastructure for storing and processing scientific articles and seamlessly integrated our solution into the client's system through API integration. The tool automatically downloads and processes medical abstracts, continually updating its database with new publications, all of which are stored in an ElasticSearch database.
results
Initially, we developed an NLP abstract processing model based on the BioBERT model, pre-trained in biomedical language. Training our model involved utilizing a structured dataset comprising 6000 training and 2000 test mutation-disease compounds extracted from nearly 600 abstracts. Each paper in the dataset could contain multiple mutation-disease connections. Additionally, to ensure comprehensive understanding, we incorporated the SpaCy model to identify and extract named entities, particularly focusing on demographic information like ethnicity. Following the development phase, we established an infrastructure for storing and processing scientific articles and seamlessly integrated our solution into the client's system through API integration. The tool automatically downloads and processes medical abstracts, continually updating its database with new publications, all of which are stored in an ElasticSearch database.
tech stack
tech stack
