The IMPACT-IBM team successfully improved the performance of their language models by incorporating domain-specific vocabulary into their INDUS model. This resulted in better results compared to more general language models on various benchmarks for biomedical tasks, scientific question-answering, and Earth science entity recognition tests. INDUS is designed to handle a variety of linguistic tasks and uses a combination of retrieval and generation techniques to process researcher questions, find relevant documents, and generate answers.
In order to meet the needs of latency-sensitive applications, the team also created smaller and faster versions of both the encoder and sentence transformer models. These modifications allow for quicker processing times without sacrificing the quality of the model’s output. The success of the INDUS model demonstrates the importance of incorporating domain-specific knowledge in language models to improve their performance on specialized tasks.
Article Source
https://science.nasa.gov/open-science/ai-language-model-science-research/
