Galaxy is a platform used in computational biology that aims to make genomics research accessible to those without computer programming experience. BioNTech, a research team, faced challenges with its growth, including IT bottlenecks and a lack of streamlined workflow. They collaborated with AWS Professional Services to migrate and customize Galaxy on AWS to streamline their research processes.
The solution involved deploying Galaxy on AWS using Amazon EKS and the CDK accelerator to implement well-managed EKS groupings. Galaxy allows computational biologists to create interfaces for command-line applications for various research operations. Challenges with the on-premise deployment led BioNTech to evaluate using Kubernetes on AWS for a scalable and declarative container orchestration solution.
The architecture included EKS clusters and various resources added by AWS Professional Services. To address security requirements, private EKS clusters and VPC endpoints were configured. User authentication was enabled using OpenID Connect and Azure Active Directory. Custom tools were developed using AWS Cloud9 to streamline the tool development process for computational biologists.
Job scheduling and auto-scaling were implemented using the Cluster Autoscaler Add and Module Priority Class to size EKS clusters based on job requirements. Custom rules were configured for job destinations and GPU-based workloads were supported using the NVIDIA GPU operator. Observability was enhanced using Fluent Bit to send container logs to Amazon CloudWatch.
Future improvements for Galaxy on EKS clusters are under evaluation by BioNTech to enhance flexibility and cost-effectiveness. The collaboration between BioNTech and AWS resulted in the successful modernization of the Galaxy platform, transforming it into a scalable and secure platform integrated with BioNTech’s cloud infrastructure.
Article Source
https://aws.amazon.com/blogs/containers/how-biontech-modernized-galaxy-server-deployments-at-scale-using-amazon-eks/