Amazon OpenSearch Ingestion is a managed serverless pipeline that enables users to ingest, filter, transform, enrich, and route data to an Amazon Open Search Service domain or Amazon OpenSearch Serverless collection. It supports a variety of data sources and provides built-in processors for complex data transformation tasks. The latest enhancements to OpenSearch Ingestion focus on improving the user interface and blueprint discovery features.
Instead of starting from scratch to create a pipeline definition, users can now utilize configuration blueprints, which are preconfigured templates for common ingestion scenarios like trace analytics or Apache logs. These blueprints streamline the pipeline provisioning process without the need to create configurations from scratch. The new user interface allows for easy search and discovery of plans, enhancing the user experience within the AWS Management Console.
The updated catalog page in the OpenSearch Service console provides a clear overview of all supported sources and sinks for OpenSearch Ingestion. Plans are presented as numbered tiles with icons, organized thematically for easy navigation. The navigation menu lists blueprints by use case or service, allowing users to quickly search for specific plans or choose shortcuts for popular services like Amazon DynamoDB.
After selecting a blueprint, users are guided through the Pipeline Configuration and Settings page, which offers a personalized getting started guide for each data source. This guide outlines key steps to set up a successful integration and can be customized based on the chosen pipeline model. The configuration templates come preloaded with options for modifying specific sources, IAM roles, and sinks, making it easier to tailor the pipeline to individual needs.
In addition to the existing YAML support, OpenSearch Ingestion now provides JSON pipeline configuration capabilities in the console. This allows users to view and edit configurations in JSON format, offering more flexibility and consistency when copying and pasting configurations. The visual overhaul aims to simplify the configuration process and ensure accuracy when making modifications to pipeline configurations.
These new features are available in all commercial AWS Regions where OpenSearch Ingestion is supported. Ingestion OCUs remain priced at 0.24 cents per hour, with billing at per-minute granularity. Users can control costs by configuring the maximum number of OCUs a pipeline can scale to, providing flexibility and cost efficiency.
Overall, the updates to OpenSearch Ingestion focus on enhancing the user experience through improved UI features and simplified configuration options. Users can explore the full capabilities of OpenSearch Ingestion to create scalable pipelines for their data ingestion needs.
Article Source
https://aws.amazon.com/blogs/big-data/introducing-blueprint-discovery-and-other-ui-enhancements-for-amazon-opensearch-ingestion/