Amazon OpenSearch Ingestion is a fully managed, serverless data pipeline that simplifies the process of ingesting data into Amazon OpenSearch Service and OpenSearch Serverless collections. Some key concepts include:
- Source – Input component that specifies how the pipeline ingests the data. Each pipeline has a single source which can be either push-based and pull-based.
- Processors – Intermediate processing units that can filter, transform, and enrich records before delivery.
- Sink – Output component that specifies the destination(s) to which the pipeline publishes data. It can publish records to one or more destinations.
- Buffer – It is the layer between the source and the sink. It serves as temporary storage for events, decoupling the source from the downstream processors and sinks. Amazon OpenSearch Ingestion also offers a persistent buffer option for push-based sources
- Dead-letter queues (DLQs) – Configures Amazon…
