Event Filtering

Filter events at SQS, Kinesis, DynamoDB Streams, and other sources so AWS Lambda only runs for relevant records and you stop paying for noise.

Overview

Filter events at source (SQS, Kinesis, DynamoDB Streams, Amazon MSK, and more) to eliminate unnecessary Lambda executions.

Pay only for relevant invocations.

Lambda event filtering is supported for several event sources including SQS, Kinesis, DynamoDB Streams, Amazon MSK, and Amazon MQ; see the AWS documentation for the latest list, JSON pattern syntax, and limits.1

SQS Event Filtering

Filter messages before Lambda invocation. For SQS, the filter pattern is evaluated against the JSON representation of the event, where body is the (optionally JSON-parsed) message body and attributes / messageAttributes map to the SQS metadata:

{
  "FilterCriteria": {
    "Filters": [
      {
        "Pattern": "{\"body\": {\"status\": [\"COMPLETED\"]}}"
      }
    ]
  }
}

DynamoDB Streams Filtering

Process only specific changes:

{
  "FilterCriteria": {
    "Filters": [
      {
        "Pattern": "{\"eventName\": [\"INSERT\", \"MODIFY\"]}"
      }
    ]
  }
}

Kinesis Filtering

Filter records by attributes:

{
  "FilterCriteria": {
    "Filters": [
      {
        "Pattern": "{\"data\": {\"priority\": [\"HIGH\"]}}"
      }
    ]
  }
}

Implementation

Add filters in event source mapping:

aws lambda create-event-source-mapping \
  --function-name my-function \
  --event-source-arn arn:aws:sqs:region:account:queue \
  --filter-criteria file://filters.json

For high-volume workloads, combine event filtering with batching and reduce log output: first drop irrelevant events at the source, then process the remaining records in efficient batches and keep log volume under control.

Footnotes

  1. Using event source mapping filter criteria with AWS Lambda