S3 Event Notifications: Automate with Lambda, SQS, SNS Integration

Key Takeaway

S3 Event Notifications automatically send alerts to Lambda, SQS, SNS, or EventBridge when events like object creation/deletion occur. Only one destination per event type; use SNS or EventBridge for Fan-Out.

What are S3 Event Notifications?

A feature that automatically triggers other AWS services when specific events occur in an S3 bucket.

Supported Event Types

Event	Description	Use Case
s3:ObjectCreated:*	Object creation (PUT, POST, COPY, MultipartUpload)	Generate thumbnail on image upload
s3:ObjectRemoved:*	Object deletion	Record deletion logs
s3:ObjectRestore:*	Glacier restore complete	Send restore notification
s3:Replication:*	Replication failure/completion	Replication monitoring
s3:LifecycleExpiration:*	Lifecycle expiration	Expiration logging
s3:ObjectTagging:*	Tag add/remove	Tag change auditing

Notification Destinations

Four Destinations

[S3 Bucket]
    │ Event occurs
    │
    ├── [Lambda Function] ── Direct code execution
    │
    ├── [SQS Queue] ── Queue message then process
    │
    ├── [SNS Topic] ── Fan-out to multiple subscribers
    │
    └── [Amazon EventBridge] ── Rule-based routing

Destination	Characteristics	Suitable For
Lambda	Direct async invocation	Immediate processing (thumbnails, transforms)
SQS	Store in message queue	Reliable processing, batching, decoupling
SNS	Fan-out to multiple subscribers	One event → multiple destinations
EventBridge	Rule-based filtering/routing	Complex routing, SQS FIFO

Destination Comparison

Direct Lambda Invocation:
[S3] → [Lambda]
→ Simple, minimum latency
→ Watch Lambda concurrency limits

Via SQS:
[S3] → [SQS] → [Lambda]
→ Buffer during traffic spikes
→ Preserve failed messages with DLQ
→ Batch processing possible (cost savings)

SNS Fan-out:
[S3] → [SNS] → [SQS 1]
              → [SQS 2]
              → [Lambda]
              → [Email]
→ Deliver one event to multiple destinations

Fan-Out Pattern

SNS-Based Fan-Out

Multiple tasks on image upload:

[S3: Image Upload]
        │
        ▼
   [SNS Topic]
        │
        ├── [SQS 1] → [Lambda: Thumbnail Generation]
        ├── [SQS 2] → [Lambda: Metadata Extraction]
        └── [Lambda: Record to DynamoDB]

EventBridge-Based Fan-Out

When using EventBridge:

[S3: Event Occurs]
        │
        ▼
   [EventBridge]
        │
        ├── Rule 1: *.jpg → [Lambda: Image Processing]
        ├── Rule 2: *.csv → [Step Functions: ETL]
        ├── Rule 3: All files → [SQS FIFO: Order Guaranteed]
        └── Rule 4: Deletion → [SNS: Admin Notification]

EventBridge vs Traditional S3 Event Notifications

Aspect	S3 Event Notifications	Amazon EventBridge
Destinations	Lambda, SQS, SNS	18+ AWS services
Filtering	Prefix, suffix only	Object size, metadata, etc.
Same Event Multiple Destinations	No	Yes
SQS FIFO	No	Yes
Archiving/Replay	No	Yes
Cost	Free	$1/million events
Setup	Bucket level	Enable in EventBridge

Permission Configuration

S3 → Lambda

Lambda Resource-based Policy:
{
  "Effect": "Allow",
  "Principal": {"Service": "s3.amazonaws.com"},
  "Action": "lambda:InvokeFunction",
  "Condition": {
    "ArnLike": {"AWS:SourceArn": "arn:aws:s3:::my-bucket"}
  }
}

S3 → SQS

SQS Access Policy:
{
  "Effect": "Allow",
  "Principal": {"Service": "s3.amazonaws.com"},
  "Action": "sqs:SendMessage",
  "Condition": {
    "ArnLike": {"aws:SourceArn": "arn:aws:s3:::my-bucket"}
  }
}

S3 → SNS

SNS Access Policy:
{
  "Effect": "Allow",
  "Principal": {"Service": "s3.amazonaws.com"},
  "Action": "sns:Publish",
  "Condition": {
    "ArnLike": {"aws:SourceArn": "arn:aws:s3:::my-bucket"}
  }
}

Filtering

Use prefix and suffix to trigger events only for specific objects.

Filter Example:
Event: s3:ObjectCreated:*
Prefix: images/
Suffix: .jpg

→ images/photo.jpg upload triggers ✅
→ images/photo.png upload doesn't trigger ❌
→ docs/file.jpg upload doesn't trigger ❌

Practical Architecture Examples

Image Processing Pipeline

[User] → [S3: uploads/]
                │ ObjectCreated
                ▼
           [Lambda: Image Resize]
                │
                ├── [S3: thumbnails/] ← Save result
                └── [DynamoDB] ← Record metadata

Log Analysis Pipeline

[Application] → [S3: logs/]
                     │ ObjectCreated
                     ▼
                [SQS Queue] ← Buffer
                     │
                     ▼
                [Lambda: Log Parsing]
                     │
                     ├── [OpenSearch] ← Search/Analysis
                     └── [CloudWatch] ← Metrics

SAA-C03 Exam Focus Points

1. ✅ Auto Processing: "S3 upload → auto-process = Event Notification + Lambda"
2. ✅ Fan-Out: "One event → multiple destinations = SNS or EventBridge"
3. ✅ SQS FIFO: "S3 → SQS FIFO directly not possible, use EventBridge"
4. ✅ Decoupling: "Handle traffic spikes = S3 → SQS → Lambda"
5. ✅ Permissions: "Target service needs S3 access permissions (resource-based policy)"

Frequently Asked Questions (FAQ)

Q: Are S3 Event Notifications free?

S3 Event Notifications themselves are free. However, destination service costs (Lambda execution, SQS messages, SNS publishing) are charged. EventBridge has per-event pricing.

Q: Can event notifications be missed?

S3 Event Notifications guarantee "at-least-once" delivery. Rarely, duplicate delivery can occur, so processing logic should be idempotent.

Q: Are notifications applied immediately after configuration?

After creating or changing notification configuration, it may take about 5 minutes to apply.

Q: Can I set multiple notifications on the same bucket?

Yes. Multiple notifications can be set with different event types or different prefix/suffix combinations. Only one destination is possible for the same event type and same filter combination.

Q: Is cross-region event notification possible?

S3 Event Notification destinations must be in the same region. For cross-region processing, use EventBridge's cross-region event bus or have Lambda invoke services in other regions.

Related Posts

• S3 Basics: Buckets, Objects, Permissions
• S3 Replication (CRR, SRR)
• Lambda Function Optimization

References

• S3 Event Notification Configuration
• S3 Event Notification Tutorial
• Amazon EventBridge and S3