S3 Lifecycle Policies: Automate Storage Cost Optimization

Key Takeaway

S3 Lifecycle policies automatically transition objects to cheaper storage classes or delete them to optimize storage costs. Use Lifecycle for predictable access patterns; use Intelligent-Tiering for unpredictable patterns.

What is S3 Lifecycle?

S3 Lifecycle automates storage actions throughout an object's lifetime. Transition to cheaper storage classes or delete objects after specified periods.

Lifecycle Policy Example (Log Files):

Created → 30 days → 90 days → 365 days
   │         │         │          │
   ↓         ↓         ↓          ↓
Standard  Standard-IA  Glacier   Delete
($0.023)  ($0.0125)   ($0.004)
   │         │          │
   └─────────┴──────────┴── Cost Savings

Lifecycle Action Types

1. Transition Actions

Automatically move objects to different storage classes.

Transition Paths (Waterfall Model):

Standard
    ↓
Standard-IA ←→ Intelligent-Tiering
    ↓                ↓
One Zone-IA    Glacier Instant Retrieval
    ↓                ↓
Glacier Flexible Retrieval
    ↓
Glacier Deep Archive

2. Expiration Actions

Automatically delete objects.

Expiration Type	Description
Current version expiration	Delete object (non-versioned) or add delete marker (versioned)
Noncurrent version deletion	Delete previous versions of versioned objects
Delete marker cleanup	Remove expired delete markers
Incomplete multipart cleanup	Clean up incomplete multipart uploads

Storage Class Transition Rules

Transition Constraints

1. Minimum Storage Duration

Each storage class has a minimum storage period. Transitioning earlier still charges for the minimum period.

Storage Class	Minimum Duration
Standard	None
Standard-IA	30 days
One Zone-IA	30 days
Glacier Instant Retrieval	90 days
Glacier Flexible Retrieval	90 days
Glacier Deep Archive	180 days

Incorrect Configuration:

Standard → (5 days) → Glacier Instant Retrieval → (10 days) → Deep Archive
                              ↑
                         Minimum 90 days required!

Correct Configuration:
Standard → (5 days) → Glacier Instant Retrieval → (95 days) → Deep Archive
                              │                        │
                              └── 90 days minimum ─────┘

2. Minimum Object Size

Objects smaller than 128KB are not transitioned to Standard-IA, One Zone-IA, or Glacier classes by default. Transition costs may exceed storage savings.

3. Transition Direction

Once transitioned, objects cannot be moved back to more expensive classes via Lifecycle. Copy and delete to manually move.

Lifecycle Policy Design

Policy Components

{
  "Rules": [
    {
      "ID": "Log Archiving Rule",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "logs/"
      },
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    }
  ]
}

Filter Options

Filter	Description	Example
Prefix	Filter by prefix	logs/, backup/
Tag	Filter by tag	Environment: Production
Object Size	Filter by size	Transition only 128KB+
And	Combine conditions	Prefix + Tag

Use Case Examples

Case 1: Log File Management

Requirements:
- Recent logs need fast access
- Rarely accessed after 30 days
- Archive for compliance after 1 year
- Delete after 7 years

Policy Design:
┌─────────────────────────────────────────────────────────┐
│ Period       │ Storage Class              │ Reason      │
├─────────────┼─────────────────────────────┼─────────────┤
│ 0-30 days   │ Standard                   │ Active use  │
│ 30-90 days  │ Standard-IA                │ Rare access │
│ 90-365 days │ Glacier Instant Retrieval  │ Occasional  │
│ 365-2555 d  │ Glacier Deep Archive       │ Compliance  │
│ After 2555d │ Delete                     │ Retention   │
└─────────────────────────────────────────────────────────┘

Case 2: Versioned Bucket Management

Requirements:
- Current version: Keep long-term
- Previous versions: Delete after 30 days
- Delete markers: Auto cleanup

Policy:
{
  "NoncurrentVersionTransitions": [
    {"NoncurrentDays": 30, "StorageClass": "GLACIER"}
  ],
  "NoncurrentVersionExpiration": {
    "NoncurrentDays": 90
  },
  "ExpiredObjectDeleteMarker": true
}

Lifecycle vs Intelligent-Tiering

When to Use Which?

Condition	Choice
Access pattern is predictable	Lifecycle policies
Access pattern is unpredictable	Intelligent-Tiering
Compliance requires specific retention	Lifecycle policies
Auto-deletion required	Lifecycle policies
Minimize monitoring/transition costs	Lifecycle policies

Cost Comparison

Intelligent-Tiering:
├── Monitoring cost: $0.0025/1,000 objects/month
├── Auto tiering: Standard → IA (30 days), Archive (90 days)
└── Transition cost: None

Lifecycle Policies:
├── Monitoring cost: None
├── Manual rule configuration required
└── Transition cost: Per-request charges

Transition Cost Considerations

Transition Request Costs

Each object transition incurs request costs:

Transition Target	Cost per 1,000 requests
Standard-IA	$0.01
One Zone-IA	$0.01
Glacier Instant	$0.02
Glacier Flexible	$0.03
Glacier Deep Archive	$0.05

Small Object Warning

Transition Break-even for <128KB Objects:

Object size: 10KB
Standard monthly: $0.023 × 0.01 = $0.00023
Standard-IA monthly: $0.0125 × 0.128 = $0.0016 (128KB minimum charge)
Transition cost: $0.01/1000 = $0.00001

→ Standard-IA is MORE expensive! (128KB minimum billing)

Best Practices

1. Staged Transitions

Recommended: Standard → Standard-IA → Glacier
Not ideal: Standard → Glacier (direct)

Reasons:
- Lower retrieval costs for unexpected access
- Step-by-step validation

2. Tag-Based Classification

Apply tags at upload:
- DataType: logs, media, backup
- Retention: 30d, 90d, 1y, 7y

Create rules per tag:
- Retention: 30d → Delete after 30 days
- Retention: 7y → Deep Archive after 365 days

3. Incomplete Multipart Cleanup

Always include this rule:

"AbortIncompleteMultipartUpload": {
  "DaysAfterInitiation": 7
}

Reason: Incomplete multiparts are invisible but incur costs

SAA-C03 Exam Focus Points

1. ✅ Transition actions: Auto-move to cheaper classes
2. ✅ Expiration actions: Auto-delete objects, cleanup delete markers
3. ✅ Minimum storage: IA 30 days, Glacier Instant 90 days, Deep Archive 180 days
4. ✅ 128KB limit: Small objects excluded from IA/Glacier (default)
5. ✅ Lifecycle vs Intelligent-Tiering: Predictable vs unpredictable patterns
6. ✅ Cost priority: Multiple rules → most cost-effective action wins

Frequently Asked Questions

Q: When do Lifecycle policies run?

Evaluated daily at UTC 00:00. Exact execution time not guaranteed; may take up to 48 hours after rule creation for first run. Large object counts may take time to complete.

Q: Can I transition objects back to Standard?

Not via Lifecycle. Cannot directly transition from Glacier to Standard. Must restore (Restore) then copy as new object and delete original.

Q: How does Lifecycle work with versioned buckets?

Manages current and noncurrent versions separately. Transitions apply to current versions; NoncurrentVersionTransitions apply to previous versions. Use ExpiredObjectDeleteMarker to clean up delete markers.

Q: What happens if I delete before minimum storage duration?

Charged for the minimum period. Delete from Glacier Instant Retrieval after 10 days = still pay 90 days of storage.

Q: Can I use Lifecycle with Intelligent-Tiering together?

Yes. Create Lifecycle rules to transition from Standard to Intelligent-Tiering. IT then auto-tiers based on access patterns.

Related Posts

• S3 Storage Classes Complete Guide
• S3 Intelligent-Tiering
• S3 Encryption Options

References

• Setting Lifecycle Configuration
• Managing Object Lifecycle
• Transitioning Objects with Lifecycle