devops

S3 Basics

Buckets, objects, storage classes, and essential S3 CLI commands

What is S3?

Amazon Simple Storage Service (S3) is object storage — think of it as an infinitely scalable hard drive in the cloud. You store objects (files) inside buckets (containers).

Objects can be anything: images, videos, logs, backups, static websites, zip archives
Maximum object size: 5 TB
Bucket names are globally unique across all of AWS

Core Concepts

Term	Meaning
Bucket	Top-level container (like a folder root). One per name globally.
Object	A file + its metadata stored in a bucket
Key	The object’s “path” within a bucket, e.g. `images/logo.png`
Prefix	A key segment used like a folder, e.g. `images/`
Region	Where the bucket physically lives. Choose close to your users.

Storage Classes

Class	Use case	Retrieval	Cost
S3 Standard	Frequently accessed data	Instant	$$
S3 Standard-IA	Infrequently accessed, but needs fast retrieval	Instant	$
S3 One Zone-IA	Infrequently accessed, single AZ (cheaper, less durable)	Instant	$
S3 Glacier Instant	Archive with instant retrieval	Instant	¢
S3 Glacier Flexible	Archive, retrieved in minutes to hours	Minutes/Hours	¢
S3 Glacier Deep Archive	Long-term archive (7-10 years), rare access	Hours	¢¢

Use S3 Intelligent-Tiering if access patterns are unpredictable — it moves objects between tiers automatically.

Creating a Bucket (CLI)

# Create a bucket (replace region and name)
aws s3api create-bucket \
  --bucket my-devops-notes-bucket \
  --region ap-south-1 \
  --create-bucket-configuration LocationConstraint=ap-south-1

# List all your buckets
aws s3 ls

Common S3 CLI Commands

# Upload a file
aws s3 cp myfile.txt s3://my-bucket/myfile.txt

# Upload entire folder
aws s3 cp ./dist/ s3://my-bucket/dist/ --recursive

# Download a file
aws s3 cp s3://my-bucket/myfile.txt ./myfile.txt

# List objects in a bucket
aws s3 ls s3://my-bucket/

# List with human-readable sizes
aws s3 ls s3://my-bucket/ --human-readable --summarize

# Delete a file
aws s3 rm s3://my-bucket/myfile.txt

# Sync a folder (only upload changed/new files)
aws s3 sync ./dist/ s3://my-bucket/ --delete

Bucket Policy Example

Make all objects in a bucket publicly readable (for a static website):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

# Apply the policy
aws s3api put-bucket-policy \
  --bucket my-bucket \
  --policy file://bucket-policy.json

Static Website Hosting

S3 can serve static HTML/CSS/JS sites (like this blog!) with no server needed:

# Enable static website hosting
aws s3 website s3://my-bucket/ \
  --index-document index.html \
  --error-document 404.html

# Upload site files
aws s3 sync ./dist/ s3://my-bucket/ --delete

The site will be available at: http://my-bucket.s3-website.<region>.amazonaws.com

For a custom domain + HTTPS, put CloudFront in front of the S3 bucket.

Versioning

Keep every version of an object — great for backups:

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket my-bucket \
  --versioning-configuration Status=Enabled

# List versions of a specific object
aws s3api list-object-versions \
  --bucket my-bucket \
  --prefix myfile.txt

Key Security Tips

Block all public access by default — turn it on unless you explicitly need public objects
Enable versioning on important buckets to protect against accidental deletes
Enable server-side encryption — SSE-S3 is free and on by default for new buckets
Use IAM roles, not access keys, when accessing S3 from EC2 or Lambda
Enable S3 Access Logs to audit who accessed what