Cloud Storage

The Three Storage Paradigms

Every cloud offers three fundamental storage models, and choosing correctly is the single biggest lever on both cost and performance. Object storage holds unstructured blobs addressed by a key over HTTP; block storage presents raw volumes you attach to a VM and format with a filesystem; file storage exposes a shared filesystem over NFS or SMB that many clients mount at once.

Service Names Across Clouds

The concepts are identical; only the branding differs. Memorising this mapping lets you translate any architecture between providers.

Object Storage in Practice

An object lives in a bucket (a globally-named container) and is identified by a key. Objects are immutable — a write replaces the whole object. Buckets are private by default; public access requires explicit opt-in, which you should almost never grant.

Creating buckets and moving data

# AWS S3
aws s3 mb s3://acme-app-assets --region us-east-1
aws s3 cp ./photo.jpg s3://acme-app-assets/media/photo.jpg
aws s3 sync ./build/ s3://acme-app-assets/site/ --delete

# Azure Blob
az storage container create --name assets --account-name acmestorage
az storage blob upload --account-name acmestorage \
  --container-name assets --name media/photo.jpg --file ./photo.jpg

# Google Cloud Storage
gcloud storage buckets create gs://acme-app-assets --location=US
gcloud storage cp ./photo.jpg gs://acme-app-assets/media/photo.jpg
gcloud storage rsync ./build gs://acme-app-assets/site --delete-unmatched-destination-objects

Presigned URLs

A presigned URL grants time-limited access to a private object without exposing credentials — the standard pattern for browser uploads and download links.

# Valid for 1 hour (3600s)
aws s3 presign s3://acme-app-assets/media/photo.jpg --expires-in 3600

gcloud storage sign-url gs://acme-app-assets/media/photo.jpg --duration=1h

Storage Classes & Tiering

Storage classes trade cheaper storage for higher retrieval cost and latency. Hot data belongs in Standard; data touched rarely belongs in Infrequent Access; data kept only for compliance belongs in Archive, where retrieval can take minutes to hours.

Lifecycle policies

Instead of moving objects by hand, declare a lifecycle rule that transitions and expires objects by age.

# S3 lifecycle configuration (lifecycle.json)
{
  "Rules": [{
    "ID": "logs-tiering",
    "Filter": { "Prefix": "logs/" },
    "Status": "Enabled",
    "Transitions": [
      { "Days": 30,  "StorageClass": "STANDARD_IA" },
      { "Days": 90,  "StorageClass": "GLACIER" }
    ],
    "Expiration": { "Days": 365 }
  }]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket acme-app-assets --lifecycle-configuration file://lifecycle.json

# GCS equivalent (lifecycle.yaml, applied via gcloud)
rule:
  - action:
      type: SetStorageClass
      storageClass: NEARLINE
    condition:
      age: 30
  - action:
      type: Delete
    condition:
      age: 365

gcloud storage buckets update gs://acme-app-assets --lifecycle-file=lifecycle.yaml

Durability, Availability & Versioning

Object stores advertise eleven nines (99.999999999%) of durability by replicating each object across multiple devices and availability zones. That protects against hardware failure — not against you deleting the wrong file. For that you need versioning and, for compliance, Object Lock (WORM).

# Turn on versioning so overwrites and deletes are recoverable
aws s3api put-bucket-versioning --bucket acme-app-assets \
  --versioning-configuration Status=Enabled

# Object Lock in compliance mode — cannot be deleted before the retain date
aws s3api put-object-retention --bucket acme-app-assets --key vault/tax-2026.pdf \
  --retention '{"Mode":"COMPLIANCE","RetainUntilDate":"2033-01-01T00:00:00Z"}'

Encryption

Data at rest is encrypted by default on all three clouds. The real decision is who owns the key: the provider-managed key (SSE-S3), a key you manage in KMS (SSE-KMS), or a key you supply per request (SSE-C). KMS keys give you audit logs and revocation — the right default for regulated data.

# Enforce KMS encryption on upload
aws s3 cp report.pdf s3://acme-app-assets/vault/report.pdf \
  --sse aws:kms --sse-kms-key-id alias/acme-data-key

# Deny any unencrypted or non-TLS request via bucket policy (excerpt)
{ "Effect": "Deny", "Principal": "*", "Action": "s3:*",
  "Resource": "arn:aws:s3:::acme-app-assets/*",
  "Condition": { "Bool": { "aws:SecureTransport": "false" } } }

Practice Exercises

1. Create a private bucket on the cloud of your choice, upload a folder with the sync command, and generate a presigned URL that expires in 15 minutes. Verify the link stops working after it expires.
2. Write a lifecycle policy that keeps objects under logs/ in Standard for 30 days, moves them to an IA/Nearline tier for 60 days, then deletes them. Apply it and confirm with a describe/get command.
3. Enable versioning, overwrite a file, then restore the previous version. Explain what a delete marker is and how to remove one.
4. Given 5 TB of backups restored roughly twice a year, decide between Standard-IA, Glacier Flexible, and Deep Archive. Justify your choice using minimum-duration and retrieval-latency requirements.
5. Configure a bucket policy that denies any request made without TLS and any upload that is not KMS-encrypted. Test it by attempting a plain upload.
6. Estimate the monthly bill for 10 TB in Standard storage plus 2 TB of internet egress, and describe two changes that would cut the egress line item.