WES custom hosting

Removing Dead Files During an S3 Publish

Why this is needed

WCH uploads one zip to your bucket and never deletes anything. Unpacking that zip with CodePipeline's Extract file before deploy writes new files over old ones, but pages and assets you removed from your site stay in the bucket. Over time the bucket fills with old Webflow pages and fingerprinted CSS and JavaScript that nothing links to.

This page replaces the simple unpack step with a CodePipeline and CodeBuild deployment that uploads safely and cleans up orphans after a grace period.

What WCH gives you to work with

  • One zip per publish, at the object key you set in the destination's File Name field, by default project.zip.
  • Every page as a folder: index.html at the root, about/index.html, and the 404 page at 404/index.html.
  • No manifest. WCH does not ship a list of files. The build generates one from the zip contents.
  • No versioning or cache headers. WCH sets neither on the object. You enable versioning on the bucket and set Cache-Control during the build.

This process needs a complete zip. If WCH support has enabled incremental deploys for your project, the zip contains only files changed since the last deploy to that destination. A partial zip makes the manifest comparison below treat every unchanged file as an orphan. Confirm incremental deploys are off before using this.

Recommended flow

WCH
  ↓
Versioned incoming S3 bucket
  ↓
CodePipeline
  ↓
CodeBuild
  ↓
Customer serving bucket

WCH produces and uploads the zip. AWS handles validation, deployment, and cleanup.

Cleanup method: manifest-based deletion

Each publish produces a manifest containing every live object key:

index.html
404/index.html
css/site.123abc.css
js/site.456def.js
images/logo.svg
about/index.html

During deployment:

  1. Generate a manifest from the new package.
  2. Upload the new site.
  3. Store the new manifest.
  4. Compare it with the previous manifest.
  5. Identify keys that existed previously but do not exist now.
  6. Wait through the grace period.
  7. Delete only those orphaned keys.

This is safer than running aws s3 sync --delete blindly, because the cleanup step has an explicit list of expected live files.

CodePipeline structure

Stage 1: Source

Configure an S3 source action using the bucket and key where WCH uploads the zip. The source bucket must have versioning enabled. Example source key:

incoming/boldo/project.zip

Trigger the pipeline when that object is created or updated.

Stage 2: Deploy

Use CodeBuild to:

  • Validate the source artifact.
  • Generate the new manifest.
  • Upload non-HTML files.
  • Upload HTML files.
  • Save the new manifest.
  • Queue or perform delayed orphan cleanup.

CodePipeline passes the source output as an artifact. CodeBuild extracts it into its source directory. If your WCH project has a Root directory set in Settings, the site files sit inside that subfolder. Confirm whether the files appear directly in $CODEBUILD_SRC_DIR or inside a top-level folder before finalizing the build commands.

CodeBuild IAM permissions

The CodeBuild service role needs:

Source bucket

s3:GetObject
s3:GetObjectVersion
s3:ListBucket

Serving bucket

s3:GetObject
s3:PutObject
s3:ListBucket
s3:DeleteObject

DeleteObject is only required for the cleanup step. Limit the resources to the specific incoming and serving bucket prefixes.

Example deployment buildspec

This example assumes CodePipeline has extracted the website package directly into $CODEBUILD_SRC_DIR.

env:
  variables:
    SERVE_BUCKET: "customer-site-bucket"
    DEPLOYMENT_PREFIX: ".wch"

phases:
  pre_build:
    commands:
      - set -euo pipefail
      - SITE_DIR="$CODEBUILD_SRC_DIR"

      # Validate required files before changing production.
      - test -f "$SITE_DIR/index.html" || { echo "Missing index.html"; exit 1; }
      - test -f "$SITE_DIR/404/index.html" || { echo "Missing 404/index.html"; exit 1; }

      # Generate a normalized manifest of the new site.
      - |
        cd "$SITE_DIR"
        find . -type f \
          ! -path "./.wch/*" \
          -print \
          | sed 's#^\./##' \
          | LC_ALL=C sort \
          > /tmp/new-manifest.txt

      # Download the previous manifest when one exists.
      - |
        aws s3 cp \
          "s3://$SERVE_BUCKET/$DEPLOYMENT_PREFIX/current-manifest.txt" \
          /tmp/previous-manifest.txt \
          --only-show-errors \
          || touch /tmp/previous-manifest.txt

  build:
    commands:
      # Step 1: Upload assets first. Do not delete anything.
      - |
        aws s3 sync "$SITE_DIR" "s3://$SERVE_BUCKET" \
          --exclude "*" \
          --include "*.css" \
          --include "*.js" \
          --include "*.mjs" \
          --include "*.json" \
          --include "*.xml" \
          --include "*.txt" \
          --include "*.svg" \
          --include "*.png" \
          --include "*.jpg" \
          --include "*.jpeg" \
          --include "*.gif" \
          --include "*.webp" \
          --include "*.avif" \
          --include "*.ico" \
          --include "*.woff" \
          --include "*.woff2" \
          --include "*.ttf" \
          --include "*.otf" \
          --cache-control "public,max-age=31536000,immutable" \
          --only-show-errors

      # Step 2: Upload any remaining non-HTML files.
      - |
        aws s3 sync "$SITE_DIR" "s3://$SERVE_BUCKET" \
          --exclude "*.html" \
          --exclude "$DEPLOYMENT_PREFIX/*" \
          --only-show-errors

      # Step 3: Flip HTML files after their dependencies exist.
      - |
        aws s3 sync "$SITE_DIR" "s3://$SERVE_BUCKET" \
          --exclude "*" \
          --include "*.html" \
          --content-type "text/html; charset=utf-8" \
          --cache-control "public,max-age=60,must-revalidate" \
          --only-show-errors

  post_build:
    commands:
      # Calculate files that existed previously but are absent now.
      - |
        comm -23 \
          /tmp/previous-manifest.txt \
          /tmp/new-manifest.txt \
          > /tmp/orphaned-files.txt

      # Save deployment records before deleting anything.
      - DEPLOYMENT_ID="$(date -u +%Y%m%dT%H%M%SZ)"
      - |
        aws s3 cp \
          /tmp/new-manifest.txt \
          "s3://$SERVE_BUCKET/$DEPLOYMENT_PREFIX/manifests/$DEPLOYMENT_ID.txt" \
          --content-type "text/plain" \
          --cache-control "no-store" \
          --only-show-errors

      - |
        aws s3 cp \
          /tmp/new-manifest.txt \
          "s3://$SERVE_BUCKET/$DEPLOYMENT_PREFIX/current-manifest.txt" \
          --content-type "text/plain" \
          --cache-control "no-store" \
          --only-show-errors

      - |
        aws s3 cp \
          /tmp/orphaned-files.txt \
          "s3://$SERVE_BUCKET/$DEPLOYMENT_PREFIX/orphans/$DEPLOYMENT_ID.txt" \
          --content-type "text/plain" \
          --cache-control "no-store" \
          --only-show-errors

      - echo "Publish complete. Orphan cleanup should run after the grace period."

Delayed cleanup job

Run cleanup separately from the main deployment. Good options include:

  • A second CodeBuild project invoked after a delay
  • A scheduled Lambda function
  • An EventBridge Scheduler job
  • A scheduled cleanup pipeline

The cleanup job reads the orphan file generated by the deployment and deletes those keys:

set -euo pipefail

SERVE_BUCKET="customer-site-bucket"
ORPHAN_FILE="/tmp/orphaned-files.txt"

while IFS= read -r key; do
  [ -z "$key" ] && continue

  case "$key" in
    .wch/*)
      echo "Skipping protected key: $key"
      continue
      ;;
  esac

  aws s3 rm \
    "s3://$SERVE_BUCKET/$key" \
    --only-show-errors
done < "$ORPHAN_FILE"

Only process orphan lists that are older than the configured grace period.

Simpler alternative: delayed sync --delete

A separate cleanup job can unpack the same deployment and run:

aws s3 sync site "s3://$SERVE_BUCKET" \
  --delete \
  --exclude ".wch/*" \
  --only-show-errors

This is simpler but less controlled. Use it only when:

  • The bucket or prefix contains only generated website files.
  • The cleanup runs after a grace period.
  • Operational files are explicitly excluded.
  • The deployment package is retained and known to be complete.

Do not run this command in the primary publish pass.

Rollback

Keep versioning enabled on the incoming source bucket, previous project zips, historical manifests, and serving-bucket versioning when practical. WCH also keeps the 10 most recent archives on the project's Archives page, and any of them can be promoted again.

To roll back:

  1. Select the previous source zip version, or promote the earlier archive from WCH.
  2. Run the deployment again.
  3. Upload its assets.
  4. Flip its HTML files back into place.
  5. Generate a new cleanup comparison.

Do not restore deleted assets by reverting HTML alone. Redeploy the previous package so all of its assets are present before its HTML goes live.

Cache-Control guidance

Fingerprinted files

Use Cache-Control: public,max-age=31536000,immutable. This is appropriate only when changing the file contents also changes the filename, which is true for Webflow's CSS and JavaScript bundles.

HTML

Use Cache-Control: public,max-age=60,must-revalidate. A short HTML cache lifetime helps browsers discover the new site promptly and limits the required cleanup grace period.

Non-fingerprinted assets

Do not give a one-year immutable cache policy to files such as robots.txt, sitemap.xml, llms.txt, favicon.ico, or manifest.json. These keep the same key while their contents change. Give them a shorter cache policy or handle them separately.

Reliability checklist

  • Incoming S3 bucket has versioning enabled.
  • CodePipeline is triggered by the intended zip key, matching the WCH File Name field.
  • Build fails when index.html is absent.
  • No deletion occurs during the primary publish.
  • A new manifest is generated for every publish.
  • Previous and current manifests are compared.
  • Operational prefixes are protected.
  • Cleanup runs only after a grace period.
  • Cleanup has access to the exact orphan list.
  • Previous zip versions are retained for rollback.
  • Fingerprinted and non-fingerprinted files receive appropriate cache headers.
  • Incremental deploys are not enabled for the project.