Build, Test, and Serve Strategy - Unified Guide

Build, Test, and Serve Strategy - Unified Guide

Overview

This document provides a comprehensive understanding of all build, test, and serve strategies in the Astro version of Takazudo Modular. The build system uses a single build command with environment-based behavior.

Quick Reference

Common Workflows

# Development
pnpm dev                    # Start Astro dev server (port 34434, auto-creates image symlink)

# Dev with API (3 environments)
pnpm dev:full               # Astro + local Netlify Functions (port 9999, offline blobs)
pnpm dev:full:preview       # Astro proxying API to preview deploy
pnpm dev:full:prod          # Astro proxying API to production

# Image Setup for Local Dev
pnpm r2:download            # Download processed images from R2 (first time)
pnpm setup:local-images     # Create symlink (done automatically by pnpm dev)

# Local Testing (Full Flow)
pnpm setup:local-images     # One-time: Create image symlink
pnpm build                  # Build site (includes search index + sitemap + RSS)
pnpm serve                  # Serve dist/ on port 34434
pnpm test:e2e:critical      # Run E2E tests

# Pre-Push Checklist
pnpm check                  # TypeScript + ESLint + formatting
pnpm test                   # Unit + critical E2E tests

# CI Build
pnpm build                  # Build (generates search index for MiniSearch)
pnpm serve                  # Serve dist/ on port 34434
pnpm test:e2e:full-prod     # Full E2E suite (Playwright intercepts images in CI)

Port Mapping Reference

Preview Deployment URLs:

Example: expreview/tag-management deploys to https://expreview-tag-management--takazudomodular.netlify.app/.

Branch name length rule: Netlify limits project name + branch name to 61 chars. With takazudomodular (15 chars), branch names must be ≤ 46 chars total.

Note: Port 34434 is used for both dev and serve (production build) to avoid port conflicts.

Build Strategy

Single Build Command

The build system uses one command for all environments:

pnpm build

What it does:

1. Prepares public/ directory (excludes images)
2. Generates metadata-db.json
3. Generates search index for MiniSearch
4. Runs Astro build (includes sitemap via integration)
5. Generates RSS feed
6. Outputs to dist/

Build Process Flow

graph TD
    A[pnpm build] --> B[Prepare public/ directory]
    B --> C[Build metadata-db.json]
    C --> D[Generate search index]
    D --> E[Astro static build]
    E --> F[Generate RSS]
    F --> G[Output to dist/]

    B --> B1[rsync static/ to public/]
    B1 --> B2[Exclude images/p]

    C --> C1[Read static/images/p/*/metadata.json]
    C1 --> C2[Consolidate into metadata-db.json]
    C2 --> C3[116x faster than individual reads]

    E --> E1[Generate static HTML]
    E1 --> E2[All pages are SSG]
    E --> E3[Sitemap generated via integration]

Local Image Setup

For local testing with images:

# One-time setup (or after cleaning)
pnpm setup:local-images

# This creates: public/images/p → ../../static/images/p

How it works:

• Creates relative path symlink
• Works on any machine (not absolute path)
• Dev server creates this automatically
• For builds, run manually if needed

Why symlink?

• No copying ~2.5GB of images
• Fast startup (~0.25s vs several seconds)
• Saves disk space

Build Command Details

{
  "build": "pnpm run _prepare && pnpm run build:metadata && pnpm run build:search-index && astro build && pnpm run generate:rss",
  "build:metadata": "node scripts/build-metadata-db.mjs",
  "build:search-index": "node scripts/generate-search-index.mjs",
  "setup:local-images": "mkdir -p public/images && ln -sf ../../static/images/p public/images/p"
}

Build Output:

• Astro outputs to dist/ (static output mode)
• Sitemap is generated automatically via the Astro sitemap integration
• Search index is generated before the Astro build so it can be included in the output

Serve Strategies

Development Server

pnpm dev

Characteristics:

• Runs on port 34434 (zmod.localhost:34434)
• Hot module replacement (HMR) enabled
• Reads from source files directly
• Auto-creates image symlink
• NOT suitable for E2E testing (use build + serve)

Production Server (Local)

pnpm serve

Characteristics:

• Runs on port 34434 (same as dev)
• Serves pre-built dist/ directory
• Uses serve package (Node.js static file server)
• Serves static files from dist/
• Suitable for E2E testing
• Mimics production environment

Testing Strategies

Test Types Overview

Test Command Reference

# Quick Checks
pnpm test                   # Unit + critical E2E
pnpm check                  # TypeScript + ESLint + formatting

# Unit Tests
pnpm test:unit              # Run all unit tests
pnpm test:unit:watch        # Watch mode
pnpm test:unit:coverage     # With coverage report

# Link Validation (requires server running)
pnpm validate:links         # Check all internal links (220 pages)

# E2E Tests (require server running)
pnpm test:e2e:critical      # 13 critical pages
pnpm test:e2e:full-prod     # All pages from sitemap
pnpm test:e2e:ui            # With Playwright UI
pnpm test:e2e:debug         # Debug mode

# Comprehensive Testing
pnpm test:all               # Unit + full E2E
pnpm test:comprehensive     # Run ./scripts/pre-merge-check.sh

E2E Test Environments

Local E2E Testing

# Terminal 1: Build and serve
pnpm setup:local-images     # If not done yet
pnpm build
pnpm serve

# Terminal 2: Run tests
pnpm test:e2e:critical

Features:

• Uses full-check-production.spec.js
• Checks for 404s, JS errors, broken images, broken links
• Runs against http://zmod.localhost:34434

CI E2E Testing

# Automated in GitHub Actions
1. Build site
2. Start server on port 34434
3. Run full-check-production.spec.js (Playwright intercepts images via page.route())

Differences from Local:

• Uses full-check-production.spec.js (same test file as local)
• Playwright route interception returns 1x1 pixel images (no LFS download needed)
• Single-job execution (no sharding)

CI Workflow Separation (Build vs E2E)

Separated Workflows:

• pr-checks.yml: Quality checks + Build (uploads artifacts)
• pr-e2e.yml: E2E tests (triggered after PR Checks completes)

Why Separate?

1. Faster builds: Build artifacts are cached between runs
2. Fail fast: Build failures don’t waste time on E2E setup
3. Sequenced execution: E2E tests run only if build succeeds
4. Cache efficiency: Build cache is preserved

Workflow Flow:

graph LR
    A[PR Created] --> B[pr-checks.yml]
    B --> C{Build Success?}
    C -->|Yes| D[pr-e2e.yml triggers]
    C -->|No| E[Stop - No E2E]
    D --> F[Download artifacts]
    F --> G[Run E2E tests with Playwright image interception]

Local CI Simulation

For simulating the full CI workflow locally:

# Simulates CI: clean build + E2E with image interception
pnpm test:simulate-ci-e2e

# What it does:
# 1. pnpm run clean       # Clean .astro, dist, public
# 2. pnpm run build       # Full build
# 3. CI=true pnpm run test:e2e:full-prod  # E2E with Playwright image interception

CI Image Handling

In CI, Playwright’s page.route() intercepts all product image requests (**/images/p/**) at the browser level and returns tiny 1x1 pixel placeholder responses. This eliminates the need for placeholder image files on disk.

• CI: Playwright intercepts image requests, returns 1x1 pixel images (no network, no disk files)
• Local: No interception, tests hit real CDN images (catches real issues)

The interception helper is at tests/e2e/helpers/ci-image-interceptor.js.

Quality Checks

Automated Checks (Pre-Commit Hook)

When you commit, Lefthook automatically runs:

# Via lefthook pre-commit
- ESLint on modified .js/.ts/.tsx/.mjs files
- Prettier on modified code/config files (.js, .ts, .json, .css, .yml, .yaml, .astro)
- mdx-formatter on .md/.mdx files

Manual Quality Checks

# Quick check
pnpm check                  # TypeScript + ESLint + formatting

# With auto-fix
pnpm check:fix              # Fix ESLint + formatting issues

# Individual checks
pnpm typecheck              # TypeScript type checking
pnpm lint                   # ESLint
pnpm format                 # Prettier (code) + mdx-formatter (md/mdx)

Comprehensive Pre-Merge Check

Before merging major PRs:

./scripts/pre-merge-check.sh

What it runs:

1. TypeScript type checking
2. ESLint checks
3. Formatting checks (Prettier + mdx-formatter)
4. Unit tests
5. Astro build
6. 404 checker (all pages from sitemap)
7. Full E2E test suite

Environment-Specific Workflows

Local Development Workflow

# 1. Start development
pnpm dev                    # Auto-creates image symlink

# 2. Make changes
# ... edit files ...

# 3. Check quality
pnpm check

# 4. Test locally
pnpm test:unit

# 5. Commit
git add .
git commit -m "feat: ..."   # Pre-commit hook runs automatically

# 6. Push
git push

Local Testing with Production Build

# 1. Setup images (one-time)
pnpm setup:local-images

# 2. Build
pnpm build

# 3. Serve
pnpm serve

# 4. Test
pnpm test:e2e:critical

CI/CD Workflow (GitHub Actions)

graph TD
    A[Push to preview branch] --> B[Install dependencies]
    B --> C[Run quality checks]
    C --> D[Build metadata-db.json]
    D --> E[Generate search index]
    E --> F[Astro build]
    F --> G[Generate RSS]
    G --> H[Run unit tests]
    H --> I[Start HTTP server]
    I --> J[Run E2E tests with Playwright image interception]
    J --> K{All pass?}
    K -->|Yes| L[Deploy to Netlify]
    K -->|No| M[Fail build]

Production Deployment Workflow

# Automated on merge to main
1. All CI checks pass
2. Build Astro site (output to dist/)
3. Deploy to Netlify production
4. Images served via CDN proxy

Preview Deployment

Preview Branch URLs

Push to these branches to trigger a full CI build + deploy to Netlify preview:

Each branch deploys to its own isolated URL. The deploy alias is derived from the branch name (slashes converted to hyphens). For example, expreview/tag-management deploys to https://expreview-tag-management--takazudomodular.netlify.app/.

Branch name length rule: Netlify limits project name + branch name to 61 chars. With takazudomodular (15 chars), branch names must be ≤ 46 chars total. The expreview/ prefix uses 10 chars, leaving up to 36 chars for the topic name.

What Gets Deployed

Preview deployments include:

• Full Astro static site
• Storybook under /storybook/ (built via pnpm storybook:build in PR Checks CI)
• Netlify Functions (notify-signup, reservation, admin APIs)
• _redirects file for image CDN proxy and admin API routing
• Netlify Blobs using preorder-data-preview store (separate from production data)

Production domain proxies storybook from the preview deploy: https://takazudomodular.com/storybook/ → https://preview--takazudomodular.netlify.app/storybook/ (200 rewrite in netlify.toml).

Preview vs Production Differences

No-CI Branch Strategy

Purpose

The noci/* branch pattern allows developers to skip all CI processing for prototype branches and non-app-related changes, saving GitHub Actions minutes.

When to Use

Ideal for:

• Design prototypes (noci/top-page-v1)
• Documentation updates (noci/docs-update)
• Configuration files that don’t affect the app (noci/config-cleanup)
• Non-code maintenance (noci/cleanup-files)

Workflow Behavior

Cost Savings

• Traditional PR: ~20-25 minutes of runner time
• No-CI PR: 0 minutes of runner time (no workflows run)
• Savings: 100% reduction in CI costs for prototype and non-code changes

Sub-Package Commands

All sub-packages use namespace prefixes for clarity:

# Documentation (zudo-doc at doc2/)
cd doc2 && pnpm dev         # Start zudo-doc dev server (port 32342)
cd doc2 && pnpm build       # Build documentation
cd doc2 && pnpm preview     # Preview built documentation

# Storybook
pnpm storybook:dev          # Start Storybook (zstorybook.localhost:44321)
pnpm storybook:build        # Build Storybook

# Mercari Viewer
pnpm mercari:dev            # Start Mercari CSV viewer (zmercari.localhost:23234)
pnpm mercari:build          # Build Mercari viewer
pnpm mercari:preview        # Preview Mercari build

# Products Viewer
pnpm products:dev           # Start Products viewer (zproducts.localhost:9755)
pnpm products:build         # Build Products viewer
pnpm products:preview       # Preview Products build

# zmdpreview
pnpm zmdpreview:dev         # Start MD preview (zmdviewer.localhost:12188)
pnpm zmdpreview:build
pnpm zmdpreview:serve

# Image Mixer
pnpm image-mixer:dev        # Start Image mixer tool

# Preorder Admin (zpreorder)
pnpm zpreorder:dev          # Start zpreorder (mock mode, port 9876)
pnpm zpreorder:dev:full     # zpreorder + local Netlify Functions (port 9999)
pnpm zpreorder:dev:preview  # zpreorder pointing to preview API
pnpm zpreorder:dev:prod     # zpreorder pointing to production API

Common Issues and Solutions

Issue: metadata-db.json not found

Symptoms:

❌ No metadata-db.json file found.

Solution:

pnpm build:metadata
# or
pnpm build  # Includes metadata generation

Issue: E2E tests fail with 404 for images

Symptoms:

404 Resources: /images/p/product-slug/1200w.webp

Solution for Local Testing:

# Create image symlink
pnpm setup:local-images

# Then build and test
pnpm build
pnpm serve
pnpm test:e2e:critical

Solution for CI:

• CI uses Playwright route interception for images (no placeholder files needed)
• The interceptor activates automatically when CI=true

Issue: Server port conflict

Symptoms:

Error: listen EADDRINUSE: address already in use :::34434

Solution:

# Kill process on port
lsof -ti:34434 | xargs kill -9

# Or use the built-in command
pnpm kill-port

Issue: Stale build cache

Symptoms:

• Changes not reflecting in build
• Unexpected build errors

Solution:

pnpm clean  # Remove .astro, dist, public, sb-build
pnpm build  # Rebuild from scratch

Issue: Symlink not working

Symptoms:

• Images not loading in local dev/build
• Symlink shows as broken

Solution:

# Remove old symlink
rm public/images/p

# Recreate with setup command
pnpm setup:local-images

# Verify symlink
ls -la public/images/p
# Should show: p@ -> ../../static/images/p

Best Practices

Development

1. Use dev server for development: pnpm dev has HMR and is faster
2. Run quality checks before committing: pnpm check
3. Test locally before pushing: pnpm test
4. One-time image setup: Run pnpm setup:local-images once

Building

1. Single build command: Just use pnpm build for everything
2. Environment-based: Build adapts to environment automatically
3. Clean when stuck: pnpm clean && pnpm build
4. Local images: Use setup:local-images, not copying

Testing

1. Unit test during development: Fast feedback loop
2. E2E test before major changes: Catch integration issues
3. Use critical E2E for quick checks: 13 pages in ~10 seconds
4. Use full E2E before merging: Comprehensive coverage
5. CI uses Playwright image interception: Faster and more reliable than real images

CI/CD

1. Let CI run all checks: Don’t skip workflows
2. Fix failing tests immediately: Don’t let issues accumulate
3. Review E2E test results: Check for console errors and 404s
4. Monitor build times: Optimize if builds take too long
5. Trust the build: Search index is generated automatically

Summary

Key Takeaways

• Single build command: pnpm build for all environments
• Environment-based: Build adapts to environment automatically
• Image symlinks: Fast local development with setup:local-images
• Namespace prefixes: Clear sub-package commands (doc:*, storybook:*, etc.)
• All pages are static (SSG): No server-side rendering at runtime
• Images served via CDN: Not included in main build
• Playwright route interception enables fast CI: No image downloads needed (~2.5GB saved)
• metadata-db.json is critical: Required for builds

Quick Decision Matrix

When to use each command:

• Development: pnpm dev
• Local testing: pnpm setup:local-images (once) + pnpm build + pnpm serve
• Pre-push checks: pnpm check + pnpm test
• Pre-merge checks: ./scripts/pre-merge-check.sh
• Sub-packages: pnpm <namespace>:dev (e.g., storybook:dev, mercari:dev)

Command Name Changes

Removed (simplified):

• ❌ build:local - Use setup:local-images + build
• ❌ build:all - Merged into build
• ❌ build:full - Use check + build separately
• ❌ build:with-storybook - Use storybook:build separately
• ❌ build:with-docs - Use cd doc2 && pnpm build separately
• ❌ docbuild - Use cd doc2 && pnpm build
• ❌ storybook - Use storybook:dev
• ❌ build-storybook - Use storybook:build

Added (new):

• ✅ setup:local-images - One-time image symlink setup

Renamed (consistency):

• image-mixer → image-mixer:dev
• mdpreview:* → zmdpreview:*

Last Updated: 2026-03-20 Astro Version: 6 Framework: Astro with static output (output: 'static', outputs to dist/)