Takazudo Modular Docs

Type to search...

to open search from anywhere

Image Processor Sub-Package

Image Processor Sub-Package

A standalone image processing system for Takazudo Modular that handles batch conversion, optimization, and metadata generation for product images.

Architecture Overview

The Image Processor is a self-contained sub-package that implements a three-tier architecture, generating all image data at processing time for build-time integration.

Isolation Benefits

  • Clean separation of concerns: Image processing logic isolated from main Next.js site
  • Dedicated dependencies: Sharp, blurhash, and related tools contained within sub-package
  • Independent testing: Isolated test environment with dedicated fixtures
  • Simplified deployment: Main project only depends on processed outputs

Sub-Package Structure

sub-packages/image-processor/
├── package.json                 # Isolated dependencies
├── src/
│   ├── index.mjs               # Main API exports
│   ├── process-images.mjs      # Core image processing pipeline
│   ├── process-images-mercari.mjs          # Mercari PNG generation
│   └── process-images-mercari-refactored.mjs
├── __tests__/
│   ├── fixtures/               # Test image files
│   └── process-images-rotation.test.js     # Rotation handling tests
├── bin/
│   └── process-images-cli.mjs  # CLI interface
└── README.md                   # Sub-package documentation

Core Processing Pipeline

Input Sources

The processor reads from the main project’s image directory:

/Users/takazudo/Dropbox/takazudoModular/_web-img-storage  # Symlinked as /imgs/

Output Structure

Generates optimized images in the main project’s static directory:

/static/images/p/[image-slug]/
├── metadata.json    # Dimensions, blurhash, variant paths
├── 600w.webp       # Responsive variants
├── 900w.webp
├── 1200w.webp
├── 1600w.webp
├── 2000w.webp
├── ogp.jpg         # 1200x630 for social sharing (only for __og / __ogonly files)
└── mercari.png     # Fixed 1600px width for Mercari marketplace

Supported Format Processing

# Format conversion pipeline
JPEG (.jpg, .jpeg) → WebP variants + OGP + Mercari PNG
PNG (.png) → WebP variants + OGP + Mercari PNG
HEIC (.heic) → JPEG → WebP variants + OGP + Mercari PNG
GIF (.gif) → Original preserved (no resizing, no Mercari PNG for animated)

API Reference

Main Processing Functions

processAllImages(inputDir, outputDir, config)

Batch processes all images in the input directory.

Parameters:

  • inputDir (string): Source directory path
  • outputDir (string): Target directory path
  • config (object): Processing configuration

processImage(imagePath, outputDir, config)

Processes a single image file.

Parameters:

  • imagePath (string): Path to source image
  • outputDir (string): Target directory path
  • config (object): Processing configuration

generateBlurhash(imagePath)

Generates blurhash placeholder for an image.

Returns: String blurhash value

generateMercariImage(imagePath, outputPath)

Creates Mercari-specific PNG image.

Parameters:

  • imagePath (string): Source image path
  • outputPath (string): Target PNG path

Configuration Options

const config = {
  widths: [600, 900, 1200, 1600, 2000],  // Responsive breakpoints
  formats: ['webp'],                           // Output formats
  quality: 85,                                 // WebP quality
  generateOGP: true,                          // Generate OGP images
  generateMercariPNG: true,                   // Generate Mercari PNG
  autoRepair: true,                           // Auto-repair corrupted images
};

OGP Image Generation

Overview

OGP (Open Graph Protocol) images are 1200x630 JPEGs used when pages are shared on social media. The pipeline uses __ (double underscore) as a delimiter for conversion rules:

Source fileSlugGenerates
foobar.heicfoobarWebP + mercari + metadata (NO ogp.jpg)
foobar__og.heicfoobar__ogWebP + mercari + metadata + ogp.jpg
foobar__ogonly.heicfoobar__ogonlyogp.jpg ONLY

No slug stripping — the slug IS the filename without extension. The --force-ogp CLI flag can force OGP generation on regular files.

Generation Methods

The processor selects between two methods based on the source image aspect ratio (threshold: OGP_LANDSCAPE_THRESHOLD = 1.5):

Composite method (square/portrait, ratio < 1.5)

Used for most product images since they are typically 1

square photos.

// generate-ogp-image.mjs — creates a blurred background composite
const config = {
  foregroundSize: 600,   // Centered foreground card size (px)
  blurSigma: 45,         // Background blur intensity
  quality: 85,           // JPEG quality
  cornerRadius: 0,       // Foreground card corner radius
};

Pipeline:

  1. Create 1200x630 canvas from blurred + darkened source image
  2. Apply gradient overlay for depth
  3. Center the source image at 600px with drop shadow
  4. Output as JPEG

Landscape crop method (ratio >= 1.5)

Used for wide source images that already fit the OGP aspect ratio.

// Resize and crop to 1200x630
generateOGPFromLandscape(inputPath, outputPath, { quality: 85 });

Key Functions

FunctionFilePurpose
generateSmartOGP()process-images.mjsShared OGP dispatch — detects aspect ratio, selects method
generateOGPImage()generate-ogp-image.mjsBlurred background composite for square/portrait
generateOGPFromLandscape()process-images.mjsSimple crop for landscape sources

Caching

Each __og / __ogonly file produces a .ogp-hash sidecar file containing the source hash. On subsequent runs, if the hash matches, the file is skipped. Delete .ogp-hash files to force regeneration.

Frontmatter integration

  • Use imgOgp frontmatter field when a page needs a different OGP image from its display image (e.g., imgThumb: bird with imgOgp: bird-ogp__ogonly)

Mercari PNG Specialization

Purpose

Generates marketplace-ready PNG images with consistent dimensions for Mercari product listings.

Specifications

// Mercari PNG requirements
const MERCARI_WIDTH = 1600;        // Fixed width requirement
const MERCARI_FILENAME = 'mercari.png';

// PNG optimization settings
const PNG_OPTIONS = {
  quality: 95,
  compressionLevel: 9,
  palette: false,
};

Features

  • Fixed Width: Exactly 1600px width for all images
  • Aspect Ratio Preservation: Height calculated to maintain proportions
  • Upscaling Support: Smaller images upscaled to meet width requirement
  • Animation Handling: Animated GIFs skipped to preserve animation
  • Error Recovery: Graceful handling of corrupted source images

Processing Logic

// Aspect ratio calculation
const aspectRatio = metadata.height / metadata.width;
const targetWidth = MERCARI_WIDTH;
const targetHeight = Math.round(targetWidth * aspectRatio);

// Resize with upscaling enabled
const buffer = await sharp(inputPath)
  .resize(targetWidth, targetHeight, {
    fit: 'inside',
    withoutEnlargement: false,  // Allow upscaling
  })
  .png(PNG_OPTIONS)
  .toBuffer();

Metadata Schema

Each processed image generates a metadata entry. Individual metadata.json files are aggregated into metadata-db.json by pnpm build:metadata:

{
  "slug": "image-slug",
  "blurhash": "LKO2?U%2Tw=w]~RBVZRi};RPxuwH",
  "width": 3024,
  "height": 3024,
  "aspectRatio": 100,
  "hasVariants": true,
  "hash": "content-hash",
  "processedAt": "2025-08-29T06:10:37.520Z",
  "originalFormat": "jpeg",
  "hasOgp": false
}

Key fields:

  • hasVariants: true for images with WebP variants, false for GIFs (original only)
  • originalFormat: Source format ("jpeg", "png", "gif", "heic", etc.)
  • hasOgp: true when ogp.jpg exists (added by build-metadata-db.mjs)
  • aspectRatio: Pre-calculated (height / width) * 100 for CSS aspect-ratio tricks

Usage from Main Project

NPM Script Integration

The main project automatically uses this sub-package via npm scripts:

# Primary processing command
pnpm convimgs

# Clean and reprocess all images
pnpm convimgs:cleanup

Script Implementation

// In main project's package.json scripts
"convimgs": "node sub-packages/image-processor/bin/process-images-cli.mjs"

Build Integration Flow

graph TB
    A[Source Images<br/>/imgs/] --> B[Image Processor<br/>Sub-package]
    B --> C[WebP Variants<br/>600w-2000w]
    B --> D[OGP Image<br/>1200x630]
    B --> E[Mercari PNG<br/>1600px width]
    B --> F[metadata.json<br/>files]

    F --> G[Next.js Build<br/>metadata-db.json]
    G --> H[Server Components]

    F --> I[Remark Plugin<br/>MDX processing]
    I --> J[MDX Components<br/>with metadata]

    H --> K[Static HTML<br/>Output]
    J --> K

    C --> M[/static/images/p/]
    D --> M
    E --> M
    F --> M

Special Processing Cases

HEIC Image Handling

HEIC files undergo automatic format conversion:

// Detection and conversion
if (fileExtension === '.heic') {
  await sharp(inputPath).jpeg().toFile(tempJpegPath);
  // Continue processing tempJpeg for all variants
}

GIF Animation Preservation

Animated GIFs receive special handling to preserve animation:

// Animated GIF detection
const metadata = await sharp(inputPath).metadata();
if (metadata.format === 'gif' && metadata.pages > 1) {
  // Skip resizing and Mercari PNG generation
  // Copy original to output directory
}

Image Upscaling

Images smaller than target dimensions are upscaled:

// Example: 800x600 source image
// Result: 1600x1200 Mercari PNG (maintains aspect ratio)
const wasUpscaled = originalWidth < MERCARI_WIDTH;

Testing Framework

Test Environment

The sub-package includes isolated testing:

# Run rotation handling tests
pnpm test

# Test specific functionality
pnpm test -- --grep "rotation"

Test Coverage

  • ✅ EXIF rotation handling for HEIC images
  • ✅ PNG generation at exactly 1600px width
  • ✅ Aspect ratio preservation during resize
  • ✅ Upscaling validation for smaller images
  • ✅ Animated GIF skip logic
  • ✅ Corrupted image error handling
  • ✅ Directory creation and permissions
  • ✅ Compression settings verification

Test Fixtures

__tests__/fixtures/
├── rotated-heic-image.heic     # EXIF rotation test
├── small-image.jpg             # Upscaling test
├── animated.gif                # Animation preservation test
└── corrupted.jpg               # Error handling test

CLI Interface

Main Project Usage

# Process all images
pnpm convimgs

# Process single image file
pnpm convimgs filename.jpg

# Process with cleanup (remove old files first)
pnpm convimgs:cleanup

Direct Sub-Package Usage

# Navigate to sub-package
cd sub-packages/image-processor

# Install dependencies
pnpm install

# Process all images
node bin/process-images-cli.mjs

# Process single image file
node bin/process-images-cli.mjs filename.jpg

# Process with cleanup
node bin/process-images-cli.mjs --cleanup

CLI Options

# Available options
--cleanup       Remove old processed files before generating new ones
filename        Process specific image file (e.g., 'image.jpg', 'photo.heic')

Performance Optimizations

Parallel Processing

The processor implements concurrent image processing:

// Process multiple images simultaneously
const results = await Promise.allSettled(
  imageFiles.map(file => processImage(file, outputDir, config))
);

Incremental Processing

Skip unchanged images based on content hash:

// Hash-based change detection
const currentHash = await generateFileHash(imagePath);
if (existingMetadata.hash === currentHash) {
  console.log(`⏭️  Skipping unchanged: ${slug}`);
  return;
}

Memory Management

Efficient memory usage for large image batches:

// Sharp instance cleanup
await sharp.cache(false);  // Disable cache for batch processing
await sharp.concurrency(1); // Limit concurrent operations

Troubleshooting

Common Issues and Solutions

Images Render as Dots (0px dimensions)

Cause: Metadata not generated or build cache stale Solution:

pnpm convimgs
pnpm clean && pnpm build

Missing Mercari PNG Files

Cause: Feature disabled or animated GIF source Solution:

# Check sub-package config
cat sub-packages/image-processor/src/process-images.mjs | grep generateMercariPNG

# Verify source isn't animated
file /imgs/your-image.gif

HEIC Processing Errors

Cause: Sharp HEIC support not available Solution:

# Reinstall sharp with HEIC support
cd sub-packages/image-processor
npm uninstall sharp
pnpm install sharp

Debugging Commands

# Verify processed outputs
ls -la static/images/p/*/mercari.png

# Check specific metadata
cat static/images/p/[slug]/metadata.json | jq '.mercariPNG'

# Test sub-package directly
cd sub-packages/image-processor
pnpm test

# Verbose processing
node bin/process-images-cli.mjs --verbose

# Single image processing test
node bin/process-images-cli.mjs test-image.jpg

Integration with Main Project

Build-Time Integration

The main Next.js project consumes the processed images:

  1. Metadata DB: pnpm build:metadata aggregates metadata.json files into metadata-db.json
  2. Remark Plugin: Injects metadata into MDX components during build
  3. Component Rendering: Uses metadata for responsive image rendering

Component Usage Examples

// Direct metadata access in MDX
<ExImg
  src="/images/p/product-photo"
  alt="Product"
/>

// Server component with metadata lookup
const ProductImage = ({ slug }) => (
  <ResponsiveImage
    slug={slug}
    alt="Product"
  />
);

Future Enhancements

Planned Improvements

  • AVIF Format Support: Better compression than WebP
  • Smart Cropping: AI-based focal point detection for Mercari images
  • CDN Integration: Optional variant upload to external CDN
  • Build Caching: Persist metadata between builds
  • Batch Export Tools: Generate CSV with image URLs for bulk Mercari upload
  • Progressive Processing: Support for very large image collections

API Stability

The sub-package API is designed for stability:

  • Semantic versioning for breaking changes
  • Backward-compatible configuration options
  • Graceful degradation for missing features
  • Clear error messages for troubleshooting