Last updated: Jul 25, 2025, 10:08 AM UTC

Convert to Markdown - Unified API Deployment Guide

Generated: 2025-07-24 10:42 UTC
Status: Production Ready
Region: us-east4 (Northern Virginia)
Domain: convert-to-markdown.knowcode.tech
API URL: https://convert-api-qpg64cvnga-uk.a.run.app

Overview

This guide documents the successful deployment of a unified API for document conversion on Google Cloud Platform, replacing individual Cloud Functions with a single endpoint router to solve domain mapping limitations.

What We Learned

What Worked

  1. Single Endpoint Router Architecture

    • Solved Cloud Run's one-service-per-domain limitation
    • Cleaner API design with /v1/convert/* paths
    • Easier maintenance and deployment

  2. Busboy Instead of Multer

    • Fixed Node.js 20.5+ compatibility issues
    • No fs-temp dependency problems
    • Works seamlessly with Cloud Functions Gen2

  3. Structured Error Handling

    • ApiError class with error codes
    • Google Cloud Logging compatible
    • Helpful error messages for developers

  4. Clean Migration Approach

    • No backward compatibility complexity
    • Fresh start with v2 API
    • Simplified documentation

What Didn't Work

  1. Multer with Node.js 20

    • fs-temp@1.x uses fs.WriteStream(null) which fails
    • Container startup failures in Cloud Run
    • Solution: Switched to Busboy

  2. Multiple Domain Mappings

    • Cloud Run limitation: one service per domain
    • Can't map individual functions to paths
    • Solution: Single router architecture

  3. Complex Migration Strategies

    • Maintaining v1 compatibility added complexity
    • Solution: Clean cut migration

Current Architecture

Unified API Endpoints

Base URL: https://convert-api-qpg64cvnga-uk.a.run.app

Method Path Description Authentication
GET /v1/health Health check None
GET /v1 List endpoints None
POST /v1/convert/excel-to-json Excel → JSON API Key
POST /v1/convert/excel-to-markdown Excel → Markdown API Key
POST /v1/convert/pdf-to-markdown PDF → Markdown API Key
POST /v1/convert/word-to-html Word → HTML API Key
POST /v1/convert/word-to-markdown Word → Markdown API Key
GET /v1/usage/summary Usage summary API Key
GET /v1/usage/history Usage history API Key
GET /v1/usage/current Current usage API Key

Authentication

Two methods supported:

# X-API-Key header
curl -H "X-API-Key: your-key-here"

# Bearer token
curl -H "Authorization: Bearer your-key-here"

Deployment Steps

1. Create Unified API Structure

# Create new project structure
mkdir unified-api
cd unified-api

# Initialize with package.json
npm init -y
npm install @google-cloud/functions-framework express cors busboy
npm install @google-cloud/firestore uuid dotenv
npm install xlsx pdf-parse mammoth exceljs

2. Key Implementation Files

index.js - Main router with all endpoints
middleware/auth.js - API key validation
middleware/fileUpload.js - Busboy file handling
middleware/usage.js - Usage tracking
lib/errors.js - Structured error handling
lib/converters/ - Converter functions

3. Deploy to Cloud Run

# Deploy the unified API
gcloud functions deploy convert-api \
  --gen2 \
  --runtime=nodejs20 \
  --region=us-east4 \
  --entry-point=api \
  --memory=1GB \
  --timeout=60s \
  --allow-unauthenticated \
  --trigger-http

4. Update Domain Mapping

# Map domain to new service
gcloud run domain-mappings create \
  --service=convert-api \
  --domain=convert-to-markdown.knowcode.tech \
  --region=us-east4

Testing

Local Testing

# Start local server
npm start

# Test health endpoint
curl http://localhost:8080/v1/health

# Test converter (no auth for local testing)
curl -X POST \
  -F "file=@test.pdf" \
  http://localhost:8080/v1/convert/pdf-to-markdown

Production Testing

# Test with API key
curl -X POST \
  -H "X-API-Key: your-production-key" \
  -F "file=@document.xlsx" \
  https://convert-api-qpg64cvnga-uk.a.run.app/v1/convert/excel-to-json

Key Learnings for Future Projects

1. Domain Mapping Strategy

  • Plan for single service architecture if custom domains needed
  • Consider API Gateway for complex routing
  • Document Cloud Run limitations early

2. Node.js Dependencies

  • Test with exact runtime version (Node.js 20)
  • Avoid packages with native dependencies
  • Prefer maintained packages (Busboy over Multer)

3. API Design

  • Start with versioned endpoints (/v1/)
  • Implement authentication from day one
  • Use structured error handling
  • Include health checks

4. Migration Approach

  • Clean migrations can be simpler than compatibility layers
  • Communicate breaking changes clearly
  • Provide migration windows

Configuration Updates

Environment Variables

# .env file
GOOGLE_APPLICATION_CREDENTIALS=path/to/service-account.json
NODE_ENV=production
PORT=8080

Firestore Collections

  • apiKeys - API key validation
  • usage - Usage tracking
  • users - User profiles
  • analytics - Detailed analytics

Monitoring and Logs

View Logs

# View function logs
gcloud functions logs read convert-api --region=us-east4

# Filter errors only
gcloud functions logs read convert-api \
  --region=us-east4 \
  --filter="severity>=ERROR"

Key Metrics

  • Request count by endpoint
  • Error rates by error code
  • Response times per converter
  • File sizes processed

Cost Optimization

Current Configuration

  • Memory: 1GB (optimal for large files)
  • Timeout: 60s (handles large PDFs)
  • Min Instances: 0 (scale to zero)
  • Max Instances: 100 (prevent runaway)

Estimated Costs

  • Idle: $0 (scales to zero)
  • Active: ~$0.40 per million requests
  • Bandwidth: Included in free tier

Security Considerations

API Key Management

  • Keys stored hashed in Firestore
  • Rate limiting per key
  • Usage tracking enabled
  • Key rotation supported

File Processing

  • No files stored permanently
  • Memory-only processing
  • Automatic cleanup
  • Size limits enforced (10MB)

Future Enhancements

  1. Add PowerPoint Support

    • Implement pptx-to-markdown converter
    • Add to unified API

  2. Batch Processing

    • Accept multiple files
    • ZIP file support

  3. Webhook Callbacks

    • Async processing for large files
    • Progress notifications

  4. Advanced Authentication

    • OAuth2 support
    • JWT tokens
    • Team accounts

Support and Troubleshooting

Common Issues

  1. 401 Unauthorized

    • Check API key is valid
    • Verify header format

  2. 413 File Too Large

    • Current limit: 10MB
    • Consider chunking

  3. 500 Internal Error

    • Check logs for details
    • Verify file format

Contact

This deployment successfully migrated from individual Cloud Functions to a unified API, solving domain mapping limitations while improving the overall architecture and developer experience.