MinusNowDocumentation

Troubleshooting & Recovery

Diagnose crashes, fix service failures, recover from outages, and resolve common production issues in minutes.

Application Crash Recovery Critical

When the MinusNow application process stops or enters a crash loop, follow this escalation path:

1. Check PM2 status 2. Read error logs 3. Fix root cause 4. Restart service 5. Verify health

Step 1: Check if the process is running

# Check PM2 process status pm2 status # If PM2 is not installed or not found npx pm2 status # Check if Node.js is listening on port 5000 sudo lsof -i :5000 sudo ss -tlnp | grep 5000

Step 2: Read the error logs

# PM2 error logs (most recent 100 lines) pm2 logs minusnow --err --lines 100 # Or read the log file directly tail -100 ~/.pm2/logs/minusnow-error.log # Combined output + error logs pm2 logs minusnow --lines 200

Step 3: Common crash causes & fixes

Error MessageCauseFix
ECONNREFUSED 127.0.0.1:5432PostgreSQL is not runningsudo systemctl start postgresql
ENOMEM / JavaScript heap out of memoryServer ran out of RAMNODE_OPTIONS="--max-old-space-size=4096" pm2 restart minusnow
EADDRINUSE :::5000Port 5000 already in usesudo fuser -k 5000/tcp then restart
MODULE_NOT_FOUNDMissing dependency after updatenpm install then pm2 restart minusnow
relation "..." does not existDatabase migrations not appliednpm run db:push then restart
SIGKILL / OOM KillerLinux killed process for memoryIncrease server RAM or add swap: sudo fallocate -l 2G /swapfile
ERR_HTTP2_PROTOCOL_ERRORNginx proxy misconfigurationCheck Nginx proxy_pass points to http://127.0.0.1:5000

Step 4: Restart the application

# Standard restart pm2 restart minusnow # If PM2 process doesn't exist, start fresh cd /opt/minusnow pm2 start dist/index.cjs --name minusnow -- --port 5000 # Save PM2 config for auto-restart on reboot pm2 save pm2 startup

Step 5: Verify health

# Health endpoint curl -s http://localhost:5000/api/health | jq . # Verify all subdomains are responding curl -I https://app.minusnow.com curl -I https://www.minusnow.com curl -I https://docs.minusnow.com
Emergency restart: If PM2 is completely unresponsive, kill all Node processes and start fresh:
sudo pkill -9 node && cd /opt/minusnow && pm2 start dist/index.cjs --name minusnow

Pages Not Loading / Blank Pages Common

SPA Routes returning 404

If React SPA routes (like /incidents, /sprints, /sla-heatmap) return 404 in production, the route is missing from server/static.ts.

# Check which routes are registered in the SPA_ROUTES array grep -n "SPA_ROUTES" dist/index.cjs | head -5 # The fix: add missing routes to SPA_ROUTES in server/static.ts # Then rebuild and restart: npm run build pm2 restart minusnow

Marketing Site Pages (404 on /site/products, /site/pricing, etc.)

Static HTML pages must be listed in the ALLOWED_PAGES array in server/static.ts.

# Verify the HTML file exists ls -la dist/public/site/*.html # If file exists but 404s, add pagename to ALLOWED_PAGES in server/static.ts npm run build && pm2 restart minusnow

Blank white page (JavaScript error)

1. Open browser DevTools (F12)

Check the Console tab for red JavaScript errors. Note the exact error message and file name.

2. Check Network tab

Look for failed requests (red). CSS/JS assets returning 404 means the build output is missing.

3. Rebuild assets

Run npm run build to regenerate all client assets, then pm2 restart minusnow.

4. Clear cache

Hard refresh: Ctrl+Shift+R. Old cached JS can cause hydration errors after deploy.

Subdomain not resolving

# Verify DNS resolution dig app.minusnow.com +short dig docs.minusnow.com +short # Should return your server's IP address # If empty, DNS A record is missing. Add it in your DNS provider. # Verify Nginx is routing the subdomain sudo nginx -t sudo grep -r "server_name" /etc/nginx/sites-enabled/

Database Issues Critical

PostgreSQL not running

# Check PostgreSQL status sudo systemctl status postgresql # Start if stopped sudo systemctl start postgresql # Enable auto-start on boot sudo systemctl enable postgresql # Check PostgreSQL logs for errors sudo tail -50 /var/log/postgresql/postgresql-16-main.log

Connection refused in the app

CheckCommandExpected
PostgreSQL is runningsudo systemctl status postgresqlActive (running)
Listening on port 5432sudo ss -tlnp | grep 5432LISTEN 0.0.0.0:5432
DATABASE_URL is setecho $DATABASE_URLpostgresql://user:pass@localhost:5432/minusnow
Can connect manuallypsql "$DATABASE_URL" -c "SELECT 1" ?column? = 1

Migration / schema errors

# Push latest schema to database (non-destructive) npm run db:push # If schema is severely out of sync, generate migration npx drizzle-kit generate npx drizzle-kit migrate # Verify tables exist psql "$DATABASE_URL" -c "\dt"

Database backup & restore

# Create a full backup pg_dump "$DATABASE_URL" -F c -f /backups/minusnow-$(date +%Y%m%d).dump # Restore from backup pg_restore -d "$DATABASE_URL" --clean --if-exists /backups/minusnow-20260331.dump
Never run DROP DATABASE in production. Always use pg_dump before any schema changes. Keep at least 7 days of backups.

Nginx & SSL Issues Common

502 Bad Gateway

Nginx received the request but cannot reach the Node.js backend.

# Is the app running? pm2 status # Is Node.js actually listening? curl -s http://127.0.0.1:5000/api/health # Check Nginx error log sudo tail -30 /var/log/nginx/error.log # Verify Nginx config sudo nginx -t # If config is good, reload sudo systemctl reload nginx

SSL certificate expired

# Check certificate expiry sudo certbot certificates # Renew all certificates sudo certbot renew # Force renewal if near expiry sudo certbot renew --force-renewal # Reload Nginx after renewal sudo systemctl reload nginx

Nginx config for MinusNow subdomains

# Test full configuration sudo nginx -t # List all configured server_names sudo grep -r "server_name" /etc/nginx/sites-enabled/ | sort # Expected server_names for MinusNow: # minusnow.com www.minusnow.com # app.minusnow.com # docs.minusnow.com # downloads.minusnow.com # founder.minusnow.com # status.minusnow.com # api.minusnow.com # support.minusnow.com # *.minusnow.com (tenant wildcard)
Wildcard SSL: Use sudo certbot certonly --manual --preferred-challenges=dns -d "*.minusnow.com" -d "minusnow.com" to issue a single certificate that covers all subdomains.

Performance Issues Optimization

Diagnose high CPU / memory

# PM2 monitoring dashboard (live) pm2 monit # System-level resource usage htop # Node.js memory usage pm2 show minusnow | grep -E "memory|cpu|restart" # Check if app is restart-looping (high restart count = crash loop) pm2 status

Slow API responses

Check database latency

psql "$DATABASE_URL" -c "SELECT now()" -t — should return instantly. Slow = disk I/O issue.

Check active connections

psql "$DATABASE_URL" -c "SELECT count(*) FROM pg_stat_activity" — over 100 suggests connection pool exhaustion.

Check disk space

df -h — if root or data partition is >90%, performance degrades severely.

Restart to reclaim memory

pm2 restart minusnow — Node.js garbage collector sometimes needs a clean restart.

Memory leak detection

# Watch memory over time (every 5 seconds) watch -n 5 "pm2 jlist | jq '.[0].monit'" # If memory grows continuously without dropping, likely a leak. # Schedule periodic restarts as a mitigation: pm2 restart minusnow --cron-restart="0 4 * * *" # Restarts at 4 AM daily

Complete Recovery Playbook Emergency

If the entire application is down and you need to recover from scratch, follow these steps in order:

# ===== STEP 1: Check system health ===== uptime # Server is reachable? df -h # Disk space available? free -m # Memory available? # ===== STEP 2: Start PostgreSQL ===== sudo systemctl start postgresql sudo systemctl status postgresql # ===== STEP 3: Verify database connectivity ===== psql "$DATABASE_URL" -c "SELECT 1" # ===== STEP 4: Navigate to app directory ===== cd /opt/minusnow # ===== STEP 5: Pull latest code (if git-deployed) ===== git pull origin main # ===== STEP 6: Install dependencies ===== npm install # ===== STEP 7: Build the application ===== npm run build # ===== STEP 8: Push database schema ===== npm run db:push # ===== STEP 9: Start/restart with PM2 ===== pm2 delete minusnow 2>/dev/null; pm2 start dist/index.cjs --name minusnow pm2 save # ===== STEP 10: Restart Nginx ===== sudo nginx -t && sudo systemctl reload nginx # ===== STEP 11: Verify everything ===== curl -s http://localhost:5000/api/health | jq . curl -I https://minusnow.com curl -I https://app.minusnow.com

Docker Recovery Container

# Check container status docker compose ps # Restart all services docker compose down && docker compose up -d # View application logs docker compose logs -f --tail=100 app # View database logs docker compose logs -f --tail=100 db # Rebuild from scratch (keeps database volume) docker compose down docker compose build --no-cache docker compose up -d # Nuclear option: full reset (WARNING: destroys data) # docker compose down -v && docker compose up -d

Quick Reference Commands

ActionCommand
Check app statuspm2 status
View live logspm2 logs minusnow --lines 200
Restart apppm2 restart minusnow
Stop apppm2 stop minusnow
Check health endpointcurl http://localhost:5000/api/health
Start PostgreSQLsudo systemctl start postgresql
Push DB schemanpm run db:push
Full rebuildnpm install && npm run build
Test Nginx configsudo nginx -t
Reload Nginxsudo systemctl reload nginx
Check SSL cert expirysudo certbot certificates
Check disk spacedf -h
Check memoryfree -m
Kill stuck portsudo fuser -k 5000/tcp
Backup databasepg_dump "$DATABASE_URL" -F c -f backup.dump