Optimize CI: separate fast tests from live infrastructure tests #315

jumski · 2025-11-04T10:29:09Z

Split test targets into two categories:

test: Fast tests without infrastructure (vitest, types)
test:live: Tests requiring live infrastructure (pgtap, integration)

Enable Nx Cloud caching for verification tasks:

Remove local: true from verify-* targets
Add db:verify meta-target for verification pipeline
Verification results now cached across CI jobs

Restructure CI workflow:

Job 1: db-verification (runs once, caches to Nx Cloud)
Job 2: fast-tests (restores cache, runs all packages in parallel)
Job 3-5: *-live-tests (separate jobs per package infrastructure)
Job 6: edge-worker-e2e (unchanged)

Benefits:

verify-migrations runs once instead of multiple times
Fast tests complete in ~2-3 min with full parallelization
Simple commands: nx affected -t test (fast), nx affected -t test:live (infra)
Nx handles dependency resolution and caching automatically

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

changeset-bot · 2025-11-04T10:29:13Z

⚠️ No Changeset found

Latest commit: ec56e39

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

jumski · 2025-11-04T10:29:25Z

Optimize CI: separate fast tests from live infrastructure tests #315 👈 (View in Graphite)
main

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

merge:queue - adds this PR to the back of the merge queue
hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

nx-cloud · 2025-11-04T10:30:10Z

🤖 Nx Cloud AI Fix Eligible

An automatically generated fix could have helped fix failing tasks for this run, but Self-healing CI is disabled for this workspace. Visit workspace settings to enable it and get automatic fixes in future runs.

_{To disable these notifications, a workspace admin can disable them in workspace settings.}

View your CI Pipeline Execution ↗ for commit ec56e39

Command	Status	Duration	Result
`nx run edge-worker:test:live`	❌ Failed	4m 16s	View ↗
`nx run cli:test:live`	❌ Failed	3s	View ↗
`nx run client:test:live`	✅ Succeeded	2m 44s	View ↗
`nx affected -t build --configuration=production...`	✅ Succeeded	22s	View ↗
`nx affected -t lint typecheck test --parallel -...`	✅ Succeeded	1m 11s	View ↗
`nx run core:test:live`	✅ Succeeded	1s	View ↗
`nx run core:db:verify`	✅ Succeeded	3m 11s	View ↗

☁️ Nx Cloud last updated this comment at 2025-11-08 13:18:20 UTC

graphite-app · 2025-11-04T10:30:40Z

.github/workflows/ci.yml

  edge-worker-e2e:
    runs-on: ubuntu-latest
    env:


The edge-worker-e2e job should include a dependency on the db-verification job to maintain consistency with the other test jobs. Adding needs: [db-verification] would ensure that database verification completes successfully before running the E2E tests, preventing potential issues from unverified database changes.

Suggested change

edge-worker-e2e:

runs-on: ubuntu-latest

env:

edge-worker-e2e:

runs-on: ubuntu-latest

needs: [db-verification]

env:

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

.github/workflows/ci.yml

pkgs/client/project.json

.github/workflows/ci.yml

pkgs/core/project.json

graphite-app · 2025-11-04T20:12:10Z

.github/workflows/ci.yml

+      - name: Run core live tests
+        run: pnpm nx run core:test:live


The core-live-tests job unconditionally runs pnpm nx run core:test:live without checking if core is affected by the changes. This is inconsistent with client-live-tests (line 119-121) and edge-worker-live-tests (line 153-155), which both check if they are affected before running.

This means core's live tests will run on every CI run, even when core is not affected by any changes, wasting CI resources and time.

Fix: Add an affected check similar to the other packages:

- name: Check if core is affected id: check-affected run: | if pnpm nx show projects --affected --base="$NX_BASE" --head="$NX_HEAD" | grep -q "^core$"; then echo "affected=true" >> $GITHUB_OUTPUT echo "Core is affected by changes" else echo "affected=false" >> $GITHUB_OUTPUT echo "Core is not affected by changes - skipping" fi - name: Run core live tests if: steps.check-affected.outputs.affected == 'true' run: pnpm nx run core:test:live

Suggested change

- name: Run core live tests

run: pnpm nx run core:test:live

- name: Check if core is affected

id: check-affected

run: |

if pnpm nx show projects --affected --base="$NX_BASE" --head="$NX_HEAD" | grep -q "^core$"; then

echo "affected=true" >> $GITHUB_OUTPUT

echo "Core is affected by changes"

else

echo "affected=false" >> $GITHUB_OUTPUT

echo "Core is not affected by changes - skipping"

fi

- name: Run core live tests

if: steps.check-affected.outputs.affected == 'true'

run: pnpm nx run core:test:live

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

github-actions · 2025-11-04T20:20:32Z

🔍 Preview Deployment: Website

✅ Deployment successful!

🔗 Preview URL: https://pr-315.pgflow.pages.dev

📝 Details:

Branch: ci-optimize-test-separation
Commit: 1c3e5d403bd0d02e7db0e6d0d79c6f2db646916c
View Logs

_Last updated: _

pkgs/core/src/database-types.ts

graphite-app · 2025-11-05T16:58:02Z

.github/workflows/ci.yml

-# ─────────────────────────────────────── 2. EDGE-WORKER E2E ──────────────────────────────────────
-  edge-worker-e2e:
+# ─────────────────────────────────────── 3. CLI LIVE TESTS ──────────────────────────────────────
+  cli-live-tests:


Consider running live tests sequentially instead of in parallel by making cli-live-tests depend on db-verification, core-live-tests depend on cli-live-tests, client-live-tests depend on core-live-tests, and edge-worker-live-tests depend on client-live-tests. This would prevent resource conflicts at the cost of longer CI run times.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T17:03:54Z

.github/workflows/ci.yml

+      - name: Run edge-worker live tests
        if: steps.check-affected.outputs.affected == 'true'
-        run: pnpm nx affected -t test:e2e --parallel --base="$NX_BASE" --head="$NX_HEAD"
+        run: pnpm nx run edge-worker:test:live


Add a timeout to prevent the edge-worker live tests from being terminated prematurely. Change to: run: pnpm nx run edge-worker:test:live --timeout=10m

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T17:11:24Z

.github/workflows/ci.yml

+  core-live-tests:
+    needs: [db-verification]


The core-live-tests job should depend on fast-tests instead of db-verification to ensure it runs after fast-tests completes, creating a sequential chain of jobs.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T17:11:27Z

scripts/ensure-supabase.sh

+cd "$SUPABASE_DIR"
+
+if pnpm supabase status &>/dev/null; then
+  echo "✓ Supabase already running in $SUPABASE_DIR"
+  exit 0
+fi
+
+echo "Starting Supabase in $SUPABASE_DIR..."
+pnpm supabase start


Add a locking mechanism to prevent multiple processes from starting Supabase simultaneously. This could be implemented using flock or a simple file-based lock to ensure only one process can start Supabase at a time.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T19:16:39Z

pkgs/cli/scripts/test-install

 # Initialize a fresh Supabase project
 echo "🏗️ Creating new Supabase project"
-npx -y supabase@latest init --force --with-vscode-settings --with-intellij-settings
+supabase init --force --with-vscode-settings --with-intellij-settings


This line is missing the '--force' flag which is causing the test to fail with an EEXIST error when trying to create a directory that already exists. The command should be 'supabase init --force --with-vscode-settings --with-intellij-settings' to match the other similar scripts in the codebase.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T19:23:24Z

pkgs/edge-worker/project.json

    "test:e2e": {
      "executor": "nx:run-commands",
-      "dependsOn": ["supabase:reset", "serve:functions:e2e"],
+      "dependsOn": ["supabase:reset", "sync-e2e-deps"],
      "local": true,
      "inputs": ["default", "^production"],
      "options": {
        "cwd": "pkgs/edge-worker",
        "commands": [
+          "../../scripts/ensure-supabase.sh .",
          "deno test --config deno.test.json --allow-all --env=supabase/functions/.env tests/e2e/"
        ],
        "parallel": false


Add retry mechanism for the supabase db reset command to handle potential interruptions. The current implementation fails with SIGINT (code 130) which suggests the command is being interrupted, possibly due to race conditions with other jobs.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-05T19:23:25Z

scripts/ensure-supabase.sh

+#!/usr/bin/env bash
+set -e
+
+# Usage: ensure-supabase.sh <path-to-supabase-parent-dir>
+if [ -z "$1" ]; then
+  echo "ERROR: Supabase directory path required"
+  echo "Usage: ensure-supabase.sh <path-to-supabase-parent-dir>"
+  exit 1
+fi
+
+SUPABASE_DIR="$1"
+
+if [ ! -d "$SUPABASE_DIR/supabase" ]; then
+  echo "ERROR: No supabase/ directory found in $SUPABASE_DIR"
+  exit 1
+fi
+
+cd "$SUPABASE_DIR"
+
+if pnpm supabase status &>/dev/null; then
+  echo "✓ Supabase already running in $SUPABASE_DIR"
+  exit 0
+fi
+
+echo "Starting Supabase in $SUPABASE_DIR..."
+pnpm supabase start


Enhance script to handle potential race conditions by adding retries and better error handling. The script should be more resilient when multiple CI jobs call it simultaneously.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-06T08:40:08Z

.github/actions/setup/action.yml

+    - name: Install sqruff
+      uses: quarylabs/install-sqruff-cli-action@main


This step is failing because the 'quarylabs/install-sqruff-cli-action@main' action cannot successfully download and install the sqruff-linux-x86_64-musl.tar.gz file. Either pin this action to a specific stable version (e.g., 'quarylabs/install-sqruff-cli-action@v1.0.0') instead of using the potentially unstable '@main' reference, or temporarily remove this step until the upstream action is fixed.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-06T08:45:19Z

pkgs/edge-worker/project.json

        "commands": [
+          "../../scripts/ensure-supabase.sh .",
          "mkdir -p supabase/migrations/",
          "rm -f supabase/migrations/*.sql",
          "cp ../core/supabase/migrations/*.sql supabase/migrations/",


The 'supabase db reset' command is likely hanging or timing out. We should add a timeout and potentially retry logic. Replace the last command with 'timeout 60s supabase db reset || (echo "Retrying database reset..." && timeout 60s supabase db reset)' to give the command a 60-second timeout and one retry attempt if it fails.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-06T13:11:05Z

scripts/ensure-supabase.sh

+if pnpm supabase status &>/dev/null; then
+  echo "✓ Supabase already running in $SUPABASE_DIR"
+  exit 0
+fi
+
+echo "Starting Supabase in $SUPABASE_DIR..."
+pnpm supabase start


The script doesn't handle timeouts or potential failures when starting Supabase. It should include a timeout mechanism and retry logic to prevent hanging indefinitely, which is likely causing the SIGINT termination in CI.

Spotted by Graphite Agent (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

Split test targets into two categories: - test: Fast tests without infrastructure (vitest, types) - test:live: Tests requiring live infrastructure (pgtap, integration) Enable Nx Cloud caching for verification tasks: - Remove local: true from verify-* targets - Add db:verify meta-target for verification pipeline - Verification results now cached across CI jobs Restructure CI workflow: - Job 1: db-verification (runs once, caches to Nx Cloud) - Job 2: fast-tests (restores cache, runs all packages in parallel) - Job 3-5: *-live-tests (separate jobs per package infrastructure) - Job 6: edge-worker-e2e (unchanged) Benefits: - verify-migrations runs once instead of multiple times - Fast tests complete in ~2-3 min with full parallelization - Simple commands: nx affected -t test (fast), nx affected -t test:live (infra) - Nx handles dependency resolution and caching automatically 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>