What you will learn

Run a batch of independent jobs in parallel with a hard concurrency cap, using only Bash job control — no GNU parallel dependency — and collect a clean pass/fail summary at the end.

Before you begin

Requires Bash 4.3+ for wait -n. Decide your concurrency cap based on CPU cores or downstream rate limits, not an arbitrary round number.

1. Queue the work and cap concurrency with wait -n

#!/usr/bin/env bash
set -Eeuo pipefail

MAX_PARALLEL=4
declare -a JOBS=("process-a" "process-b" "process-c" "process-d" "process-e" "process-f")
declare -A RESULTS=()

run_one() {
    local name="$1"
    if ./run-job.sh "$name" > "/tmp/${name}.log" 2>&1; then
        echo "ok"
    else
        echo "fail"
    fi
}

2. Fan out without exceeding the cap

running=0
declare -A PID_TO_NAME=()

for name in "${JOBS[@]}"; do
    ( run_one "$name" > "/tmp/${name}.status" ) &
    PID_TO_NAME[$!]="$name"
    running=$((running + 1))

    if (( running >= MAX_PARALLEL )); then
        if wait -n; then :; fi
        running=$((running - 1))
    fi
done

wait   # drain whatever's left

wait -n returns as soon as any one backgrounded job finishes, freeing a slot immediately rather than waiting for a whole batch of MAX_PARALLEL to complete together — which would idle finished workers.

3. Collect results without relying on exit-code ordering

fail_count=0
for name in "${JOBS[@]}"; do
    status="$(cat "/tmp/${name}.status" 2>/dev/null || echo "fail")"
    RESULTS[$name]="$status"
    [[ "$status" == "fail" ]] && fail_count=$((fail_count + 1))
    echo "${name}: ${status}"
done

echo "---"
echo "${#JOBS[@]} jobs, ${fail_count} failed"
(( fail_count == 0 ))

Writing each job's outcome to its own status file sidesteps the fact that wait -n's own exit code tells you a job finished and its status, not which job it was.

Example output

process-a: ok
process-b: ok
process-c: fail
process-d: ok
process-e: ok
process-f: ok
---
6 jobs, 1 failed

Verify the result

Force one job to fail deliberately and confirm the summary line counts it without the script dying mid-batch (that's what > status file plus checking at the end, not set -e on the loop, buys you).

Troubleshooting

If jobs seem to run serially, confirm run_one is actually backgrounded (& on the subshell) and that run-job.sh doesn't itself block on a shared lock. If PIDs collide unexpectedly, remember $! is only valid immediately after the &.

Next steps

For hundreds of jobs, generate them from a file with mapfile -t JOBS < joblist.txt instead of a literal array, and consider ssh-fleet-command-executor if the jobs are per-host commands.