Simulation Orchestrator

Goal

Provide tools to manage multi-simulation campaigns: generate parameter sweeps, track job execution status, and aggregate results from completed runs.

Requirements

• Python 3.10+
• No external dependencies (uses Python standard library only)
• Works on Linux, macOS, and Windows

Inputs to Gather

Before running orchestration scripts, collect from the user:

base_config.json

dt:[1e-4,1e-2],kappa:[0.1,1.0]

grid

lhs

linspace

./campaign_001

python sim.py --config {config}

Decision Guidance

Choosing a Sweep Method

Need every combination (full factorial)?
├── YES → Use grid (warning: exponential growth with parameters)
└── NO → Is space-filling coverage needed?
    ├── YES → Use lhs (Latin Hypercube Sampling)
    └── NO → Use linspace for uniform sampling per parameter

grid

linspace

lhs

Campaign Size Guidelines

Script Outputs (JSON Fields)

scripts/sweep_generator.py

configs

parameter_space

sweep_method

total_runs

scripts/campaign_manager.py

campaign_id

status

jobs

progress

scripts/job_tracker.py

job_id

status

start_time

end_time

exit_code

scripts/result_aggregator.py

summary

statistics

best_run

failed_runs

Workflow

Step 1: Generate Parameter Sweep

Create configurations for all parameter combinations:

python3 scripts/sweep_generator.py \
    --base-config base_config.json \
    --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \
    --method linspace \
    --output-dir ./campaign_001 \
    --json

Step 2: Initialize Campaign

Create campaign tracking structure:

python3 scripts/campaign_manager.py \
    --action init \
    --config-dir ./campaign_001 \
    --command "python sim.py --config {config}" \
    --json

Step 3: Track Job Status

Monitor running jobs:

python3 scripts/job_tracker.py \
    --campaign-dir ./campaign_001 \
    --update \
    --json

Step 4: Aggregate Results

Combine results from completed runs:

python3 scripts/result_aggregator.py \
    --campaign-dir ./campaign_001 \
    --metric objective_value \
    --json

CLI Examples

# Generate 5x3=15 runs varying dt (5 values) and kappa (3 values)
python3 scripts/sweep_generator.py \
    --base-config sim.json \
    --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \
    --method linspace \
    --output-dir ./sweep_001 \
    --json

# Generate LHS samples for 4 parameters with budget of 20 runs
python3 scripts/sweep_generator.py \
    --base-config sim.json \
    --params "dt:1e-4:1e-2,kappa:0.1:1.0,M:1e-6:1e-4,W:0.5:2.0" \
    --method lhs \
    --samples 20 \
    --output-dir ./lhs_001 \
    --json

# Check campaign status
python3 scripts/campaign_manager.py \
    --action status \
    --config-dir ./sweep_001 \
    --json

# Get summary statistics from completed runs
python3 scripts/result_aggregator.py \
    --campaign-dir ./sweep_001 \
    --metric final_energy \
    --json

Conversational Workflow Example

User: I want to run a parameter sweep on dt and kappa for my phase-field simulation. I want to try 5 values of dt between 1e-4 and 1e-2, and 4 values of kappa between 0.1 and 1.0.

Agent workflow:

1. Calculate total runs: 5 x 4 = 20 runs
2. Generate sweep configurations:

   python3 scripts/sweep_generator.py \
       --base-config simulation.json \
       --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:4" \
       --method linspace \
       --output-dir ./dt_kappa_sweep \
       --json
   

3. Initialize campaign:

   python3 scripts/campaign_manager.py \
       --action init \
       --config-dir ./dt_kappa_sweep \
       --command "python phase_field.py --config {config}" \
       --json
   

4. After user runs simulations, aggregate results:

   python3 scripts/result_aggregator.py \
       --campaign-dir ./dt_kappa_sweep \
       --metric interface_width \
       --json
   

Error Handling

Base config not found

Invalid parameter format

name:min:max:count

name:min:max

Output directory exists

--force

No completed jobs

Metric not found

Integration with Other Skills

The simulation-orchestrator works with other simulation-workflow skills:

parameter-optimization          simulation-orchestrator
        │                              │
        │ DOE samples ────────────────>│ Generate configs
        │                              │
        │                              │ Run simulations
        │                              │
        │<──────────────────────────── │ Aggregate results
        │                              │
        │ Sensitivity analysis         │
        │ Optimizer selection          │

Typical Combined Workflow

1. Use

   parameter-optimization/doe_generator.py

   to get sample points
2. Use

   simulation-orchestrator/sweep_generator.py

   to create configs
3. Run simulations (user's responsibility)
4. Use

   simulation-orchestrator/result_aggregator.py

   to collect results
5. Use

   parameter-optimization/sensitivity_summary.py

   to analyze

Security

Input Validation

• Metric names are validated against

  [a-zA-Z_][a-zA-Z0-9_.]*

  to prevent traversal or injection via crafted keys

• campaign_manager.py

  validates command templates to reject shell chaining operators (

  ;

  ,

  |

  ,

  &

  , backticks,

  $

  )

• --params

  format strings are parsed and validated (

  name:min:max:count

  with finite numeric bounds and positive integer counts)

• --method

  is validated against a fixed allowlist (

  grid

  ,

  linspace

  ,

  lhs

  )

• --samples

  is validated as a positive integer with an upper bound

• --action

  is validated against a fixed allowlist (

  init

  ,

  status

  )

File Access

• sweep_generator.py

  reads a single base config file (JSON) specified by

  --base-config

  and writes generated configs to

  --output-dir

• result_aggregator.py

  enforces a 10 MB file-size limit per result file, maximum JSON nesting depth, and strict numeric type checking (rejects

  bool

  ,

  NaN

  ,

  Inf

  )
• All string values from result files are sanitized (truncated, control characters stripped) before surfacing them
• Config paths interpolated into shell commands are validated against a safe-character allowlist and escaped with

  shlex.quote()

Tool Restrictions

• Read: Used to inspect script source, references, base configs, and campaign status files
• Write: Used to save generated sweep configs, campaign manifests, and aggregated results; writes are scoped to the user's working directory
• Grep/Glob: Used to locate campaign files, result files, and search references
• The skill's

  allowed-tools

  excludes

  Bash

  to prevent the agent from executing arbitrary commands when processing untrusted simulation outputs

Safety Measures

• No

  eval()

  ,

  exec()

  , or dynamic code generation
• All subprocess calls use explicit argument lists (no

  shell=True

  )
• Reduced tool surface (no Bash) limits the agent to read/write operations only
• Command templates are validated but never executed by the skill itself; execution is the user's responsibility

Limitations

• Not a job scheduler: Does not submit jobs to SLURM/PBS; generates configs and tracks status
• No parallel execution: User must run simulations externally (can use GNU parallel, SLURM, etc.)
• File-based tracking: Status tracked via files; no database or real-time monitoring
• Local filesystem: Assumes all files accessible from local machine

References

• references/campaign_patterns.md

  - Common campaign structures

• references/sweep_strategies.md

  - Parameter sweep design guidance

• references/aggregation_methods.md

  - Result aggregation techniques

Version History

• v1.0.0 (2024-12-24): Initial release with sweep, campaign, tracking, and aggregation