Carrying a pinned OpenFOAM case to AWS Batch: hashed results, two meshes and a drag comparison with the local run

The “车身气动” (body aerodynamics) lane in pai-design-workbench does not let a model decide whether a body design is good. It hands that to an OpenFOAM RANS solve: the same Ahmed-type body is solved on two meshes, and four checks give the verdict. When an AWS Batch job definition is configured, each case runs as its own Batch job, and when the results come back the host checks their hashes and the solver version before any check runs. This article follows docs/AERO.md, native/cfd_review.py, native/cfd_job.py and a saved review record, docs/evidence/aero-batch.json, all pinned at commit f225835: what each step hands over, what it verifies, and how far apart the local and Batch drag coefficients in that record actually are.

Every number here comes from the project’s own documentation, code and records ([O], reported by the project). This site has not run these cases or reproduced the Batch review. The numbers hold only for their meshes, iteration counts and turbulence model, the same rule taught in the course lesson Physical AI Lab 0.4: Ten Thousand Tries in a Virtual World — Putting the Conditions Back on Three Simulation Numbers; here it is applied to an aerodynamics case whose files can be checked one by one.

What the lane computes and how it judges

According to AERO.md, the geometry is a parametric Ahmed-type body generated with CadQuery 2.8 / OCCT 7.9. The Ahmed body is the standard automotive bluff-body benchmark (Ahmed, Ramm & Faltin, 1984): 1.044 × 0.389 × 0.288 m, a 0.1 m nose radius, a 0.222 m rear slant and a 50 mm ground clearance. The 25° body is the reference in every review.

The solver is OpenFOAM v2512 from the official OpenCFD image opencfd/openfoam-default:2512, pinned by digest through the environment variable PAI_OPENFOAM_IMAGE. Meshing runs blockMesh, surfaceFeatureExtract, then snappyHexMesh; the flow uses simpleFoam, k-ω SST, wall functions, 40 m/s, a moving ground, and a half model with a symmetry plane. The four checks (pai-cfd-checks-1):

Check	Definition	Default threshold
drag-coefficient	Fine-mesh (level 4) Cd, mean of the last 400 of 1200 SIMPLE iterations	Cd ≤ 0.24
grid-convergence	|Cd(level 4) − Cd(level 3)| / Cd(level 4); the documentation states this is two-level mesh dependence, not a GCI	≤ 12 %
iterative-convergence	Stationarity of the averaged Cd: |mean of the last 400 − mean of the last 200| / mean, worst of both meshes; the raw oscillation band is kept in cfd.json	≤ 1 %
mesh-quality	checkMesh reports “Mesh OK” on both meshes	Both meshes pass

The iterative check was rewritten after a rejection. The documentation records that at level 4, steady RANS of the 12.5° body oscillates quasi-periodically: Cd 0.2255–0.2313, a period of about 300 iterations, a 2.6 % band over 100 iterations. The first end-to-end run used that band as its check and rejected the body. The check now averages over 400 iterations and judges whether that average is stationary. Results enter the same feedback, recheck and release path as the other lanes, as EvalArc JUnit (EvalArc is introduced in EvalArc: the score went up, so where did the agent regress?).

How one case goes to Batch and back to the host

cfd_review.py supports two runners: local docker (--network none, the host user, the case directory as the only mount) and batch. The Batch path runs in this order:

1. The host writes the case with cfd_case.py (dictionaries and body.stl), packs it as case.tar and uploads it to S3 under the prefix jobs/<run>/<name>-l<level>/. run.sh is excluded from the archive.
2. It submits one Batch job. Dockerfile.cfd is built FROM the same digest-pinned OpenCFD image and adds only Python and boto3; each job gets 16 vCPU and 32 GiB and is not retried.
3. On the job side, cfd_job.py requires a prefix that starts with jobs/ and contains no ... Before unpacking it accepts only plain files and directories and refuses absolute paths and paths containing ..; links are refused because they are not plain files. A code comment says members must be under case/, but the code does not check this. Extraction uses filter="data", and afterwards case/system/controlDict must exist. The run.sh it runs comes from the image itself, never from the input.
4. The job writes coefficient.dat, run.json, log.checkMesh and result.json (schema pai-cfd-job-1, with the SHA-256 of each file, the OpenFOAM version, the image tag, the elapsed time and the CPU count), and uploads them with ChecksumSHA256.
5. The host polls the job every 10 seconds, with a default deadline of 7200 seconds. A failed or overdue job is reported, never resubmitted. On success the host requires status to be measured and openfoam to be v2512, then downloads the three files one by one and compares each with the digest in result.json; any mismatch is an error.
6. The host computes Cd from coefficient.dat and judges the four checks itself. The case directories are then deleted: the summarised evidence is kept, not the roughly 100 MB of field data.

Note the limit of the hash comparison. The host downloads result.json from the same S3 prefix as the three files, so a match shows only that the downloaded files agree with the digests in the downloaded result.json. It catches corruption, or a file changed without result.json being updated; it does not catch someone who can write to that prefix replacing both. The ChecksumSHA256 attached when the job uploads protects integrity during that upload. Nor does the comparison independently prove that the solve itself is right. Whether the solve can be trusted still depends on the mesh and iteration checks and on the limitations below.

The saved review: two bodies, two meshes, four jobs

aero-batch.json records a 12.5° body review completed on the project’s hosted site (checkedAt 2026-10-03T21:50:00Z, verdict accepted-aero-body, 0 blocking items, 9 minutes). The reference is the 25° body and the candidate the 12.5° body; each body has two meshes, giving 4 Batch jobs, each reporting 16 CPUs:

Body	Mesh	Cells	Cd	Host timing (s)	Job-reported (s)
Reference 25°	level 3	61,310	0.25751	141.9	61
Reference 25°	level 4	219,622	0.23406	352.5	269.1
Candidate 12.5°	level 3	61,573	0.25366	152	61.6
Candidate 12.5°	level 4	221,354	0.22926	262.1	169.2

All four checks are marked passed for both bodies: the reference has a grid change of 0.1002 and an iterative drift of 0, the candidate a grid change of 0.1064 and a drift of 0.00012, and both pass mesh-quality on 2 meshes. The record does not state the thresholds used in this review; against the documented defaults, grid changes of 10.0 % and 10.6 % sit close to the 12 % limit. Host timing starts before submission and ends after the results are parsed, so its excess over the job-reported time includes upload, queueing, polling and download; the record does not break this down.

Local versus Batch: reading the drag coefficients and timings

The record’s localComparison gives a Cd of 0.23406 for the reference body both locally and on Batch; for the candidate, 0.22945 locally and 0.22926 on Batch. AERO.md calls them “equal”, but the candidate values actually differ by 0.00019, about 0.08 % of the local value. The record does not explain the difference, and this article does not speculate.

For timing, the record gives 94 minutes locally on 8 cores and 9 minutes on Batch. This is one saved development run: locally 8 cores, on Batch four jobs of 16 vCPU each, with different hardware, core counts and scheduling, and the record gives no cost. It does not show that Batch is ten times faster in general, and it is not a cost comparison.

Reproducing it locally

The local run commands given in AERO.md are below; run them from the repository root at commit f225835:

docker pull opencfd/openfoam-default:2512
echo "PAI_OPENFOAM_IMAGE=$(docker image inspect opencfd/openfoam-default:2512 --format '{{index .RepoDigests 0}}')" >> .state/demo.env
npm run test:aero   # 35° body vs the Cd target → feedback → 12.5° body re-solved on two meshes → feedback closed

The comment describes the test: the 35° body is checked against the Cd target, feedback is raised, then the 12.5° body is re-solved on two meshes until the feedback is closed. The second line writes the digest of the image your machine pulled into the environment file, so it pins the digest you pulled, which is not necessarily the one used in the recorded review (the record shows it truncated as sha256:33fb575a…). The supplied documentation gives no steps for cloning the repository or installing npm dependencies and the CadQuery environment; checking out the commit pins the repository’s files, not the versions of those dependencies.

Through the API the request is POST /api/projects/:id/aero with a body containing requestId, projectRevision and parameters (slantAngleDeg, noseRadius, length, height), with requirements optional. The AI tool aero-body only proposes typed parameters, which still need a person’s confirmation. Batch is used only when PAI_SOLVER_CFD_JOB_DEFINITION is configured; the supplied material does not include the job queue, role or network definitions, so this article gives no deployment steps. If you build your own, grant the job role read and write only on the jobs/ prefix of the bucket in use, and on AccessDenied check the prefix rather than widening permissions.

Limitations

• Mesh dependence is still significant: Cd changes by 9–10 % from level 3 to level 4, and the documentation itself says a finer mesh is still needed for absolute values. The lane currently runs without prism layers; the documentation lists layers and finer levels as the next step on AWS Batch.
• Drag is under-predicted: locally at level 4 the 25° body gives Cd 0.234 against a measured 0.285; at level 3 the 30° body gives 0.263 against 0.378. The documentation attributes this to a known weakness of steady RANS with wall functions: the measured high-drag state at 30° depends on unsteady C-pillar vortices and partial separation on the slant.
• The ranking holds only at the low-drag end: the 12.5° body is the lowest-drag body both in the experiment and in the computation, and at level 4 its 0.230 matches the measured 0.230. But in the same table, level 4 puts the 35° body (0.249) above the 25° body (0.234), whereas the experiment puts 25° (0.285) above 35° (0.260), the opposite order. Comparing other geometry ranges with it needs separate validation.
• Every record carries physicalValidation: false. This is a design-comparison tool, not a wind-tunnel, road or any physical safety validation.
• The saved Batch record shows that one hosted review ran to completion; it is not an independent reproduction, and this site has not re-run it.
• The module docstring at the top of cfd_review.py still describes a “mean of the last 100 iterations” and a (max − min) / mean band, which disagrees with the code in the same file and with the 400/200 stationarity check in AERO.md; this article follows the code and AERO.md.

Links

• docs/AERO.md: the aerodynamics lane (commit f225835)
• native/cfd_review.py: two-mesh solve, Batch result verification and the four checks
• native/cfd_job.py: one case as a container job
• docs/evidence/aero-batch.json: the saved Batch review record
• Physical AI Lab 0.4: Ten Thousand Tries in a Virtual World — Putting the Conditions Back on Three Simulation Numbers
• EvalArc: the score went up, so where did the agent regress?
• Physical AI Lab: a physical AI course where every number can be checked (course outline)