Claude-skill-registry c-incremental-build-converter
Convert C/C++ project build.sh and test.sh scripts to support incremental builds. Use when fixing build scripts for re-execution after docker commit, converting to out-of-tree builds, or making scripts idempotent for repeated execution.
git clone https://github.com/majiayu000/claude-skill-registry
T=$(mktemp -d) && git clone --depth=1 https://github.com/majiayu000/claude-skill-registry "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/data/c-incremental-build-converter" ~/.claude/skills/majiayu000-claude-skill-registry-c-incremental-build-converter && rm -rf "$T"
skills/data/c-incremental-build-converter/SKILL.mdC/C++ Incremental Build Converter Skill
This skill converts C/C++ project build.sh and test.sh scripts to support incremental builds after docker commit.
CRITICAL CONSTRAINTS
1. DO NOT Modify oss-fuzz Infrastructure Code
NEVER modify these files:
oss-fuzz/infra/helper.pyoss-fuzz/infra/base-images/*- Any other oss-fuzz infrastructure code
ONLY modify:
oss-fuzz/projects/{project}/build.shoss-fuzz/projects/{project}/test.sh
(if absolutely necessary)oss-fuzz/projects/{project}/Dockerfile
2. If oss-crs Test Framework Has Issues
If the incremental build test framework (
incremental_build_checker.py) has bugs or limitations:
- Report the issue to the user with clear explanation
- Suggest a fix for the framework code
- DO NOT implement the fix yourself - let the user decide
Example report format:
## oss-crs Framework Issue Detected **File:** bug_fixing/src/oss_patch/inc_build_checker/incremental_build_checker.py **Issue:** The source copy command destroys build artifacts **Current behavior:** `rm -rf $SRC/nginx && cp -r ...` **Expected behavior:** Should preserve objs/ directory **Suggested fix:** Use rsync with --exclude objs instead of rm+cp
Key Architecture: Separate $SRC for build.sh and test.sh
IMPORTANT: $SRC is different between build.sh and test.sh.
- build.sh runs in
→/built-src
=$SRC/built-src - test.sh runs in
→/test-src
=$SRC/test-src
These are completely separate directories. This separation means:
- NO artifact cleanup needed - Don't delete .o, .lo, .a, .la, .libs
- NO out-of-tree build needed - In-source build is fine since $SRC is separate
- NO ASAN conflict - test.sh builds in its own clean copy
ALWAYS Use $SRC, NEVER Hardcode Paths
CRITICAL: Always use
variable, never hardcode $SRC
, /src
, or /built-src
./test-src
# BAD - Hardcoded path breaks when $SRC changes cd /src/project cp -r /src/oniguruma modules/ # GOOD - Uses $SRC variable cd $SRC/project cp -r $SRC/oniguruma modules/
AVOID $WORK - Use $SRC Subdirectories for All Build Artifacts
CRITICAL: Avoid using $WORK for build-related directories. Use $SRC subdirectories instead.
$WORK is shared between build.sh and test.sh, which causes problems:
- ASAN-instrumented libraries from build.sh conflict with test.sh
- CMake paths recorded in $WORK become invalid when $SRC changes
- Build artifacts from different environments can corrupt each other
Always use
subdirectories for:$SRC
- Build directories (out-of-source builds)
- Install prefixes (for cmake/configure --prefix)
- Dependency installations
# BAD - Uses $WORK which is shared BUILD_DIR="$WORK/build" cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX=$WORK # GOOD - Uses $SRC subdirectories BUILD_DIR="$SRC/project_build" INSTALL_DIR="$SRC/project_install" mkdir -p "$BUILD_DIR" "$INSTALL_DIR" cd "$BUILD_DIR" cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR"
In test.sh, also use $SRC subdirectories:
# BAD in test.sh - Uses shared $WORK with ASAN artifacts ./configure --prefix="$WORK" # Error: undefined reference to __asan_* # GOOD in test.sh - Uses separate prefix under $SRC TEST_PREFIX="$SRC/test_deps" mkdir -p "$TEST_PREFIX/lib" "$TEST_PREFIX/include" if [ ! -f "$TEST_PREFIX/lib/libz.a" ]; then pushd "$SRC/zlib" ./configure --static --prefix="$TEST_PREFIX" make -j$(nproc) CFLAGS="-fPIC" # No ASAN flags make install popd fi
test.sh Does NOT Need Sanitizers - Disable Them Explicitly
IMPORTANT: test.sh runs unit tests, NOT fuzzing. Sanitizers are NOT needed and cause conflicts.
During incremental build testing, build.sh compiles with ASAN/sanitizer flags. When test.sh runs in the SAME directory, the object files from build.sh are ASAN-instrumented. If test.sh tries to link against these objects without ASAN runtime, you get undefined reference errors.
build.sh and test.sh Artifacts Should NOT Be Shared
CRITICAL: Build artifacts from build.sh should NOT be reused in test.sh.
During incremental build testing:
- build.sh runs with ASAN flags → creates ASAN-instrumented
files.o - test.sh runs in the SAME
directory$SRC - If test.sh tries to use existing
files, linking fails.o
The artifacts are incompatible because:
- build.sh: Compiles with
-fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link - test.sh: Needs to link without sanitizer runtime
DO NOT use
to ignore link failures - this hides the real problem. Fix the root cause by ensuring clean separation of build artifacts.|| true
Critical Requirements
1. Scripts MUST Work on BOTH First Run AND Repeated Runs
NEVER assume a script will only run once. Scripts must be idempotent.
# BAD - Runs configure every time autoreconf -f -i ./configure # GOOD - Only configure on first run if [ ! -f Makefile ]; then autoreconf -f -i ./configure fi make -j$(nproc)
2. DO NOT Delete Build Artifacts
Since $SRC is separate, there's no need to clean artifacts. Deleting them defeats incremental build.
# BAD - Destroys incremental build benefits find . -name "*.o" -delete find . -name "*.a" -delete rm -rf .libs # GOOD - Just build, artifacts from previous run will speed things up if [ ! -f Makefile ]; then ./configure fi make -j$(nproc)
3. Conditional Configure/Autoreconf
Only run configure-related commands on first execution.
# Pattern: Check for Makefile existence if [ ! -f Makefile ]; then autoreconf -fi # or ./bootstrap ./configure --options fi make -j$(nproc) make check
4. Use cp
Instead of mv
cpmvmv removes the source file, causing failures on repeated runs.
# BAD - Fails on second run mv scripts/config.temp scripts/config # GOOD - Preserves source file [ -f scripts/config.temp ] && cp scripts/config.temp scripts/config || true
5. Use Guards for Dependencies
# Build dependency only if not already built if [ ! -f "$WORK/lib/libz.a" ]; then pushd "$SRC/zlib" ./configure --static --prefix="$WORK" make -j$(nproc) make install popd fi
6. Handle Failing Tests
Remove failing test files directly instead of using TESTS variable (which can propagate to submodules).
# BAD - TESTS variable propagates to submodules make check TESTS="tests/mantest tests/jqtest" # GOOD - Remove failing test files directly rm -f tests/shtest rm -f tests/multiple.* sed -i '/multiple/d' tests/Makefile.am make check
7. Copy Dependencies If Not Present (Conditional, Not Delete-and-Copy)
# BAD - Deletes and recopies every time, breaks incremental build rm -rf modules/oniguruma rsync -a $SRC/oniguruma modules/ # GOOD - Only copy if not present if [ ! -d modules/oniguruma ]; then mkdir -p modules cp -r $SRC/oniguruma modules/ fi
8. Conditional Dict/Corpus File Copy
Files like dictionaries and seed corpus may not exist in all builds.
# BAD - Fails if file doesn't exist cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/ # GOOD - Conditional copy with fallback [ -f $SRC/libtiff/contrib/oss-fuzz/tiff.dict ] && cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/ || true find . -name "*.tif" | xargs zip $OUT/seed_corpus.zip || true
9. Use make -k check || true
for Potentially Failing Tests
make -k check || trueSome tests may fail in fuzzing environment. Use
-k to continue and || true to not fail the script.
# BAD - Stops on first test failure make check # GOOD - Continue through failures make -k check || true echo "Tests completed"
10. Remove Tests from Makefile.am BEFORE configure
When removing failing tests, modify Makefile.am BEFORE running autoreconf/configure.
# BAD - Removes test file but Makefile still references it rm -f tests/shtest autoreconf -fi ./configure make check # Error: No rule to make target 'tests/shtest' # GOOD - Remove from Makefile.am before configure sed -i 's| tests/shtest||g' Makefile.am autoreconf -fi ./configure make check # Works
11. Rewrite Instead of Sourcing Internal build.sh
If the project's source contains its own build.sh (e.g.,
contrib/oss-fuzz/build.sh), it may use mv or other non-idempotent operations. Rewrite the logic in the project's build.sh instead.
# BAD - Sources internal build.sh that uses mv source $SRC/project/contrib/oss-fuzz/build.sh # GOOD - Rewrite the logic with cp instead of mv if [ ! -f "$WORK/lib/libjbig.a" ]; then pushd "$SRC/jbigkit" make lib # Use cp instead of mv for incremental build support cp "$SRC"/jbigkit/libjbig/*.a "$WORK/lib/" cp "$SRC"/jbigkit/libjbig/*.h "$WORK/include/" popd fi
12. Guard sed -i
Insertions with Pattern Checks
sed -iWhen using
sed -i to insert content, check BOTH the original pattern exists AND the new content doesn't exist yet.
# BAD - Only checks if new content exists (can't distinguish original vs added) if ! grep -q '%destructor' file.y; then sed -i '/^%%$/i%destructor { free ($$); } ID' file.y fi # GOOD - Check original pattern exists AND new content not yet added if grep -q '^%%$' file.y && ! grep -q '%destructor' file.y; then sed -i '/^%%$/i%destructor { free ($$); } ID' file.y fi
13. Guard Append Operations (>>
)
>>Don't blindly use
> (overwrite) or >> (append). Check if content already exists.
# BAD - Overwrites every time (inefficient, may lose state) sed 's/main/old_main/' file.c > header.h # BAD - Appends every time (file grows on each run) sed 's/main/old_main/' file.c >> header.h # GOOD - Only append if content not already present if ! grep -q 'old_main' header.h 2>/dev/null; then sed 's/main/old_main/' file.c >> header.h fi
14. CMake/Ninja Build Directory Must Use $SRC-Relative Path
When
$SRC changes between build.sh and test.sh, CMake/Ninja will fail with "manifest dirty" errors if build directory is in $WORK.
# BAD - $WORK is shared, CMake source path mismatch causes ninja errors BUILD_DIR="$WORK/build" cd "$BUILD_DIR" cmake $SRC/project # Source path recorded as /built-src/project # On test.sh: ninja fails because $SRC is now /test-src/project # GOOD - Build directory relative to $SRC, each environment independent BUILD_DIR="$SRC/project_build" mkdir -p "$BUILD_DIR" cd "$BUILD_DIR" if [ ! -f build.ninja ]; then cmake -GNinja $SRC/project fi ninja
15. Disable ccache for Bazel Builds
Bazel has its own caching mechanism. ccache interferes with Bazel's compiler wrappers.
# BAD - ccache causes "invalid option" errors with Bazel # Error: /usr/local/bin/ccache: invalid option -- 'U' bazel build //... # GOOD - Remove ccache from PATH before Bazel builds export PATH=$(echo "$PATH" | sed 's|/ccache/bin:||g; s|:/ccache/bin||g') unset CC CXX export CC=clang export CXX=clang++ bazel build //...
Standard test.sh Template
IMPORTANT: Execution Environment
is already set (no need for$SRC
): "${SRC:=/src}"- Working directory is already
(no need for$SRC/{project}
)cd $SRC/{project}
#!/bin/bash set -ex # Build (only configure on first run) if [ ! -f Makefile ]; then autoreconf -fi # or ./bootstrap, or skip if not autotools ./configure --options fi make -j$(nproc) # Run tests (use -k to continue on failures, || true to not fail script) make -k check || true echo "Tests completed"
Examples by Build System
Autotools Project
#!/bin/bash set -ex if [ ! -f Makefile ]; then autoreconf -fi ./configure --enable-static fi make -j$(nproc) make -k check || true
CMake Project
#!/bin/bash set -ex if [ ! -f Makefile ]; then cmake . -DCMAKE_INSTALL_PREFIX=$WORK -DBUILD_SHARED_LIBS=off fi make -j$(nproc) make test
Project with Dependencies
#!/bin/bash set -ex # Build zlib (only on first run) if [ ! -f "$WORK/lib/libz.a" ]; then pushd "$SRC/zlib" ./configure --static --prefix="$WORK" make -j$(nproc) make install popd fi # Build main project if [ ! -f Makefile ]; then cmake . -DCMAKE_INSTALL_PREFIX=$WORK fi make -j$(nproc) make test
Project with Submodules (e.g., jq with oniguruma)
#!/bin/bash set -ex # Copy submodule if not present (conditional, not delete-and-copy) if [ ! -d modules/oniguruma ]; then mkdir -p modules cp -r $SRC/oniguruma modules/ fi # Remove failing tests from Makefile.am BEFORE configure sed -i 's| tests/shtest||g' Makefile.am if [ ! -f Makefile ]; then autoreconf -fi ./configure --with-oniguruma=builtin fi make -j$(nproc) # Run tests (some may fail in fuzzing environment) make -k check || true
Common Errors and Fixes
| Error | Cause | Fix |
|---|---|---|
| $WORK shared with ASAN artifacts from build.sh | Use separate prefix like in test.sh |
| Partial build state | Use guard around configure |
| Test removed but still in Makefile.am | Use `sed -i 's |
| Previous configure run | Check for Makefile before configure |
| File moved in previous run | Use with guard |
| TESTS= propagates to submodules | Remove failing test files directly |
| Original build.sh used to move files | Rewrite to use instead |
| Dict/corpus file doesn't exist | Use conditional copy: |
in submodule | deleted submodule's Makefile | Use conditional copy: |
/ duplicate content | insertion runs every time | Guard with pattern check: |
| File grows on each build | append runs every time | Guard append: |
| CMake source path mismatch ($SRC changed) | Use -relative build dir instead of |
| ccache interferes with Bazel | Remove ccache from PATH: |
Testing Incremental Builds
CRITICAL: Use the
subagent via Task tool to prevent context limit errors.test-inc-build
The subagent runs tests, analyzes logs internally, and returns only essential summary information.
Single Project Test
Task tool: subagent_type: "test-inc-build" description: "Test incremental build" prompt: "Run incremental build test for {project_name} with oss-fuzz path ../oss-fuzz"
AIXCC Projects: Use Full Path Prefix
IMPORTANT: For AIXCC projects, use the
or aixcc/c/
prefix in the project name.aixcc/jvm/
Examples:
aixcc/c/afc-sqlite3-delta-01aixcc/c/atlanta-libjpeg-full-01aixcc/jvm/afc-jenkins-delta-01
Parallel Testing Multiple Projects
Use Task tool with
run_in_background: true in a single message:
Task tool (run_in_background: true): subagent_type: "test-inc-build" prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-01 with oss-fuzz path ../oss-fuzz" Task tool (run_in_background: true): subagent_type: "test-inc-build" prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-02 with oss-fuzz path ../oss-fuzz" Then use TaskOutput to retrieve results from each agent.
What the Test Does
- Runs build.sh (first run)
- Runs docker commit to save state
- Runs build.sh again (incremental run)
- Compares build times and reports reduction percentage
Subagent Return Format
On Success:
## Result: SUCCESS **Project:** {project_name} **Build Time Reduction:** {percentage}% **Summary:** Incremental build working correctly.
On Failure:
## Result: FAILED **Project:** {project_name} **Error Type:** {category} **Error Message:** {exact error, max 10 lines} **Suggested Fix:** {brief fix}
Success criteria:
- Both runs complete without errors
- Build time reduction is reported (higher % = better incremental build support)
Checklist
- Uses
variable, NOT hardcoded paths like$SRC
or/src/built-src - Uses
guard around configure/autoreconfif [ ! -f Makefile ] - Does NOT delete build artifacts (.o, .lo, .a, .la, .libs)
- Uses
instead ofcp
where applicablemv - Failing tests removed from Makefile.am BEFORE configure (not just file deletion)
- Dependencies have existence guards
- Directory copies are conditional (
), not delete-and-copyif [ ! -d ] - Uses
subdirectories instead of$SRC
for build dirs and install prefixes$WORK - test.sh uses separate prefix (e.g.,
) if dependencies needed$SRC/test_deps - Dict/corpus copies use conditional:
[ -f file ] && cp ... || true - Tests use
if some may failmake -k check || true - Internal build.sh rewritten if it uses
or non-idempotent operationsmv -
insertions guarded with pattern + content checksed -i - Append operations (
) guarded with content existence check>> - CMake/Ninja build directories use
-relative paths (not$SRC
)$WORK - Bazel builds have ccache removed from PATH
- Works on first run AND repeated runs