Awesome-omni-skill crawl-docs-skill

Run a Crawl4AI-based doc crawler and save internal pages as Markdown using page titles as filenames. Use when the user provides a docs URL and wants all internal subpages saved as .md files. Environment setup should only use uv.

install

source · Clone the upstream repo

git clone https://github.com/diegosouzapw/awesome-omni-skill

Claude Code · Install into ~/.claude/skills/

T=$(mktemp -d) && git clone --depth=1 https://github.com/diegosouzapw/awesome-omni-skill "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/documentation/crawl-docs-skill" ~/.claude/skills/diegosouzapw-awesome-omni-skill-crawl-docs-skill && rm -rf "$T"

manifest: skills/documentation/crawl-docs-skill/SKILL.md

source content

Crawl Docs Skill

One-shot (Direct Run)

Run directly:

bash "<path-to-skill>/scripts/crawl_docs.sh" <URL> --out "$PWD/docs"

Do NOT install dependencies or scaffold files first; run immediately. The script will bootstrap uv + venv only if needed.
Do NOT run tests or create plans; just execute the script and let it fail if the system is missing prerequisites (e.g. curl).
Default output dir (skill convention):
```
"$PWD/docs"
```
(absolute path to
```
./docs
```
under the current working directory); always pass it via
```
--out
```
unless the user specifies otherwise.
Optional flags:
```
--out <dir>
```
,
```
--max-pages <int>
```

Correct Usage (from real run)

First run: execute the one-shot command exactly once with no extra steps.
If it fails: fix the missing system prerequisite (most commonly
```
curl
```
) and re-run the same command.
Do not probe versions or add max-pages unless the user asks.

Notes

Use page title (tab name) as the output filename; auto-deduplicate with numeric suffixes.
Default behavior has no max pages unless
```
--max-pages
```
is provided.