Building a CLI Tool with AI — From Idea to Published Package

I want to build a Python CLI tool called "linkcheck" that
checks markdown files for broken links.

Requirements:
- Scan one file or an entire directory recursively
- Check both internal links (relative paths) and external
  links (HTTP/HTTPS URLs)
- For external links, make HTTP HEAD requests to check
  if they return 200
- Support timeout and retry configuration
- Output results in a clean table format
- Return exit code 1 if any broken links are found
  (for CI/CD usage)
- Ignore patterns: skip links matching certain patterns
  (e.g., localhost, example.com)

Tech stack:
- Python 3.11+
- Click for CLI framework
- httpx for async HTTP requests
- Rich for terminal output

Do not implement anything yet. First, create the project
structure and a specification document.

linkcheck/
├── pyproject.toml
├── README.md
├── src/
│   └── linkcheck/
│       ├── __init__.py
│       ├── cli.py          # Click commands
│       ├── scanner.py      # Markdown parsing
│       ├── checker.py      # Link validation
│       └── reporter.py     # Output formatting
└── tests/
    ├── __init__.py
    ├── test_scanner.py
    ├── test_checker.py
    └── test_reporter.py

Implement src/linkcheck/scanner.py.

It should:
- Parse markdown files to find all links
- Extract both inline links [text](url) and
  reference links [text][ref]
- Return a list of LinkInfo objects with: url, line_number,
  source_file, link_text
- Handle relative paths by resolving them against the
  source file's directory
- Ignore code blocks (```...```) and inline code (`...`)

Use regex for parsing. Do not use a full markdown AST
parser — we want minimal dependencies.

Follow the specification from step 1.

import re
from dataclasses import dataclass
from pathlib import Path


@dataclass
class LinkInfo:
    url: str
    line_number: int
    source_file: Path
    link_text: str
    is_external: bool


# Patterns to match markdown links
INLINE_LINK = re.compile(r'\[([^\]]+)\]\(([^)]+)\)')
REFERENCE_LINK = re.compile(r'\[([^\]]+)\]\[([^\]]*)\]')
REFERENCE_DEF = re.compile(r'^\[([^\]]+)\]:\s*(.+)$', re.MULTILINE)
CODE_BLOCK = re.compile(r'```[\s\S]*?```')
INLINE_CODE = re.compile(r'`[^`]+`')


def scan_file(file_path: Path) -> list[LinkInfo]:
    """Extract all links from a markdown file."""
    content = file_path.read_text(encoding='utf-8')

    # Remove code blocks and inline code before scanning
    cleaned = CODE_BLOCK.sub('', content)
    cleaned = INLINE_CODE.sub('', cleaned)

    # Build reference map
    references: dict[str, str] = {}
    for match in REFERENCE_DEF.finditer(content):
        references[match.group(1).lower()] = match.group(2).strip()

    links: list[LinkInfo] = []
    lines = cleaned.split('\n')

    for line_num, line in enumerate(lines, start=1):
        # Find inline links
        for match in INLINE_LINK.finditer(line):
            url = match.group(2).strip()
            links.append(LinkInfo(
                url=url,
                line_number=line_num,
                source_file=file_path,
                link_text=match.group(1),
                is_external=url.startswith(('http://', 'https://')),
            ))

        # Find reference links
        for match in REFERENCE_LINK.finditer(line):
            ref_key = (match.group(2) or match.group(1)).lower()
            if ref_key in references:
                url = references[ref_key]
                links.append(LinkInfo(
                    url=url,
                    line_number=line_num,
                    source_file=file_path,
                    link_text=match.group(1),
                    is_external=url.startswith(('http://', 'https://')),
                ))

    return links


def scan_directory(dir_path: Path, recursive: bool = True) -> list[LinkInfo]:
    """Scan all markdown files in a directory."""
    pattern = '**/*.md' if recursive else '*.md'
    links: list[LinkInfo] = []

    for md_file in sorted(dir_path.glob(pattern)):
        links.extend(scan_file(md_file))

    return links

The line numbers are wrong. When you remove code blocks
from the content, the line count changes. You need to
track line numbers from the original content, not the
cleaned content.

Fix this by scanning the original lines but skipping
lines that are inside code blocks.

Implement src/linkcheck/checker.py.

It should:
- Check external links with async HTTP HEAD requests
- Check internal links by verifying the file exists
- Support configurable timeout (default 10 seconds)
- Support configurable retry count (default 2)
- Support concurrent requests (max 10 parallel)
- Return CheckResult for each link: url, status (ok/broken/
  timeout/skipped), status_code (for HTTP), error_message

Use httpx.AsyncClient for HTTP requests.
Use asyncio.Semaphore to limit concurrency.

Handle these edge cases:
- Some servers block HEAD requests — fall back to GET
- Some servers return 403 for automated requests —
  use a browser-like User-Agent
- Redirect chains — follow up to 5 redirects
- SSL certificate errors — report but do not crash

async def check_external_link(
    client: httpx.AsyncClient,
    link: LinkInfo,
    semaphore: asyncio.Semaphore,
    config: CheckConfig,
) -> CheckResult:
    """Check a single external link."""
    async with semaphore:
        for attempt in range(config.retries + 1):
            try:
                # Try HEAD first (faster)
                response = await client.head(
                    link.url,
                    timeout=config.timeout,
                    follow_redirects=True,
                )

                # Some servers block HEAD, fall back to GET
                if response.status_code == 405:
                    response = await client.get(
                        link.url,
                        timeout=config.timeout,
                        follow_redirects=True,
                    )

                if response.status_code < 400:
                    return CheckResult(
                        link=link,
                        status=Status.OK,
                        status_code=response.status_code,
                    )

                return CheckResult(
                    link=link,
                    status=Status.BROKEN,
                    status_code=response.status_code,
                    error=f"HTTP {response.status_code}",
                )

            except httpx.TimeoutException:
                if attempt < config.retries:
                    continue
                return CheckResult(
                    link=link,
                    status=Status.TIMEOUT,
                    error="Request timed out",
                )

            except httpx.ConnectError as e:
                return CheckResult(
                    link=link,
                    status=Status.BROKEN,
                    error=f"Connection failed: {e}",
                )

Implement src/linkcheck/cli.py and src/linkcheck/reporter.py.

cli.py:
- Use Click for the CLI interface
- Main command: linkcheck <path>
- Options: --timeout, --retries, --concurrent, --ignore-pattern
  (can be specified multiple times), --format (table/json)
- Return exit code 0 if all links are OK, 1 if any are broken

reporter.py:
- Table format using Rich: show file, line, link text,
  URL (truncated to 60 chars), and status with color
  (green for OK, red for broken, yellow for timeout)
- JSON format: output a JSON array of results
- Summary line: "Checked X links: Y OK, Z broken, W timeout"

Write tests for all three modules.

For scanner tests:
- Create test markdown files as fixtures
- Test inline links, reference links, links in code blocks
  (should be ignored), and links with titles
- Test directory scanning with nested directories

For checker tests:
- Mock HTTP responses with httpx mock
- Test: successful links, 404 links, timeouts, retries,
  HEAD-to-GET fallback, SSL errors
- Test internal link checking with temporary files

For CLI tests:
- Use Click's CliRunner
- Test: single file, directory, broken links (exit code 1),
  ignore patterns, JSON output

Use pytest with pytest-asyncio for async tests.

$ pytest -v
========================= test session starts =========================
tests/test_scanner.py::test_inline_links PASSED
tests/test_scanner.py::test_reference_links PASSED
tests/test_scanner.py::test_ignores_code_blocks PASSED
tests/test_scanner.py::test_links_with_titles PASSED
tests/test_scanner.py::test_directory_scan PASSED
tests/test_checker.py::test_successful_link PASSED
tests/test_checker.py::test_broken_link_404 PASSED
tests/test_checker.py::test_timeout_with_retry PASSED
tests/test_checker.py::test_head_to_get_fallback PASSED
tests/test_checker.py::test_ssl_error PASSED
tests/test_checker.py::test_internal_link_exists PASSED
tests/test_checker.py::test_internal_link_missing PASSED
tests/test_cli.py::test_single_file PASSED
tests/test_cli.py::test_directory PASSED
tests/test_cli.py::test_broken_link_exit_code PASSED
tests/test_cli.py::test_ignore_pattern PASSED
tests/test_cli.py::test_json_output PASSED
========================= 17 passed in 2.34s ==========================

Generate a README.md for the linkcheck project.

Include:
- One-line description
- Installation (pip install linkcheck)
- Quick start with 3 examples
- Full option reference
- Configuration file support (.linkcheckrc)
- CI/CD integration example (GitHub Actions)
- Contributing section

Keep it concise. Users should find what they need
in under 30 seconds.

Update pyproject.toml for PyPI publishing.

Include:
- Package name: linkcheck-cli (linkcheck is taken)
- Version: 0.1.0
- Description, author, license (MIT)
- Python 3.11+ requirement
- All dependencies with version pins
- Entry point: linkcheck = linkcheck.cli:main
- Classifiers for PyPI

Then show me the commands to build and publish.

# Build the package
python -m build

# Upload to PyPI (test first)
twine upload --repository testpypi dist/*

# Upload to production PyPI
twine upload dist/*