Rich Interactive Documentation Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Transform the static API reference into a living documentation system by adding rich, verifiable usage examples to all public methods and classes in toolkitx.

Architecture: - Use Google Style Examples: section in docstrings. - Use doctest syntax (>>>) for code verification. - Update makefile to include a target for running documentation tests.

Tech Stack: Python, pytest, doctest

---

Task 1: Enrich text_utils.py with Doctests

Files: - Modify: toolkitx/text_utils.py

• [ ] Step 1: Update truncate_text_smart docstring

Add examples for char mode, word mode, and tolerance.

def truncate_text_smart(
    text: str,
    limit: int = 100,
    mode: str = "char",
    suffix: str = "...",
    tolerance: int = 10,
) -> str:
    """
    ... (existing description) ...

    Examples:
        >>> truncate_text_smart("Hello World. This is a test.", limit=12)
        'Hello World...'
        >>> truncate_text_smart("Hello World. This is a test.", limit=15, mode="word")
        'Hello World. This is a test.'
        >>> truncate_text_smart("A very long sentence that should be truncated by word count.", limit=5, mode="word")
        'A very long sentence that...'
    """

• [ ] Step 2: Update split_text_by_word_count docstring

Add examples for splitting with and without overlap.

def split_text_by_word_count(
    text: str, max_words: int = 300, overlap: int = 0
) -> list[str]:
    """
    ... (existing description) ...

    Examples:
        >>> split_text_by_word_count("one two three four five", max_words=2)
        ['one two', 'three four', 'five']
        >>> split_text_by_word_count("one two three four five", max_words=3, overlap=1)
        ['one two three', 'three four five']
    """

• [ ] Step 3: Verify locally

Run: uv run pytest --doctest-modules toolkitx/text_utils.py Expected: PASS

• [ ] Step 4: Commit

git add toolkitx/text_utils.py
git commit -m "docs: add doctests to text_utils.py"

Task 2: Enrich html_utils.py with Doctests

Files: - Modify: toolkitx/html_utils.py

• [ ] Step 1: Update html_to_markdown docstring

Add examples for standard tables and nested tables.

def html_to_markdown(html: str, handle_nested_tables: str = "json", **kwargs) -> str:
    """
    ... (existing description) ...

    Examples:
        >>> html = "<table><tr><td>Cell 1</td><td>Cell 2</td></tr></table>"
        >>> print(html_to_markdown(html))
        |  |  |
        | --- | --- |
        | Cell 1 | Cell 2 |

        >>> nested_html = "<table><tr><td>Outer <table><tr><td>Inner</td></tr></table></td></tr></table>"
        >>> md = html_to_markdown(nested_html)
        >>> "Outer" in md and "Inner" in md
        True
    """

• [ ] Step 2: Verify locally

Run: uv run pytest --doctest-modules toolkitx/html_utils.py Expected: PASS

• [ ] Step 3: Commit

git add toolkitx/html_utils.py
git commit -m "docs: add doctests to html_utils.py"

Task 3: Enrich task_utils.py with Doctests

Files: - Modify: toolkitx/task_utils.py

• [ ] Step 1: Update TokenBucket docstring

class TokenBucket:
    """
    ... (existing description) ...

    Examples:
        >>> import time
        >>> bucket = TokenBucket(capacity=10, fill_rate=10)
        >>> bucket.consume(5)
        True
    """

• [ ] Step 2: Update with_resilience docstring

def with_resilience(...):
    """
    ... (existing description) ...

    Examples:
        >>> @with_resilience(qps=10)
        ... def my_func(x):
        ...     return x * 2
        >>> my_func(21)
        42
    """

• [ ] Step 3: Update PersistentTaskQueue docstring

Add a step-by-step example in the class docstring.

class PersistentTaskQueue:
    """
    ... (existing description) ...

    Examples:
        >>> import polars as pl
        >>> import tempfile
        >>> db = tempfile.NamedTemporaryFile(suffix=".db").name
        >>> queue = PersistentTaskQueue(db, "demo_task")
        >>> queue.setup()
        >>> df = pl.DataFrame({"batch_id": ["b1"], "input_text": ["hello"]})
        >>> queue.enqueue_dataframe(df)
        >>> queue.process(worker_func=lambda x: x.upper(), concurrency=1)
        >>> results = queue.get_results()
        >>> results["result"][0]
        'HELLO'
    """

• [ ] Step 4: Verify locally

Run: uv run pytest --doctest-modules toolkitx/task_utils.py Expected: PASS

• [ ] Step 5: Commit

git add toolkitx/task_utils.py
git commit -m "docs: add doctests to task_utils.py"

Task 4: Makefile Update and Final Verification

Files: - Modify: makefile

• [ ] Step 1: Add test-docs target to makefile

test-docs:
    uv run pytest --doctest-modules toolkitx

• [ ] Step 2: Run full verification

Run: make test-docs Expected: All doctests pass.

• [ ] Step 3: Build documentation to see results

Run: make docs-build Verify the site/ folder renders the code blocks correctly.

• [ ] Step 4: Commit

git add makefile
git commit -m "chore: add test-docs target to makefile"