PromptForge

Agent-native context engineering and experimental harness.

PromptForge is a Python layer for transforming, compiling, validating, and experimentally evaluating the context sent to language models.

• Select relevant context for a task.
• Compile alternative structural representations.
• Construct prompts from compiled context.
• Normalize provider execution results.
• Run controlled and reproducible experiments.

An agent can inspect the task and context, select relevant fields, compile a representation, validate invariants, and execute or evaluate the resulting prompt.

Primary transformation surface: harness/runner/transforms.py

Task definitions: tasks/suite.json

Provider adapters: harness/providers/

Experimental contracts: harness/prereg/

Retained evidence: harness/reports/

Regression tests: tests/

Read AGENTS.md before making experimental or architectural changes.

The public release retains the V0.9.3 size-matched control: 4 context variants, 4 conditions, 15 repetitions per variant/condition, and 240 scheduled runs.

The primary comparison is pairs_compact versus noop_padded, where the control is padded to the same serialized context size.

V0.9.3 does not establish a universal economic advantage for one representation. Its purpose is to separate structural representation from serialized context size under the tested conditions.

Relevant files: harness/prereg/v093_size_matched_control.md harness/runner/v093_size_matched_control.py harness/reports/v093_size_matched_control_deepseek.json

python -m compileall -q harness tests

python -m unittest discover -s tests -p test_*.py

PromptForge makes context transformations explicit, testable, reproducible, and comparable. Historical results are evidence about the tested conditions, not universal guarantees across every task, model, provider, or prompting strategy.

PromptForge has two public surfaces:

• Agent core: provider-agnostic context preparation for real work.
• Research harness: reproducible historical experiments, providers, schedules, reports, and preregistration.

When an agent enters the repository, start here:

1. Read README.md and AGENTS.md.
2. Use harness.agent.prepare_context() for arbitrary task data.
3. Use harness.agent.prepare() only for the public deterministic fixtures.
4. Read historical runners/reports/providers only when the task actually requires them.

PromptForge has no runtime dependency on an LLM provider.

pip install .

Then:

from promptforge import prepare_context

prepared = prepare_context(
    data={
        "user": {
            "id": "U-7",
            "name": "Ada",
        },
        "task": {
            "action": "review",
            "commentary": "noise",
        },
    },
    required=["user.id", "task.action"],
)

print(prepared["serialized_context"])

The core is usable without OpenAI, DeepSeek, Gemini, API keys, network access, or provider configuration.

The handoff artifact contains:

• context: the selected structured context.
• serialized_context: compact JSON ready for downstream use.
• selected_arm: transformation selected.
• required_paths: semantic fields that must survive.
• included_paths / excluded_paths: field-level accounting.
• required_values_preserved / validation: preservation checks.
• estimated_tokens: dependency-free planning estimate.
• context_chars_saved / estimated_tokens_saved: reduction against the noop baseline.
• candidates: the alternatives considered.
• provenance: source and inclusion metadata for required paths.

Token estimates use UTF-8 byte length divided by four and rounded up. They are a planning heuristic, not an exact provider tokenizer measurement.

The default policy is deterministic minimal_serialized_context: evaluate the public deterministic transformation arms, validate preservation, and select the smallest serialized context.

For real context under a hard budget:

from promptforge import prepare_context, POLICY_BUDGET_CONSTRAINED

prepared = prepare_context(
    data=my_context,
    required=["task.id", "task.priority"],
    policy=POLICY_BUDGET_CONSTRAINED,
    budget_tokens=1200,
)

When budget_tokens is supplied with the default minimal policy, PromptForge automatically switches to budget_constrained.

The policy is a mechanical context-size rule. It is not a claim that the smallest context is universally better for model quality.

No validation dependency is required for the core schema contract:

prepared = prepare_context(
    data={
        "task": {"id": "T-17"},
        "score": 0.87,
    },
    required=["task.id"],
    schema={
        "task.id": str,
        "score": float,
    },
)

Schema paths are also included in the required set, so the validated fields cannot be silently dropped.

For planning without transformation:

from promptforge import inspect

report = inspect(
    context,
    required=["task.id", "task.priority"],
)

This reports field counts, required-field coverage, serialized size, byte size, and estimated tokens without calling any provider.

For deterministic repository fixtures:

from harness.agent import prepare

prepared = prepare("T003")

T003 demonstrates a reduction from 85 to 48 serialized characters (43.5%) while preserving its required fields. This is a small deterministic fixture, not a general token-savings or model-quality guarantee.

Provider adapters are experimental infrastructure. They are not required by the core agent API.

The public release retains the V0.9.3 size-matched control: 4 context variants, 4 conditions, 15 repetitions per variant/condition, and 240 scheduled runs.

V0.9.3 does not establish a universal economic advantage for one representation. Its purpose is to separate structural representation from serialized context size under the tested conditions.

Relevant files:

• harness/prereg/v093_size_matched_control.md
• harness/runner/v093_size_matched_control.py
• harness/reports/v093_size_matched_control_deepseek.json

PromptForge makes context transformations explicit, testable, reproducible, and comparable. Historical results are evidence about the tested conditions, not universal guarantees across every task, model, provider, or prompting strategy.

PromptForge is not a provider-specific SDK, a universal prompt optimizer, or a claim that one representation is always superior.

The core product is a provider-agnostic context-engineering layer. The research harness exists to make hypotheses, transformations, execution conditions, and historical evidence inspectable and reproducible.