Skip to main content
Iterative baking allows you to bake already-baked models. Improve specific behaviors, add new capabilities, or fix issues your previous bake didn’t capture.
Iterative baking builds on single-prompt and multi-target baking. Complete those first.

​
Why Iterative Baking?

Use iterative baking to:
  • Refine behavior: Modify behavior issues or improve specific responses
  • Add capabilities: Layer new skills onto existing knowledge
  • Specialize further: Make an expert model even more expert
  • A/B test variations: Create model versions to compare

​
How It Works

Train once from a pretrained model to create your first baked model.Qwen/Qwen3-32B → Bake Yoda personality → Yoda v1Single bake pass from pretrained base.

​
Example: Refining Yoda

​
Scenario

Your Yoda model (v1) works well but needs improvement:
  • Sometimes too verbose (want more cryptic responses)
  • Missing some Force-specific wisdom
  • Could be calmer in certain situations
Solution: Bake Yoda v1 with refined training data for Yoda v2

​
Complete Iterative Workflow

1

Identify Your Base Model

Use your previously baked model as the starting point:
from aibread import Bread

client = Bread()

# Baked model path, for example: "johndoe/yoda_repo/bake1/35"
previous_baked_model = "[username]/[repo]/[bake]/[checkpoint_num]"
Use a model from a previous bake in the repo
2

Create Refinement Prompts

Define prompts that address specific improvements:
client.repo.set(repo_name="yoda_v2")

# Teacher: More specific, refined guidance
client.prompts.set(
    prompt_name="yoda_teacher_prompt_2",
    repo_name="yoda_v2",
    messages=[{
        "role": "system",
        "content": """
        You are Yoda at peak wisdom. 
        - Speak in 1-2 short sentences maximum
        - Use inverted syntax consistently
        - Reference the Force in mystical, cryptic ways
        - Maintain absolute calm even when provoked
        """
    }]
)

# Student: Still empty for always-on behavior
client.prompts.set(
    prompt_name="empty_student",
    repo_name="yoda_v2",
    messages=[{"role": "system", "content": ""}]
)
3

Generate Targeted Training Data

Create training examples that target your improvements:
# Add stim prompts to ellicit more Yoda-like behavior
client.targets.set(
    target_name="yoda_target_2",
    repo_name="yoda_v2",
    template="default",
    overrides={
        "generators": [{
            "type": "hardcoded",
            "numq": 3,
            "questions": [
                "Explain the Force in detail",  # Was too verbose
                "You're wrong about the dark side!",  # Test calmness
                "Why should I trust you?"  # Needs mystical response
            ]
        },
        {
            "type": "oneshot_qs",  # LLM-generated stim prompts
            "numq": 400,
            "model": "claude-sonnet-4-5-20250929",
            "temperature": 0.6
        }],
        "model_name": previous_baked_model,  # Use Yoda v1 as base
        "teacher_prompt": "yoda_teacher_prompt_2",
        "student_prompt": "empty_student"
    }
)

# Generate training data
client.targets.stim.run(
    target_name="yoda_target_2",
    repo_name="yoda_v2"
)
client.targets.rollout.run(
    target_name="yoda_target_2",
    repo_name="yoda_v2"
)
Use your baked model (previous_baked_model) here, not the original base model. This ensures refinement builds on existing behavior.
4

Bake More Prompts

Bake additional prompts into your baked model:
bake = client.bakes.set(
    bake_name="yoda_bake_v2",
    repo_name="yoda_v2",
    template="default",
    overrides={
        "datasets": [
            {"target": "yoda_target_2", "weight": 1.0}
        ],
        "model": {
            "type": "bake",
            "parent_model_name": previous_baked_model,  # Start from Yoda v1
            "baked_adapter_config": {
                "r": 8,
                "lora_alpha": 16
            }
        }
    }
)

client.bakes.run(
    bake_name="yoda_bake_v2",
    repo_name="yoda_v2"
)

​
Iterative Baking Strategies

Make improvements over multiple iterations.V1 → Prompt to be calmer → V2 → Add mysticism prompt → V3 → Reduce verbosity → V4
Each iteration adds one specific improvement.

​
Key Differences vs. Initial Baking

AspectParallel BakingSequential Baking
Base modelPretrained (e.g., Qwen)Your baked model
Training dataComprehensive coverageFocused on improvements
GoalEstablish behavior/knowledgeImprove behavior/add more knowledge

​
Multi-Target Sequential Baking

Refine a model by adding multiple refinement targets: 
# Start with general Yoda V1
base = "yoda_v1"

# Refine with multiple aspects
targets = [
    {
        "name": "conciseness_refinement",
        "weight": 0.4,
        "focus": "Shorter responses"
    },
    {
        "name": "mysticism_refinement", 
        "weight": 0.3,
        "focus": "More Force references"
    },
    {
        "name": "calmness_refinement",
        "weight": 0.3,
        "focus": "Never agitated"
    }
]

# All targets use yoda_v1 as base
# Combined bake creates refined yoda_v2

​
Best Practices

Note which version you are baking on top of if testing multiple strategies.
Validate what works and what doesn’t with V1 before creating V2. Ensure you’re refining a solid base.
Make incremental improvements. It’s easier to debug and there’s less risk of breaking existing behavior.

​
When to Sequentially Bake

Don’t use sequential baking if:
  • ❌ Your base model has major flaws - start fresh instead
  • ❌ You want completely different behavior - bake from pretrained model
  • ❌ You’re testing a new prompt - start with new base for clean comparison
Do use sequential baking when:
  • ✅ Base model is good but needs polish
  • ✅ Adding complementary skills to existing behavior
  • ✅ Fixing specific edge cases

​
Complete Example: Three Iterations

from aibread import Bread

client = Bread()

# Iteration 1: Initial Yoda
# [Run full single-prompt bake workflow]
# Result: yoda_v1

# Iteration 2: Add conciseness
client.repo.set(repo_name="yoda_v2")
# [Create refinement target focused on brevity]
# Use yoda_v1 as base model
# Result: yoda_v2 (brief + Yoda)

# Iteration 3: Add mysticism
client.repo.set(repo_name="yoda_v3")
# [Create refinement target focused on Force wisdom]
# Use yoda_v2 as base model
# Result: yoda_v3 (brief + Yoda + mystical)

​
Next Steps

Production Patterns

Add monitoring to track model improvements

Bake Configuration

Fine-tune hyperparameters for refinement

Multi-Target Baking

Combine multiple refinements in one bake

Generators

Generate focused training data for refinements