How to set a different thinking token budget per request with Gemini 2.5 Flash

When operating LLM APIs in production, there's a question you'll inevitably face at some point: "Does this request really need reasoning?" Honestly, I used to think that just leaving reasoning mode on would make everything work better. But it was only after opening the bill that my thinking changed. Seeing thinking tokens attached to a single translation API call with costs multiplied several times over is a clear signal that something is going wrong.

What makes Gemini 2.5 Flash interesting is that it doesn't treat reasoning as simply ON/OFF. A single parameter called thinkingBudget lets you control how deeply it thinks, measured in tokens. This enables a pattern where you use the same model endpoint for both simple classification tasks and complex mathematical reasoning simultaneously, tuning cost and quality differently just by changing the budget value.

This post examines those trade-offs from a production perspective: which settings work when, what pitfalls exist, how to handle monitoring, and most importantly, what criteria to use when setting per-task budgets.

Understanding What a Thinking Budget Actually Is

How Hidden Reasoning Works

Gemini 2.5 Flash can perform hidden reasoning internally before producing its final response. It's invisible to the user, but the tokens consumed in this process are billed in full. thinkingBudget is the parameter that sets the maximum number of tokens available for this internal reasoning.

The value range is 0 to 24576, with a special value of -1 (dynamic mode). The diagram below shows how the processing path diverges based on the budget value when a request arrives.

One key concept to keep in mind: the thinking budget is a ceiling, not a floor. Setting the budget to 8192 does not mean the model will necessarily use 8192 tokens. For simple prompts it may use far fewer, and you are only billed for what is actually consumed. Conversely, if the budget ceiling is hit, reasoning is cut off mid-way and the response is generated from that incomplete state — something to watch out for.

How to Check Cost Structure

Because the exact per-token rate for thinking tokens changes over time and across model versions, it is safer to check the Gemini API official pricing page directly. The important thing is that three separate rates exist: input tokens, regular output tokens, and thinking tokens. Because these three rates differ from each other, the total charge for the same request volume can vary significantly depending on how many of each token type were produced.

The best way to build intuition is to run the numbers yourself. For example, if a service sends one million requests per day and the average thinking tokens per request is 4,000, that comes to four billion thinking tokens per day. Multiplying by the official thinking token rate from the pricing page immediately tells you whether this line item needs its own budget allocation.

Setting Different Budgets Per Task Type

Code Structure (Conceptual Example)

The following is a conceptual example using the google-generativeai SDK. Because the SDK is evolving rapidly, you should check the documentation for your installed version to find where ThinkingConfig is exposed. As of August 2026, it is commonly found under the top-level genai.types namespace or under genai.protos; directly importing via the v1beta gRPC path tends to break with SDK updates.

import google.generativeai as genai
from google.generativeai import types as genai_types
 
model = genai.GenerativeModel("gemini-2.5-flash")
 
def call_with_budget(prompt: str, budget: int) -> str:
    response = model.generate_content(
        prompt,
        generation_config=genai.GenerationConfig(
            thinking_config=genai_types.ThinkingConfig(
                thinking_budget=budget
            )
        )
    )
    return response.text

Depending on the SDK version, a newer interface of the form genai.Client().models.generate_content(...) may be recommended. At project start, run pip show google-generativeai to check your installed version and follow the official examples for that version.

Routing by Task Type

A pattern commonly adopted in production is assigning different budgets to different task types while using the same model endpoint.

This routing does not require a separate classification model; it can be determined by where the request originates (which feature triggered the call) or simple rules.

BUDGET_MAP = {
    "translate": 0,
    "classify": 0,
    "moderate": 0,
    "summarize": 512,
    "qa_simple": 1024,
    "code_gen": 4096,
    "code_review": 8192,
    "math_solve": 16384,
    "realtime_chat": 1024,
}
 
def route_request(task_type: str, prompt: str) -> str:
    budget = BUDGET_MAP.get(task_type, 2048)
    return call_with_budget(prompt, budget)

The numbers themselves are just a starting point and should be adjusted to fit your service using the SLA back-calculation method described later.

Intuition for How Tasks Respond Differently

What published experiments consistently suggest is this: for tasks with short, deterministic answers — translation, classification, moderation — increasing reasoning rarely produces measurable accuracy gains while latency and cost grow. By contrast, for mathematics, multi-step reasoning, and complex code review, thinking tokens have a noticeable impact on the correctness rate of the final response. Because exact figures and multipliers vary by prompt, model version, and measurement methodology, running your own A/B tests with representative prompts from your service is the most reliable approach.

For math problems, when the model works through the logic internally, it can sometimes output only the key result in the final response without verbose explanation. Translation, on the other hand, gets no benefit from reasoning at all — no amount of thinking will produce a better translation of 'cat' than 'cat'.

Trade-offs: What You Gain and What You Lose

Common Pitfalls

Using dynamic mode (-1) in production as-is. Because the model decides the budget autonomously, it can overthink even simple tasks. The tendency to apply unnecessarily long reasoning to simple problems is also discussed academically (arXiv 2507.04023). Since cost becomes unpredictable, using a fixed value outside of development and testing is the safer choice.

Failing to monitor thinking tokens. Thinking tokens do not appear in the response text, yet they are billed. The field names under usageMetadata and which field aggregates thinking tokens have been changing across SDK and API versions; in recent versions the trend is toward a separate field such as thoughts_token_count. Always dump the actual response object to confirm which field captures the data, then log the budget setting, actual thinking tokens, and regular output tokens separately for each request.

import logging
 
def call_with_logging(task_type: str, prompt: str) -> str:
    budget = BUDGET_MAP.get(task_type, 2048)
    response = model.generate_content(
        prompt,
        generation_config=genai.GenerationConfig(
            thinking_config=genai_types.ThinkingConfig(
                thinking_budget=budget
            )
        )
    )
 
    usage = response.usage_metadata
    log_entry = {
        "task_type": task_type,
        "budget_set": budget,
        "prompt_tokens": getattr(usage, "prompt_token_count", None),
        "output_tokens": getattr(usage, "candidates_token_count", None),
        "thinking_tokens": getattr(usage, "thoughts_token_count", None),
    }
    logging.info(log_entry)
    return response.text

The reason for wrapping with getattr is that field availability differs across SDK versions. Before deploying to production, log the actual response schema once to confirm the field names.

Response quality when the budget ceiling is hit. When the budget limit is reached, the model stops reasoning and generates a response from that point. Incomplete reasoning can affect the quality of the final response. For very complex tasks with a budget set too low, a response produced from mid-cut reasoning can actually be worse than one produced with no reasoning at all.

In other words, for complex tasks the most dangerous zone is a budget that is "just slightly raised." It is safer to either use none at all (0) or give a clearly generous amount.

The Thought Process for Back-Calculating Budget from SLA

The key piece of advice this post wants to leave you with is this: don't assign numbers by gut feel per task type — define your quality SLA and latency SLA first, then back-calculate the budget from there. Approaching it roughly in this order keeps your intuition from drifting.

1. Write down the success criteria for each task type in plain language. Example: "The code review task must have a rule-violation detection rate of 90% or higher and p95 latency of 3 seconds or less."
2. Run 100–500 representative prompts at budget=0 to establish a baseline accuracy and latency.
3. Increase the budget on a log scale (0 → 512 → 2048 → 8192 → 24576) and measure accuracy, latency, and cost per request at each step.
4. Lock in the minimum budget at which the accuracy gain justifies the increase in latency and cost as the operational value for that task.
5. Re-run this measurement whenever traffic patterns change or prompts are significantly revised.

The core of this process is defining the budget value from the knee point of the SLA curve rather than from "task type." Even for code generation, the knee naturally shifts lower for internal tooling and higher for customer-facing use.

Closing Thoughts

The thinking budget is a parameter that, unlike most LLM API controls to date, genuinely hands the application layer the authority to decide how much to spend on reasoning per request. Applying the same reasoning depth to every request typically ends in either wasted cost or sacrificed quality.

A practical starting point is to begin every request at budget=0 and only raise the budget — using the back-calculation process above — for the tasks that fail to meet their SLA. Money already spent cannot be recovered. Treat dynamic mode (-1) as a development-phase tool for exploring how much reasoning a task needs, and always send a fixed value to production to keep your bill predictable.

Whether multi-model routing or budget routing is better depends on the situation. That said, it is well worth first checking whether you can simplify your operations matrix with a single model and a single budget axis.

References

• Gemini API Pricing - Google AI for Developers
• Start building with Gemini 2.5 Flash - Google Developers Blog
• Thinking | Firebase AI Logic - Google Official Docs
• The Overthinker's Guide to Reasoning Models (arXiv 2507.04023)
• Gemini 2.5 Flash API Pricing & Benchmarks | OpenRouter
• Gemini pricing in 2026 - CloudZero