Chain-of-Thought Prompting: How It Works, With Examples

# Chain-of-Thought Prompting: How It Works, With Examples

Chain of thought prompting is the habit of asking a model to show its work. Instead of jumping to a final answer, a large language model writes out the intermediate steps that lead there: a coherent series of logical steps you can read and check. When it was first tested in 2022, this simple prompt engineering method cut wrong answers on maths and logic questions. It was a significant advance in the reasoning capabilities of artificial intelligence, and it still shapes how we prompt today, even as reasoning models do more of the thinking themselves.

This guide covers what CoT is, why it works, its main variants, five worked scenarios with plug-and-play templates, and whether you still need it in late 2026. It ends with a small test we ran on a current model, with real outputs, so you can see how much control CoT gives you over the responses you get back.

What is chain-of-thought prompting?

Chain of thought prompting definition: chain-of-thought prompting (CoT prompting) is a prompt engineering technique that asks a large language model to produce a logical sequence of steps, its intermediate reasoning steps, before the final answer. The output is an answer plus reasoning. A standard few-shot prompt shows examples with answers only; a CoT prompt shows the steps too.

Humans solve a maths expression like 3+4+19-12 one operation at a time: 3 plus 4 is 7, plus 19 is 26, minus 12 is 14. Each calculation step is small, and small steps make the result reliable. CoT asks the model to break problems down the same way, mirroring human problem-solving and human reasoning.

The simplest version is a banana word problem, the kind of maths problem CoT handles well:

Prompt: Solve the following problem step by step. If you have 7 bananas, give 4 to your friend, receive 5 more, then throw out 3 rotten bananas, how many do you have left?

Output: 1. You start with 7. 2. Giving away 4 leaves 3. 3. Receiving 5 makes 8. 4. Throwing out 3 leaves 5. You have 5 bananas left.

Those four numbered steps are the chain of thought. In plain prompting, the model would just say "5". Short cues such as "describe your reasoning steps" or "explain your answer step by step" trigger the same behaviour in ChatGPT, Gemini and other LLMs.

How does chain of thought prompting work?

You get a reasoning chain in one of two ways. Instructions only: add "Solve this problem step by step" and the model writes its own step sequence. Worked examples (exemplar prompts): show a solved arithmetic problem, then ask a new one, and the model copies the pattern for new tasks:

Problem: What is the value of 3+4+19-12?
Solution: 3+4 is 7. 7+19 is 26. 26-12 is 14. The final answer is 14.

Problem: What is the value of 5+7+9-12?
Solution:

The model continues "5+7 is 12, 12+9 is 21, 21-12 is 9", so the answer is 9. (Some popular tutorials print "3+4 is 12" in this very example. Check worked examples before you reuse them, because a wrong exemplar teaches a wrong step.)

The pattern scales. Ask a model to solve the quadratic equation x² - 5x + 6 = 0 step by step and it factors it into (x - 2)(x - 3), sets each factor to zero, and reports x = 3 and x = 2, with every step open to checking.

Why chain of thought prompting works (and where it hurts)

A model generates text one token at a time. When it writes intermediate steps, each new step is produced with the earlier ones already on the page. CoT gives the model more computation per problem and a scratchpad. Complex tasks become simpler steps, and each step gets more attention.

The four benefits

CoT also makes a model easier to debug. When the answer is wrong, the chain lets you pinpoint faulty step numbers and ask the model to fix only that step. Its attention to detail is why step-by-step explanations are popular in education.

Ask "What colour is the sky?" and standard prompting replies "the sky is blue". Ask why, step by step, and the model walks through sunlight, atmospheric scattering and how eyes perceive colour: longer, but more correct, more accurate and easier to verify.

Where CoT hurts: the limitations

• Model size. In the original paper, gains appeared only at large model scale, around ~100B parameters. Smaller models produced coherent but illogical chains and did worse than standard prompting. Instruction tuning has since narrowed that gap.
• Faithfulness: reasoning ≠ answer. The stated chain is not always how the model actually reached its answer. Plausible but incorrect reasoning paths can mislead you, so check both the steps and the answer. Faithful CoT (below) is one fix, and our test at the end caught this in the wild.
• Prompt design effort and overfitting. Writing CoT prompts with good examples is labour-intensive and takes time; high-quality prompts matter. Models can overfit to the reasoning style or example types you show. Auto-CoT and analogical prompting cut this work.
• Cost. A long output means higher computational cost and latency.
• Not suitable for every task. Pattern recognition, creative writing and simple factual answers rarely gain. It is also hard to evaluate qualitative improvements in reasoning, so measure accuracy on real examples instead of judging how convincing the steps sound.

Zero-shot CoT ("Let's think step by step") vs few-shot CoT

CoT has two main types plus variants. For background on example-based prompting, see our guide to zero-shot vs few-shot prompting.

Zero-shot chain of thought

Zero-shot CoT is the simplest form: instructions only, no examples. A reasoning cue makes the model use its built-in knowledge on novel problems. The classic phrase, and the most tested variation, is "Let's think step by step" (Kojima et al., 2022). Other tested cues ask the model to work it out step by step to be sure of the right answer, or to think logically first.

Ask "What is the capital of a country that borders France and has a red-and-white flag? Let's think step by step" and the model checks France's neighbours and their flags, then reasons to Switzerland. Plain zero-shot prompting such as "Translate 'My name is' into Finnish" has no steps, so the cue adds nothing. The weakness: steps look right but may not make sense. When that happens, move to few-shot.

Few-shot chain of thought

Few-shot CoT combines few-shot prompting with reasoning examples, and it combines well with any task that has repeatable steps. Show a few solved examples that match your question type, then ask. The original paper used maths word problem examples:

Q: There are 15 trees in the grove. After workers plant more, there are 21.
   How many did they plant?
A: There were 15, then 21. So 21 - 15 = 6. The answer is 6.

Q: Shawn has five toys. He got two each from his mom and dad. How many now?
A:

Similar sets cover trees in a grove, cars in a lot, chocolates shared between sisters and lollipops given away. Published studies show few-shot CoT outperforms zero-shot CoT, with reported gains of up to 28.2% on some tasks.

Not all few-shot prompts are CoT prompts. A sentiment-classification example ("The movie was good // positive") is few-shot without CoT, and so is a business-introduction example listing Mike (engineer), Jen (recruiter) and Ben (investor) with sample opening lines. Add reasoning and they become few-shot CoT. The cost is manual work: hand-writing examples for every question type is slow.

Automatic CoT (Auto-CoT)

Auto-CoT automates the examples. Its key insight is that diverse examples matter more than perfect ones. It runs in two steps (some tutorials list four steps):

1. Question clustering: encode a varied question dataset with sentence transformers and cluster it by cosine similarity.
2. Demonstration sampling: take one or two questions per cluster and generate a zero-shot CoT chain for each. These auto-generated demonstrations go in the prompt.

Across 10 benchmarks, the benchmark ranking was Auto-CoT, then Manual-CoT, then zero-shot CoT. Generating reasoning paths and choosing reasoning paths automatically also improves scalability: "I have 3 apples and buy 5 apples" gets the chain "start with 3, add 5, total 8" with no human input. Many modern LLMs do this by default, and pre-prompting with "Provide step by step reasoning" mimics it.

Other CoT variants worth knowing

Designing the CoT prompt: chain-of-thought best practices

Vague prompts produce vague chains. Ask the model to explain before concluding, tell it to break into parts on multi-step problems, and ask it to justify its logic, which reduces hallucinations because unsupported jumps become visible. Give a worked example when format matters.

Two examples: a sky-colour explanation ("Explain why the sky appears blue, from how sunlight meets the atmosphere to how the eye perceives colour"), and "Is 153 prime? Explain your reasoning by testing divisibility rules step by step." (It isn't: 153 = 9 × 17.)

Useful CoT phrases: "Explain why...", "Explain your reasoning" or "State your reasoning", "First determine ___ then identify ___", "Follow these steps before you ___", and "If ___ then ___ else ___".

If you use CoT across a product, put these instructions in the system prompt instead of every message. See what a system prompt is for CoT in system prompts.

Where it comes from: Wei et al. 2022

Jason Wei and colleagues at Google named and tested the technique in "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (2022, arXiv 2201.11903). The original paper added a chain of thought to each few-shot exemplar and found sharp gains in arithmetic, commonsense and symbolic reasoning in very large models. Two findings still matter: CoT is an emergent ability that appears only as model scale grows, and it is cheap, needing no fine-tuning.

Later work built on it: zero-shot CoT (Kojima et al., 2022), Auto-CoT (Zhang et al., 2022) and self-consistency (Wang et al., 2022). Instruction tuning brought CoT to smaller models, and generative AI built on transformers made step-by-step reasoning a default behaviour.

Training followed prompting. To build DeepSeek-R1, DeepSeek first trained a precursor, R1-Zero, with a template asking the model to put its reasoning inside one set of tags and its answer inside another, then rewarded correct answers. That training produced CoT sequences at scale, and chain of thought moved from the prompt into the model.

Advances in CoT also reached symbolic reasoning (explicit rules, as in 2+3=5) and logical reasoning (deduction from premises). A widely shared example: from "All birds can fly, and a penguin is a bird", a model concludes a penguin can fly. The deduction is valid, but the false premise gives a false conclusion. Visible reasoning is how you catch it in this penguin syllogism. CoT is also used for creativity tasks such as structured brainstorming.

Related reasoning techniques: self-consistency and tree of thoughts

Self-consistency prompting makes the model run several times on the same CoT prompt and picks the most consistent answer by majority vote. One-off reasoning errors get outvoted, so conclusions are more reliable and more often logically sound. It earns its cost on single-answer questions where errors are expensive; 3 to 5 runs is usually enough.

Tree of thoughts prompting (Yao et al., 2023) treats reasoning as a search: at each step the model proposes several next thoughts, scores them, keeps the best and backtracks from dead ends. It earns its cost only on planning, puzzles and search problems where CoT and self-consistency both fail.

Chain of thought vs tree of thoughts (and AutoReason)

The AutoReason result generalises: a two-step framework that adds reasoning helps on harder problems and can hurt on easy ones.

Prompt chaining vs chain of thought

Prompt chaining splits a task across several prompts; CoT keeps the reasoning in a single prompt. Both give step-by-step reasoning and both are debuggable. Use CoT when one response can hold the chain, and prompt chaining when steps need different instructions or checks in between. Our upcoming guide covers prompt chaining vs chain of thought in depth.

Chain of thought prompting examples: five worked scenarios

These chain of thought prompting examples name the steps you want, which beats a generic "think step by step".

1. Math word problem

Pens cost $3.00 for a pack of 12 or $0.40 each. I need exactly 40.
Step by step: list the pack/single combinations, cost each, then state the cheapest.

Expected chain: 3 packs plus 4 singles costs $10.60; 4 packs costs $12.00; 2 packs plus 16 singles costs $12.40. Cheapest: $10.60. Mathematical reasoning is the classic use case.

2. Multi-step business decision

Contractor: $6,000/month, starts now. Full-time hire: $8,500/month, starts in 8 weeks.
Project: 6 months. First determine each option's total cost, then identify which
gives more working weeks. State your reasoning, then recommend one in a sentence.

The same structure handles complex multi-part questions, such as a cold marketing email built on a SNAP Product Strategy evaluation: analysis step first, writing step second.

3. Debugging

def average(nums): return sum(nums) / len(nums)   # crashes on []
Start at the error, step back through the code, explain where the bug
originates, then give the fix.

For software coding and debugging, trace backwards from the symptom. The same pattern helps with finding vulnerabilities and patching vulnerabilities.

4. Data analysis

Sign-ups: Jan 1,200, Feb 1,150, Mar 1,900, Apr 1,880. Why the March jump?
Follow these steps before you answer: compute each monthly change, flag the
unusual one, list three causes and the data that would confirm each.

5. Legal or policy reasoning

Policy: refunds within 30 days, unless the item is digital and downloaded.
A customer bought an e-book 12 days ago and downloaded it. Do they qualify?
If the item is digital and downloaded then name the rule that applies, else
apply the 30-day rule. Explain your reasoning, then answer yes or no.

For regulated decisions, CoT supports AI ethics and decision-making transparency because the trail can be reviewed. Treat the output as a draft for a human, not advice.

Where else CoT is used

Implementing CoT in code (LangChain)

Implementing CoT is a template change; the implementation takes minutes. This code walkthrough wraps the question in a LangChain PromptTemplate and sends it to Gemini Pro:

cot = PromptTemplate.from_template("Solve this problem step by step.\nQuestion: {query}")
llm.invoke(cot.format(query="What is the value of 5+7+9-12?"))

Next, try Auto-CoT on three or four question types. For more structures, see AI prompt frameworks and how to write better ChatGPT prompts.

Is CoT still useful with reasoning models?

Honest answer: less than it used to be. Reasoning models such as OpenAI's o-series (o1, o1-mini and later) and Claude with extended thinking reason internally before answering, so you don't need to ask. The research shows a clear pattern:

• Task complexity decides the value. In a 2024 software engineering study (Wang et al., arXiv 2411.02093), o1-mini beat GPT-4o by 16.67% on problems needing 5+ steps of reasoning, but by only 2.89% under 5 steps. When o1-mini underperforms, 24% of those losses came from over-reasoning simple tasks with fewer than 3 steps.
• "Think step by step" can hurt performance on simple tasks for a reasoning model.
• More reasoning steps still help on hard tasks. In the Medprompt study (Nori et al., arXiv 2411.03590), an extended-reasoning template telling o1 to take its time beat a quick template demanding speed, with higher accuracy as reasoning tokens grew. DeepSeek reported the same link for R1.
• Few-shot can backfire. The same study found five-shot prompting caused a significant drop in o1's accuracy.
• Watch for reasoning leak into outputs, which then need post-processing when you want clean code or JSON.

Our test: the same task with and without CoT

We ran one scheduling question six times on Claude Haiku 5.5 on 30 October 2026: three runs with no CoT ("reply with only the weekday") and three with "Let's think step by step".

Task: Today is Wednesday. A project needs 17 working days, Monday to Friday, starting tomorrow. The first Monday after work starts is a public holiday. On which weekday is the last working day? Correct answer: Monday.

What changed: accuracy did not; all six answers were right. What changed was what we could verify. Run 6 invented dates (31 October 2026 is a Saturday) yet still landed on the right weekday. Without visible reasoning we would never have seen that the model's calendar was wrong. That is the faithfulness problem, caught in the wild.

What still helps with reasoning models

• Clear success criteria: say what a good answer must include.
• Structure for the output, not the thinking: a table, a numbered list or JSON.
• One example for format, not five.
• A verification step: "Check your answer against the constraints before you reply."
• Visible reasoning when you need an audit trail for decisions someone must review.

On standard, non-reasoning models, classic CoT remains one of the most reliable ways to improve multistep answers.

FAQ

What is chain-of-thought prompting?

Chain-of-thought prompting is a prompt engineering technique that asks a large language model to write out its intermediate reasoning steps before giving a final answer. Showing the steps tends to improve accuracy on multistep problems and makes the model's reasoning easier to check.

What is an example of chain-of-thought prompting?

Add one line to a word problem: "Let's think step by step." For "You have 7 bananas, give away 4, get 5 more and throw out 3. How many are left?", the model then writes 7 minus 4 is 3, 3 plus 5 is 8, 8 minus 3 is 5, so the answer is 5.

Who introduced chain-of-thought prompting?

Jason Wei and colleagues at Google introduced it in the 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (arXiv 2201.11903). That paper is the origin of the name and the standard definition.

Why does chain-of-thought prompting work?

Writing out intermediate steps breaks a complex task into smaller pieces, so each step is generated with the earlier steps already on the page. This gives the model more computation per problem and gives you a visible trail to check.

What is zero-shot CoT ("Let's think step by step")?

Zero-shot CoT adds a reasoning cue such as "Let's think step by step" to a prompt without any worked examples. It was described by Kojima et al. in 2022 and is the simplest way to apply chain of thought. Few-shot CoT, by contrast, adds solved examples.

What is the difference between chain of thought and tree of thoughts?

Chain of thought follows one line of reasoning from start to finish. Tree of thoughts explores several possible next steps at each stage, scores them, and can backtrack, which costs many more model calls but helps on search and planning problems.

Is CoT still useful with reasoning models?

Less than before. Reasoning models already think internally, so a manual "think step by step" adds little and can hurt on simple tasks. Clear success criteria, output structure, a request to verify the answer and visible reasoning when you need an audit trail still help.

Key takeaways

Chain of thought prompting is simple, powerful and versatile: a few words or a few worked examples, auto-generated with Auto-CoT, or swapped for variants like Step-Back. It was a significant step for LLM reasoning, a leap in AI reasoning, and the ongoing work on reasoning models now builds it into the model. Your job has shifted from asking for steps to deciding when steps are worth reading. Browse our prompt frameworks hub for techniques that pair well with CoT.

Want CoT built in? Describe your task in our free prompt generator and it adds reasoning steps, examples and output structure for you.

References

1. Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903.
2. Kojima, T. et al. (2022). Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916.
3. Zhang, Z. et al. (2022). Automatic Chain of Thought Prompting in Large Language Models. arXiv:2210.03493.
4. Wang, X. et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171.
5. Yao, S. et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv:2305.10601.
6. Wang, G. et al. (2024). Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering? arXiv:2411.02093.
7. Nori, H. et al. (2024). From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond. arXiv:2411.03590.
8. DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.