o1 vs. 4o: 4 Key Insights I Learned (and You Should Know Too!)
4 Simple Yet Powerful Takeaways
One of the most important skills in AI today is knowing which model to use for which task — and how to prompt it effectively.
Choosing the wrong model can waste time and resources. For example, you wouldn’t want to wait for a complex chain-of-thought (CoT) reasoning process when a simpler approach would work just fine. On the flip side, you wouldn’t want to use a lightweight model like 4o for a visual reasoning task — it’s more likely to hallucinate.
At the 1.6-billion-dollar AI unicorn where I work, I see this play out every day. Our Slack channels are full of messages like:
I optimized the performance for Task X by switching from Model A to Model B.
In this article, we’ll explore some of the best practices for using o1 — when to use it, when not to, and how to get the most out of it.
4 Prompting Practices for o1
To get the best results from o1 models, you need to adapt your prompting strategy. Here are four key practices to follow:
Simple & direct: Write prompts that are straightforward and concise. Direct instructions yield the best results with the o1 models.
No explicit CoT required: You can skip step-by-step (CoT) reasoning prompts. The o1 models can infer and execute these itself without detailed breakdowns.
Structure: Break complex prompts into sections using delimiters like markdown, XML tags, or quotes. This structure enhances model accuracy — and simplifies your own troubleshooting.
Show rather than tell: Rather than using excessive explanation, give a contextual example to give the model understanding of the broad domain of your task.
🛑 Don’t:
Please write a short fantasy story about a young knight.
First, think step by step about why the knight is heroic and about the knight’s
personal struggles before you decide how to incorporate them into the story.
Also, thoroughly explain the kingdom’s background. Maybe mention how the knight’s
father was once a hero, and you can show that by writing his entire life story.
Step 1: Brainstorm.
Step 2: Outline.
Step 3: Draft.
Step 4: Revise.
Then finalize.
End with a big moral lesson explaining right from wrong.✅ Do:
**Story Brief**
- Title: The Emerald Sword
- Main Character: A young knight
- Setting: A mystical forest kingdom
- Style: Heroic and adventurous
**Task**
Write a short fantasy story in 3–4 paragraphs featuring:
1. The knight’s discovery of a legendary weapon.
2. A moment of moral choice that reveals the knight’s true character.
3. A conclusion hinting at a larger destiny yet to unfold.The o1 model is trained to generate a chain of thought (CoT) before answering, so you don’t need to create one yourself. The CoT generated by o1 is longer and of higher quality than what can be achieved through manual prompting. It includes behaviors such as:
Correcting errors
Trying multiple strategies
Breaking down problems into smaller steps
Key takeaway: You don’t need CoT prompting with o1!
Need a plan? Use o1!
One of o1’s key strengths is creating detailed plans to solve tasks. Once the plan is ready, you can pass it to 4o for execution — saving both time and costs.
Let’s say you have an API with several functions to manage a supply chain. Take a look at the sample prompt below for the o1 model:
You are a supply chain management assistant.
The first input you will receive is a scenario.
Your task is to review the scenario and create a detailed plan to provide a solution.
You will have access to an LLM agent responsible for executing tasks.
---
### The LLM agent has access to the following functions:
1. get_inventory_status(product_id)
- Retrieves the currently available stock of a product.
2. get_product_details(product_id)
- Retrieves the necessary components required to manufacture a product.
3. update_inventory(product_id, quantity_change)
- Updates the available inventory of a product.
- This function should be called after stock has been allocated.
4. fetch_new_orders()
- Checks the current status of new orders.
5. allocate_stock(order_id, product_id, quantity)
- Allocates the stock of a product to an order.
6. check_available_suppliers()
- Retrieves a list of available suppliers.
7. get_supplier_info(supplier_id)
- Returns the components a supplier can produce.
- This is necessary to obtain the required components before placing a purchase order.
8. place_purchase_order(supplier_id, component_id, quantity)
- Places a purchase order with a supplier for a specified component and quantity.
- To place a purchase order, you need to know the necessary components.
- Ensure the supplier has the required components available.
9. check_production_capacity(time_frame)
- Checks production capacity based on available components within a given time frame.
- If production capacity is insufficient, a purchase order is needed.
10. schedule_production_run(product_id, quantity, time_frame)
- Schedules a production run based on available production capacity.
- Any scheduled production run will reduce available capacity.
- The time frame values can match the production capacity options (e.g., "immediate").
11. calculate_shipping_options(destination, weight, dimensions)
- Determines available shipping options and costs based on the destination and product dimensions.
- Only currently available inventory can be shipped.
- The destination must match the order's destination name.
12. book_shipment(order_id, carrier_id, service_level)
- Books a shipment for a specific order.
13. send_order_update(customer_id, order_id, message)
- Sends an update to the customer about the order status.
- Keeping customers informed is crucial to ensure a good experience.
---
### Instructions for Creating a Plan:
- Main actions are numbered (e.g., 1, 2, 3).
- Sub-actions are lettered (e.g., a, b, c).
- Sub-actions should start on new lines.
- Specify conditions using clear "if...then...else" statements.
- For functions requiring input arguments:
- Ensure proper input arguments are given to the model.
- The final step should always involve calling the necessary functions to complete the task.
- Provide detailed steps when generating the plan, including each main step and sub-step.
Please find the scenario below:
We just received a major shipment of new orders.
Please generate a plan that gets the list of awaiting orders and determines the best policy to fulfill them.
The plan should include checking inventory, ordering necessary components from suppliers, scheduling production runs with available capacity, ordering new components required from suppliers, and arranging shipping to the retailer's distribution center in Los Angeles. Notify the customer before completing.
Prioritize getting any possible orders out that you can while placing orders for any backlog items.Try it yourself, and you’ll see that o1 excels at creating detailed execution plans — far better than 4o.
Once you have the plan from o1, you can pass it to 4o for execution using the following format:
You are a helpful assistant responsible for executing the policy on handling incoming orders.
Your task is to follow the policy exactly as it is written.
You must explain your decision-making process across various steps.
# Steps
1. Read and Understand Policy: Carefully read and fully understand the given policy on handling incoming orders.
2. Identify the exact step in the policy: Determine which step in the policy you are at, and execute the instructions according to the policy.
3. Decision Making: Briefly explain your actions and why you are performing them.
4. Action Execution: Perform the actions required by calling any relevant functions and input parameters.
POLICY:
{policy}Key takeaway: o1 is exceptional at generating plans to solve tasks — much better than GPT 4o!
Turn Images into Actionable Data with o1
Image reasoning with 4o can deliver good results. However, it often requires CoT prompting, few-shot examples, or fine-tuning to achieve the best performance.
Conversely, o1 performs well out of the box without the need for additional prompting or fine-tuning.
Let’s take a look at the organizational structure diagram below:
In cases like this, 4o typically struggles with spatial reasoning, leading to hallucinations. However, o1 handles these tasks much more effectively.
In real-world applications, here’s the recommended approach:
Use o1 to extract rich data (like JSON) from an image.
For follow-up questions, switch to text-based queries to avoid having o1 reprocess the image each time.
Try the prompt below to extract JSON from the org chart:
You are a consulting assistant who processes org data.
Extract the org hierarchy from the image you're provided in a structure.
The structure should be returned in JSON containing:
- arbitrary ID of the person that you can generate
- name of the person
- role of the person
- an array of IDs they report to
- an array of IDs that report to them Partial output:
[
{
"id": "1",
"name": "Juliana Silva",
"role": "CEO",
"reportsTo": [],
"directReports": ["2", "7", "11"]
},
{
"id": "2",
"name": "Kim Chun Hei",
"role": "CFO",
"reportsTo": ["1"],
"directReports": ["3"]
},
{
"id": "3",
"name": "Cahaya Dewi",
"role": "Manager",
"reportsTo": ["2"],
"directReports": ["4", "5"]
},
...
]You can now use the extracted JSON output to gather more insights about the image. For example, you can ask follow-up questions without needing to reprocess the image:
Key takeaway: Use o1 to extract rich data from an image for follow-up questions!
Make Dumb Models Smarter — Meta-Prompting with o1
One of the biggest challenges in today’s LLM applications is making them production-ready.
I’ve seen this challenge firsthand in my current role at Cresta, where we build Voice and Chat bots for Fortune 500 companies. Despite the latest advancements in techniques like prompt engineering, fine-tuning, and retrieval-augmented generation (RAG), ensuring reliable performance remains difficult.
One use case where o1 proves especially useful is optimizing prompts for less intelligent models.
Take a look at the architecture setup below:
We start with knowledge base articles designed for human consumption.
We feed them into o1 to generate a routine — for example, for a customer service chatbot.
We then evaluate the chatbot’s performance and generate a set of evaluation results.
Finally, we use the evaluation results, current routine, and knowledge base articles to improve the chatbot’s routine further.
Key takeaway: Use o1 to optimize prompts for less intelligent models!
💡 Want More Tools to Help You Grow?
I love sharing tools and insights that help others grow — both as engineers and as humans.
If you’re enjoying this post, here are two things you shouldn’t miss:
🔧 Medium Insights — A tool I built to explore the analytics behind any Medium author, helping you uncover what works and why.
📚 Everything I Learned About Life — A Notion doc where I reflect on lessons learned, practical tools I use daily, and content that keeps me growing.
👉 Find both (and more) here: Start Here