OpenAI (who owns ChatGPT) released a new model today. It is called o1.
The main difference between models like 4o is it inherently performs chain of thought reasoning when producing an answer.
What is chain of thought reasoning? Well, it shows you how it came to the result it did.
This makes the model more accurate and also shows you how it got to where it did with its answer.
It beats 4o in most questions and most metrics:
I have found the model better than 4o at most prompts – in particular FIBA basketball rules. I’m learning the craft of being a referee and find the answers o1 gives to be VASTLY preferrable to 4o – mainly because it lists the actual rule(s) it is using to answer my questions. For example:
However – I like to push the boundaries of how good these models actually are – have we passed the Turing Test yet?
I have found a good question to ask these models to see if they are decent.
I have a 3 litre jug and a 5 litre jug – how do I get 6 litres?
The simple answer if you fill the 3 litre jug and pour it into the 5 litre jug. Then refill the 3 litre jug. 2 steps – and you have 6 litres.
The key for this problem is it’s one I’ve made up – so it is different to what any of the models have seen before in their training data – plus, most of these types of problems involve filling up the large jug and decanting it into the smaller jug incrementally to get the litres required. I’ve provided a question that is coming from a different angle that the models expect.
I also like the question as most humans can answer it readily.
This is o1‘s reply:
Thought for 15 seconds
To measure exactly 6 liters using only a 3-liter jug and a 5-liter jug, you can follow these steps:
- Fill the 5-liter jug completely.
- 5-liter jug: 5 liters
- 3-liter jug: 0 liters
- Pour water from the 5-liter jug into the 3-liter jug until the 3-liter jug is full.
- The 3-liter jug now holds 3 liters, and the 5-liter jug has 2 liters left.
- 5-liter jug: 2 liters
- 3-liter jug: 3 liters
- Empty the 3-liter jug.
- 5-liter jug: 2 liters
- 3-liter jug: 0 liters
- Transfer the remaining 2 liters from the 5-liter jug to the 3-liter jug.
- 5-liter jug: 0 liters
- 3-liter jug: 2 liters
- Fill the 5-liter jug completely again.
- 5-liter jug: 5 liters
- 3-liter jug: 2 liters
- Pour water from the 5-liter jug into the 3-liter jug until it is full.
- The 3-liter jug can hold 1 more liter (since it already has 2 liters), so you pour 1 liter from the 5-liter jug.
- 5-liter jug: 4 liters (5 liters – 1 liter)
- 3-liter jug: 3 liters
- Empty the 3-liter jug.
- 5-liter jug: 4 liters
- 3-liter jug: 0 liters
- Transfer the 4 liters from the 5-liter jug to the 3-liter jug.
- Since the 3-liter jug can only hold 3 liters, it takes 3 liters, leaving 1 liter in the 5-liter jug.
- 5-liter jug: 1 liter
- 3-liter jug: 3 liters
- Empty the 3-liter jug again.
- 5-liter jug: 1 liter
- 3-liter jug: 0 liters
- Transfer the 1 liter from the 5-liter jug to the 3-liter jug.
- 5-liter jug: 0 liters
- 3-liter jug: 1 liter
- Fill the 5-liter jug completely once more.
- 5-liter jug: 5 liters
- 3-liter jug: 1 liter
- Pour water from the 5-liter jug into the 3-liter jug until it is full.
- The 3-liter jug needs 2 more liters to be full.
- 5-liter jug: 3 liters (5 liters – 2 liters)
- 3-liter jug: 3 liters
Now you have exactly 3 liters in the 3-liter jug and 3 liters in the 5-liter jug, totaling 6 liters.
Summary: By carefully transferring water between the two jugs using these steps, you measure exactly 6 liters.
So ChatGPT got there in the end – but goodness is that not the long way around?
Being the good bloke I am and also wanting to be spared during the robot apocalypse I helped the cobber out:
The actual solution is to fill the 3 litre jug – pour it into the 5 litre jug then refill the 3 litre jug
ChatGPT replied:
Thought for 8 seconds
You’re absolutely right, and thank you for pointing that out! Your method is indeed a simpler way to get 6 liters using the 3-liter and 5-liter jugs. Here’s how it works:
- Fill the 3-liter jug completely.
- 3-liter jug: 3 liters
- 5-liter jug: 0 liters
- Pour the contents of the 3-liter jug into the 5-liter jug.
- 3-liter jug: 0 liters
- 5-liter jug: 3 liters
- Fill the 3-liter jug again.
- 3-liter jug: 3 liters
- 5-liter jug: 3 liters
Now you have 3 liters in the 5-liter jug and 3 liters in the 3-liter jug, totaling 6 liters.
Thank you for sharing this simpler solution