It's not really better than R1. R1 was better with math.
Artificial Intelligence thread
- Thread starter 9dashline
- Start date
Qwen 3 is an efficient model, but it doesn't do great on factuality (SimpleQA).
By comparison, R1 gets around 30%. Qwen barely hits 11% at best and often below 10%. Gemini Flash 2.5 gets over 30% too. Gemini Pro gets 53%. I think Qwen's biggest strength is how efficient it is for its smaller sizes. But it's not as good as the leading models. Not even as good as the leading open source model (R1). And DeepSeek is likely to release a new model very shortly.
Qwen is far better than Llama, but Deepseek is still the open source king.
By comparison, R1 gets around 30%. Qwen barely hits 11% at best and often below 10%. Gemini Flash 2.5 gets over 30% too. Gemini Pro gets 53%. I think Qwen's biggest strength is how efficient it is for its smaller sizes. But it's not as good as the leading models. Not even as good as the leading open source model (R1). And DeepSeek is likely to release a new model very shortly.
Qwen is far better than Llama, but Deepseek is still the open source king.
What changed?
They provided a Korean-language privacy policy which informs users it will send data to Chinese servers and requires them to consent.
two pieces of news today
Deepseek Prover V2 is out, still waiting for more info here
Xiaomi has gotten into the open source release world with MiMo
if you look at its RL model, it's quite good
Deepseek Prover V2 is out, still waiting for more info here
Xiaomi has gotten into the open source release world with MiMo
if you look at its RL model, it's quite good
is an OpenAI dataset that is all but simple.
Hera are some random questions out of the :
These are far from simple....here simpleQA means that those are single fact questions, where the answer is just a couple of words.
Small models, without many hundreds/thousands of billions of parameters cannot perform well on this test, because all these little facts are stored in the model's parameters...and you need tons of them to learn all the small, single little facts that happened in the world at that level of detail.
This is a good test to indirectly get some hints on the model size, for instance, for closed models, like the OpenAI ones, if model A has a better result on SImpleQA of model B, then almost certainly model A is bigger than B.
This test does not measure how much a model is "smart" or good at instruction following, it does not measure how much a model is useful for day by day usage.
Wall Street/Financial bros get so PSTD with DeepSeek, that they report anything related to Whale