Artificial Intelligence thread

tphuang

tphuang

General
Staff member
Super Moderator
VIP Professional
Registered Member
https://twitter.com/i/web/status/2042404881508520157

my piece on the compute side of things. Looks like the domestic chips that were severely underutilized before are finally getting put to use. Inference was never the issue. The issue for Chinese AI labs is funding and also access to large enough training cluster (most of then like Nvidia). Not everyone is like ByteDance and have 500+ Blackwell racks in Malaysia for training.
 
Wrought

Wrought

Captain
Registered Member
Piece on the AI talent exodus from US -> China last year, driven by both push and pull factors.

For decades, the Valley served as a gravitational well for global tech ambition, a place where ideas met capital and engineering prowess. But the trajectory is starting to invert. Figures such as Wu Yonghui, who gave up a senior role at Google DeepMind to lead ByteDance’s push into next-generation large language models, and Yao Shunyu, who left OpenAI to anchor Tencent’s AI development, have returned in the past year. Other returnees include Roger Jiang, a senior scientist who left OpenAI to found his own robotics start-up in Shenzhen, and Zhou Hao, a researcher Alibaba poached from Google DeepMind to refine models. And three AI-focused headhunters based in China and San Francisco say they helped hire and relocate more than 30 US-based researchers to China in the past 12 months, versus a low single-digit a year earlier. This is not just a trickle of nostalgia — it is a calculated realignment. Macro and micro factors in China are creating a magnetic pull that is becoming more difficult to resist.
Click to expand...

Please, Log in or Register to view URLs content!
 
solarz

solarz

Brigadier
bsdnf said:
1M context window is a lie.

The reality is that the upper limit for all current models is 256k. Once you hit this limit, model performance will drops significantly, you have to go context compression.

Especially now that Opus suffers from severe computational power deduction, problems that could previously be solved by brute force using powerful benchmarking capabilities after exceeding the window limit are now exposed.

Can you imagine? It can't even solve a car wash problem now.
Click to expand...

At the end of the day, LLM response generation is O(n * k), where n is the context length, and k is the response length. This means the larger the context, the longer it takes to generate the next token. Therefore, barring a new algorithm or a revolutionary improvement over the current Attention-based Transformer architecture, LLMs are not really scalable for large tasks. The only way to use them effectively is to break the problem down into small pieces (which in Software Engineering is like 90% of the work), and that's where they'll be stuck for the foreseeable future when it comes to coding.
 
bsdnf

bsdnf

Senior Member
Registered Member
solarz said:
At the end of the day, LLM response generation is O(n * k), where n is the context length, and k is the response length. This means the larger the context, the longer it takes to generate the next token. Therefore, barring a new algorithm or a revolutionary improvement over the current Attention-based Transformer architecture, LLMs are not really scalable for large tasks. The only way to use them effectively is to break the problem down into small pieces (which in Software Engineering is like 90% of the work), and that's where they'll be stuck for the foreseeable future when it comes to coding.
Click to expand...
I'm not that pessimistic. A year ago, there weren't many models even worth using up to the 200k limit.
 
M

Michael90

Senior Member
Registered Member
Wrought said:
Piece on the AI talent exodus from US -> China last year, driven by both push and pull factors.



Please, Log in or Register to view URLs content!
Click to expand...
I think it’s moving more to a two way street movement now instead of the one way movement from china to the US like before . So some now leave while others remain or come to the US . Look at Alibabas top AI leaders who moved recently to meta. So its more of a two way things now not just one way unlike before, so china is improving but still a long way to go to match the US in attracting the best talents globally
 
You must log in or register to reply here.
Top