elitemadzone.org - AI: pomoć

Ivan Dimkovic

Administrator
Član broj: 13
Poruke: 16753
*.mobile.kpn.net.

+7204 Profil

^{23.12.2024. u 05:52 - pre 4 meseca}

Citat:

In short, when comparing Nvidia’s GPUs to AMD’s MI300X, we found that the potential on paper advantage of the MI300X was not realized due to a lack within AMD public release software stack and the lack of testing from AMD.

AMD’s software experience is riddled with bugs rendering out of the box training with AMD is impossible. We were hopeful that AMD could emerge as a strong competitor to NVIDIA in training workloads, but, as of today, this is unfortunately not the case. The CUDA moat has yet to be crossed by AMD due to AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience. As fast as AMD tries to fill in the CUDA moat, NVIDIA engineers are working overtime to deepen said moat with new features, libraries, and performance updates.

Fantasticno je da AMD posle N godina i dalje nije u stanju da posveti dovoljno paznje softveru.

DigiCortex (ex. SpikeFun) - Cortical Neural Network Simulator:
http://www.digicortex.net/node/1 Videos: http://www.digicortex.net/node/17 Gallery: http://www.digicortex.net/node/25
PowerMonkey - Redyce CPU Power Waste and gain performance! - https://github.com/psyq321/PowerMonkey

Odgovor na temu

speculaas

Član broj: 337011
Poruke: 4

+1 Profil

Re: AI: pomoć - ili opasnost?

^{23.12.2024. u 11:23 - pre 4 meseca}

To je jedna od izuzetno zbunjujucih stvari za mene. Mogu da dovedu koga god hoce da sprovede taj projekat, toliko je ocigledno kakav efekat bi to imalo da bi ljudi pristali da budu placeni samo u opcijama...

Ranije ove godine je bilo pokusaja spolja, geohot je pokrenuo tinybox sa misijom "commoditizing the petaflop" i isao je all-in na amd akceleratore (6x7900XTX). Nakon gomile frustracija je morao da se prebaci na RTX4090 (uz "malu" korekciju cene, razume se...). Bilo je nekih pokusaja eskalacije AMD menadzmentu, ali na kraju nije bilo nista od toga: AMD’s Lisa Su steps in to fix driver issues with GPUs in new TinyBox AI servers

Odgovor na temu

Ivan Dimkovic

Administrator
Član broj: 13
Poruke: 16753
*.cable.dynamic.v4.ziggo.nl.

+7204 Profil

Re: AI: pomoć - ili opasnost?

^{23.12.2024. u 13:12 - pre 4 meseca}

I meni je cudno, mislim OK - nekadasnji ATI i nekadasnji AMD u finansijskim problemima, za ove prve se moglo reci da ih je bila briga za drajvere/sw, za ove druge je opravdanje da nisu imali resursa za sw... ali sad? Verovatno je korporativna kultura takva da se softveru ne daje puno znacaja osim onoga sto mora (firmware, low-level drajveri i sl.)

NVIDIA je tu eonima ispred a CUDA je verovatno bila najbolja strateska poslovna odluka koju su ikad doneli... Intel je isto imao (i mozda jos ima) jaku sw. kulturu (odlicni razvojni alati, FOSS podrska i sl.) ali na zalost trenutno nemaju hardver koji moze da konkurise (+ cesto pate od ubijanja projekata pre nego sto i mogu da imaju uspeh).

Ono... wtf, ako imas laptop sa nekom mobilnom RTX karticom, imas isto razvojno okruzenje kao za supercompute GPU klastere, koliko je tesko AMD-u da cilja na nesto slicno. Ja sam prosle godine kupio neke MI100 kartice za male pare, ali vec tada je AMD odlucio da ih obsolete-uje, cirkus.

Odgovor na temu

Nebojsa Milanovic
Pančevo

SuperModerator
Član broj: 16
Poruke: 8700
*.dynamic.sbb.rs.

+1431 Profil

Re: AI: pomoć - ili opasnost?

^{23.12.2024. u 19:50 - pre 4 meseca}

Kad si pomenuo CUDA, evo najnovije analize po tom pitanju:

Citat:

Nvidia’s relentless software ecosystem pushes its AI supremacy forward, forcing AMD’s MI300X to fight an uphill battle for datacenter dominance.

Amid Arm’s big moves in AI and Qualcomm’s partial legal victory, how might these industry shake-ups alter the balance of power across the broader chip sector—and who stands to gain the most?

Citat:

What The Chip: Nvidia’s dominance in AI training performance continues to overshadow AMD’s MI300X, according to new data from SemiAnalysis. Despite AMD’s lower price points and TCO, the software stack and real-world performance outcomes aren’t matching up to Nvidia’s H100/H200 offerings.

Details:

🔍 Performance Still Favors Nvidia: In side-by-side training benchmarks, AMD’s MI300X fell behind Nvidia’s H100/H200. While AMD’s advertised specs looked promising, however on-paper FLOPs are one thing, real-world throughput is another.

🛠️ Software Stack Pain: AMD’s MI300X requires custom builds and countless environment flags to reach anywhere near H100/H200 speeds. “We’re working on closing the gap, but it’s going to take time,” noted AMD’s VP of AI. Nvidia, meanwhile, offers an “out of the box” user experience with minimal fuss.

💰 Cost Advantages, But…: AMD’s theoretical cost savings (e.g., whitebox Ethernet vs. pricier NVIDIA networking) look tempting. However, SemiAnalysis found that “cheaper” hardware and networking don’t deliver much if suboptimal software cancels out the advantage.

⚙️ Nvidia’s InfiniBand & NVLink Edge: Nvidia’s integrated NICs, SHARP In-Network Reductions, and switched topologies keep multi-node scaling strong. AMD’s xGMI approach underperforms in multi-GPU training.

⚡ Executive Take: Dr. Lisa Su, AMD’s CEO, acknowledged in a past conference, “We believe in the potential of the MI300X platform to make a difference in HPC and AI workloads,” while also indicating more software resources would be needed. Meanwhile, Nvidia’s Ian Buck (VP of Hyperscale and HPC) remarked, “We remain committed to ensuring that HPC and AI developers have best-in-class tools.”

🤝 Future is Not Set in Stone: Multiple AMD engineers expressed optimism that with more investment, the MI300X can “bridge some of the gap.” Yet SemiAnalysis concluded that without fundamental changes to AMD’s software QA, end-users will continue to need heavy customization.

⏱️ Time Is Money: Nvidia’s next-gen Blackwell GPUs will likely land before AMD’s code matures. So if AMD doesn’t speed things up, Nvidia’s CUDA moat only grows deeper.

Why AI/Semiconductor Investors Should Care: Nvidia’s near-term advantage suggests strong momentum for its data center GPU revenues. At the same time, AMD remains a lower-cost wildcard that could gain traction if—big if—its software stack matures more quickly. For investors, this ongoing competition underscores how critical software ecosystems and out-of-the-box user experiences have become in AI training hardware decisions.

Citat:

What The Chip: Arm CEO Rene Haas just sat down for a wide-ranging interview with The Verge about the future of semiconductor design, AI workloads, and how the company balances its role as the “Switzerland of the electronics industry” while eyeing new vertical integration opportunities. If you’re watching the tech space, especially data center growth and on-device AI innovation, his insights carry big implications for investors.

Details:

⚙️ Possible ‘Arm Chips’? Rumors persist that Arm might design (or even build) its own AI chips. Haas suggested Arm needs to “understand hardware-software trade-offs more deeply,” hinting the company may test the waters without fully competing with its big-name customers.

🤝 SoftBank Influence: SoftBank CEO Masayoshi Son is still Arm’s majority owner. Haas described Son as “ambitious” and extremely hands-on with long-term strategy. Son’s readiness to fund new R&D could accelerate Arm’s AI push.

🌎 China & Geopolitics: Haas acknowledged that “a hard break” from China is difficult, given how supply chains interlock. He urged any incoming US administration to weigh the complexity of global semiconductor networks before applying heavy-handed tariffs or export bans.

⛽ Data Center Boom: AI training is fueling demand for specialized chips. Haas spotlighted AWS Graviton as an Arm-based success story, noting that half of all new AWS deployments are Arm-powered. He expects an “insatiable” appetite for inference to follow the current wave of training.

⚡ Competition vs. Partnership: Asked about Intel’s future, Haas suggested Intel should license Arm’s IP for its foundries, but Intel leadership wasn’t swayed. Still, he sees potential synergy if Intel’s IFS business embraces Arm’s ecosystem.

Why AI/Semiconductor Investors Should Care: Arm’s neutral approach has made it a linchpin of mobile, data center, and now edge AI. Any move to produce its own AI-focused chips could reshape industry alliances—and investor theses. With AI workloads blossoming from tiny wearables to massive training clusters, Arm’s architecture remains a pivot point for the entire chip sector.

Preporuka za ovu knjigu, autor je inženjer mikroelektronike za koga mislim da je najupućeniji analitičar za tu nišu:

https://mooresemiinvesting.com...=substack&utm_medium=email

Odgovor na temu

speculaas

Član broj: 337011
Poruke: 4

+1 Profil

Re: AI: pomoć - ili opasnost?

^{24.12.2024. u 16:10 - pre 4 meseca}

Citat:

Ivan Dimkovic:
Kad smo vec kod CUDA prednosti:

https://semianalysis.com/2024/...200-benchmark-part-1-training/

https://x.com/dylan522p/status/1871287937268383867

Ne znam, odgovor od Lisa Su zvuci mlako - slicno je bilo i kada je George Hotz rantovao za kvalitet drajvera

Odgovor na temu

Nebojsa Milanovic
Pančevo

SuperModerator
Član broj: 16
Poruke: 8700
*.dynamic.sbb.rs.

+1431 Profil

Re: AI: pomoć - ili opasnost?

^{25.12.2024. u 00:13 - pre 4 meseca}

Jedna od misterija marketa je činjenica da proizvođači čipova, dakle nečega što je apsolutno krucijalno u današnjoj eri, zaostaju za prosekom na marketu?!

Evo analize koja to potvrđuje:

Prikačeni fajlovi

SMH vs. SP 500.png - 177.97k

Odgovor na temu

Ivan Dimkovic

Administrator
Član broj: 13
Poruke: 16753
213.196.99.*

+7204 Profil

Re: AI: pomoć - ili opasnost?

^{25.12.2024. u 07:53 - pre 4 meseca}

Evo I vision verzije QwQ-a: QvQ

https://simonwillison.net/2024/Dec/24/qvq/

https://www.reddit.com/r/Local...oke_this_much_intelligence_is/

https://huggingface.co/spaces/Qwen/QVQ-72B-preview

Odgovor na temu

speculaas

Član broj: 337011
Poruke: 4

+1 Profil

Re: AI: pomoć - ili opasnost?

^{25.12.2024. u 13:29 - pre 4 meseca}

Kinezi su kompetitivni :)

Stize NV Blackwell Ultra: https://semianalysis.com/2024/...ce-amazon-memory-supply-chain/

Odgovor na temu

mjanjic
Šikagou

Član broj: 187539
Poruke: 3014

+775 Profil

Re: AI: pomoć - ili opasnost?

^{25.12.2024. u 16:27 - pre 4 meseca}

Kinezi su haos, imali su šansu sa onim Ascend čipom, ali dokumentacija je nikakva čak i na kineskom, a kamoli na engleskom.
Imao sam Atlas 200, koji služi za igranje i kao neki ozbiljniji inference, praktično mora na SD kartici da se instalira OS preko dekstop Linux-a, umesto kao za Raspberry Pi i slične varijante da naprave gotov "image" SD kartice i postave negde na svom sajtu.

Prisustvovao jednom onlajn webinar-u na temu njihovih Atlas kartica (300T za treniranje i 300K za inference), čak i ti koji su nam nešto pričali su totalno pogubljeni, nisu prijavljenima poslali materijal bar dan ranije, tada kao stavili na svoju WeChat grupu, a niko od slušalaca ne može da aktivira WeChat nalog bez potvrde nekoga ko već ima nalog, a ovi iz organizacije vebinara tada napisali da im se pošalje poruka, pa će oni videti da aktiviraju nalog kome treba...
Mislim, ukratko, oni možda znaju da se organizuju da nešto nabudže i naprave, ali kad treba da se organizuju da drugima nešto pokažu, onda je 90% njih pogubljeno, oni koji su zaista sposobni i sa te strane, uglavnom su već na Zapadu.

Kad sam već udavio, evo jedan link na temu o "train with mixed precision", gde se detaljnije objašnjavaju prednosti FP16, mada su se u međuvremenu pojavili FP8, kao i INT8 koji se koristi za inference, mada ima i drugih optimizacija kao što su INT4 ili modeli koji koriste "shift" opracije nad nizom bitova (ekvivalent množenju i deljenju sa 2):
https://docs.nvidia.com/deeple...-precision-training/index.html

Blessed are those who can laugh at themselves, for they shall never cease to be amused.

Odgovor na temu