Nvidia’s must-have H100 AI chip made it a multitrillion-dollar company, one that may be worth more than Alphabet and Amazon, and competitors have been fighting to catch up. But perhaps Nvidia is about to extend its lead — with the new Blackwell B200 GPU and GB200 “superchip.”
Nvidia says the new B200 GPU offers up to 20 petaflops of FP4 horsepower from its 208 billion transistors. Also, it says, a GB200 that combines two of those GPUs with a single Grace CPU can offer 30 times the performance for LLM inference workloads while also potentially being substantially more efficient. It “reduces cost and energy consumption by up to 25x” over an H100, says Nvidia, though there’s a questionmark around cost — Nvidia’s CEO has suggested each GPU might cost between $30,000 and $40,000.
Training a 1.8 trillion parameter model would have previously taken 8,000 Hopper GPUs and 15 megawatts of power, Nvidia claims. Today, Nvidia’s CEO says 2,000 Blackwell GPUs can do it while consuming just four megawatts.
On a GPT-3 LLM benchmark with 175 billion parameters, Nvidia says the GB200 has a somewhat more modest seven times the performance of an H100, and Nvidia says it offers four times the training speed.
Nvidia told journalists one of the key improvements is a second-gen transformer engine that doubles the compute, bandwidth, and model size by using four bits for each neuron instead of eight (thus, the 20 petaflops of FP4 I mentioned earlier). A second key difference only comes when you link up huge numbers of these GPUs: a next-gen NVLink switch that lets 576 GPUs talk to each other, with 1.8 terabytes per second of bidirectional bandwidth.
That required Nvidia to build an entire new network switch chip, one with 50 billion transistors and some of its own onboard compute: 3.6 teraflops of FP8, says Nvidia.
Previously, Nvidia says, a cluster of just 16 GPUs would spend 60 percent of their time communicating with one another and only 40 percent actually computing.
Nvidia is counting on companies to buy large quantities of these GPUs, of course, and is packaging them in larger designs, like the GB200 NVL72, which plugs 36 CPUs and 72 GPUs into a single liquid-cooled rack for a total of 720 petaflops of AI training performance or 1,440 petaflops (aka 1.4 exaflops) of inference. It has nearly two miles of cables inside, with 5,000 individual cables.
Each tray in the rack contains either two GB200 chips or two NVLink switches, with 18 of the former and nine of the latter per rack. In total, Nvidia says one of these racks can support a 27-trillion parameter model. GPT-4 is rumored to be around a 1.7-trillion parameter model.
The company says Amazon, Google, Microsoft, and Oracle are all already planning to offer the NVL72 racks in their cloud service offerings, though it’s not clear how many they’re buying.
And of course, Nvidia is happy to offer companies the rest of the solution, too. Here’s the DGX Superpod for DGX GB200, which combines eight systems in one for a total of 288 CPUs, 576 GPUs, 240TB of memory, and 11.5 exaflops of FP4 computing.
Nvidia says its systems can scale to tens of thousands of the GB200 superchips, connected together with 800Gbps networking with its new Quantum-X800 InfiniBand (for up to 144 connections) or Spectrum-X800 ethernet (for up to 64 connections).
We don’t expect to hear anything about new gaming GPUs today, as this news is coming out of Nvidia’s GPU Technology Conference, which is usually almost entirely focused on GPU computing and AI, not gaming. But the Blackwell GPU architecture will likely also power a future RTX 50-series lineup of desktop graphics cards.
Share to your social below!
Hi it’s me, I am also visiting this website daily, this site is
really fastidious and the visitors are actually
sharing nice thoughts.
Feel free to visit my blog – vpn coupon code 2024
What’s Happening i’m new to this, I stumbled upon this I have
discovered It positively useful and it has helped
me out loads. I’m hoping to contribute & help different users
like its helped me. Great job.
my web site: vpn special coupon code 2024
Howdy are using WordPress for your blog platform?
I’m new to the blog world but I’m trying to get started and create my
own. Do you need any coding expertise to make your own blog?
Any help would be really appreciated!
Visit my web page vpn coupon code 2024
Nice post. I learn something totally new and challenging on websites I stumbleupon on a daily basis.
It will always be helpful to read articles from other authors and practice something from their
web sites.
My site vpn 2024
Wow, marvelous blog layout! How lengthy have you been blogging for?
you make running a blog glance easy. The total glance of your site is
excellent, let alone the content! You can see
similar here sklep internetowy
Wow, wonderful blog format! How lengthy have you been blogging for?
you made running a blog look easy. The entire look of your
website is wonderful, as neatly as the content!
You can see similar here dobry sklep
I rattling thankful to find this site on bing, just what I was searching for : D as well saved to my bookmarks.
Wow, superb weblog structure! How long have you ever been running a blog for?
you make blogging look easy. The full look of
your web site is fantastic, as smartly as
the content material!
Also visit my homepage … facebook vs eharmony to find love online
My partner and I stumbled over here different web page and thought I should check
things out. I like what I see so i am just following you.
Look forward to looking at your web page again.
my site; eharmony special coupon code
I am genuinely thankful to the holder of this website who has shared this great paragraph at at this place.
my page :: nordvpn special coupon code 2024
Have you ever considered creating an ebook or guest authoring on other websites? I have a blog based on the same subjects you discuss and would love to have you share some stories/information. I know my readers would enjoy your work. If you’re even remotely interested, feel free to send me an e-mail.
It’s the best time to make a few plans for the future and it’s time to be happy. I have learn this post and if I may just I wish to suggest you some attention-grabbing things or suggestions. Maybe you could write subsequent articles referring to this article. I desire to read even more things about it!
After examine a number of of the weblog posts in your web site now, and I truly like your way of blogging. I bookmarked it to my bookmark web site checklist and will probably be checking back soon. Pls take a look at my website as well and let me know what you think.
Very nice post. I just stumbled upon your blog and wished to say that I’ve truly enjoyed browsing your blog posts. In any case I’ll be subscribing to your rss feed and I hope you write again soon!
You are a very smart person!
Very interesting information!Perfect just what I was looking for!
Regards for this post, I am a big fan of this internet site would like to keep updated.
Fantastic beat ! I would like to apprentice while you amend your web site, how can i subscribe for a blog web site? The account helped me a acceptable deal. I had been tiny bit acquainted of this your broadcast offered bright clear concept
Heya i am for the first time here. I came across this board and I find It truly useful & it helped me out much. I hope to give something back and help others like you aided me.
I loved as much as you will receive carried out right here. The sketch is tasteful, your authored subject matter stylish. nonetheless, you command get bought an impatience over that you wish be delivering the following. unwell unquestionably come more formerly again as exactly the same nearly very often inside case you shield this increase.
It is perfect time to make a few plans for the longer term and it is time to be happy. I’ve read this put up and if I may just I want to counsel you some attention-grabbing issues or tips. Perhaps you can write next articles regarding this article. I wish to learn even more issues about it!
It’s really a nice and helpful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.
I like the valuable info you provide in your articles. I’ll bookmark your weblog and check again here regularly. I’m quite sure I’ll learn plenty of new stuff right here! Best of luck for the next!
What does the Lottery Defeater Software offer? The Lottery Defeater Software is a unique predictive tool crafted to empower individuals seeking to boost their chances of winning the lottery.
I¦ve learn some just right stuff here. Certainly worth bookmarking for revisiting. I wonder how much attempt you put to create such a fantastic informative web site.
Excellent website you have here but I was wanting to know if you knew of any user discussion forums that cover the same topics talked about in this article? I’d really like to be a part of community where I can get feed-back from other experienced individuals that share the same interest. If you have any recommendations, please let me know. Thank you!
Magnificent goods from you, man. I’ve understand your stuff previous to and you’re just extremely fantastic. I really like what you have acquired here, certainly like what you are saying and the way in which you say it. You make it enjoyable and you still take care of to keep it smart. I can not wait to read much more from you. This is actually a great website.
F*ckin¦ tremendous issues here. I¦m very satisfied to peer your article. Thanks a lot and i am taking a look forward to touch you. Will you kindly drop me a mail?
Very interesting details you have observed, appreciate it for putting up. “My work is a game, a very serious game.” by M. C. Escher.
Hi my friend! I wish to say that this post is awesome, great written and include almost all important infos. I¦d like to peer extra posts like this .
Simply wanna remark on few general things, The website design and style is perfect, the subject matter is real wonderful : D.
Hi would you mind stating which blog platform you’re using? I’m going to start my own blog soon but I’m having a hard time making a decision between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something completely unique. P.S Sorry for being off-topic but I had to ask!
That is the appropriate blog for anybody who desires to find out about this topic. You realize a lot its virtually onerous to argue with you (not that I really would need…HaHa). You definitely put a new spin on a topic thats been written about for years. Great stuff, just great!
Hiya very cool blog!! Man .. Excellent .. Amazing .. I will bookmark your blog and take the feeds also…I’m satisfied to seek out so many helpful info here in the put up, we’d like develop extra techniques in this regard, thanks for sharing.
Wow, incredible blog layout! How long have you been blogging for? you made blogging look easy. The overall look of your web site is wonderful, let alone the content!
I just couldn’t depart your site before suggesting that I extremely loved the standard information a person provide in your visitors? Is gonna be back incessantly to check out new posts.