At OCP Summit 2022, Wiwynn showed off a neat solution for cooling next-generation accelerators. The company was showcasing a design for cooling 8x 1kW accelerators in a single chassis. As one may imagine, this was a liquid cooling solution, making it look very interesting indeed. The company always brings interesting hardware to these shows, so we wanted to highlight a few bits from the company’s booth.
At first, one looking to this chassis will see that there are two openings for liquid nozzles on the front of the chassis, but what is behind them is more interesting.
Inside we have large cold plates. These are for direct-to-chip liquid cooling and one can see the massive copper designs. As for the form factor, these are designed to handle OAM/ OAI modules. OAM is the form factor we see even some current-generation public accelerators like the AMD Instinct MI250x, Intel Habana Gaudi2, Intel Ponte Vecchio and others already utilizing.
Here is a close-up of the nozzles. If you want to learn more about liquid cooling, you can see our pieces at CoolIT in Alberta, Canada earlier this year.
Internally, these 8x liquid cooling blocks are mated to an internal manifold to distribute the fluids along cool inlet and warm outlet paths.
For some context here, the industry has a clear line of sight into 1kW accelerators coming on the relatively near horizon. That is why we are already seeing cooling solutions for them.
As we get to accelerator shelves using 8kW+, direct-to-chip liquid cooling has the advantage of still easily using vertical racks and therefore relatively easier integration into existing facilities. Other options are single and two-phase immersion cooling setups. Wiwynn invested in LiquidStack as an immersion cooling partner and had this demo as well.
We covered LiquidStack as part of our Liquid Cooling Solutions Run-Through from SC21 last year if this looks familiar. Wiwynn is an investor in LiquidStack.
Picking three honorable mentions from the Wiwynn booth, one was the Facebook Yosemite V3 sled, the Wiwynn SV7100G4. This can house three compute nodes, but also many accelerators. One can see the dual-M.2 accelerators here. Each of these sleds not only has the server component, but they can handle either 12x dual M.2 or 24x single M.2 accelerators or storage.
Next, we have Facebook’s hard drive solution. This is the Wiwynn SV/ST7000G4. It has up to 72x drives. Something interesting here is that Facebook is using SAS-4 for this chassis. Many of the enterprise storage arrays are still on SAS-3 these days.
While the company had an Astera Labs CXL memory expander in PCIe, it also had this. This is a prototype CXL memory expansion mezzanine card. CXL is coming, and soon.
The fun part about OCP Summit is that, while we see 700W accelerators today, we are close enough to 1kW accelerators that solutions are already being shown to cool them. For our readers, we have been running a series over the last year or more on liquid cooling and the importance of having liquid cooling capabilities going forward. In hyper-scale data centers where Wiwynn plays, moving to some form of liquid cooling is a foregone conclusion. For others, 10kW racks are going to look very sparse if you want to run high-end accelerators with perhaps only a single system in 2023, and maybe not even a full system with 8x accelerators in 2024.
For those that do not know Wiwynn, they are primarily a supplier to sophisticated hyper-scale customers (and increasingly telco.) In 2018, I was able to visit the company’s lab in Where Cloud Servers Come From Visiting Wiwynn in Taipei.