Egregoros

@lattera the density is higher i think. I've heard that they can't just swap out hardware if they want to upgrade. Requires entire tear down of the racks, power, and cooling and rebuilding it.


edit: obviously this just sounds insanely stupid and risky to not be able to repurpose the infra

@feld @lattera Basically yes as far as I understand it. The equipment comes in full racks, usually custom, and fully wired internally. And since nVidia, AMD and OCP all have different rack sizes, it probably requires an almost full teardown at best.

@feld @lattera Companies like Amazon and Google already don’t service systems within a rack. They build a whole rack, deploy it as one unit, then decommission it as one unit when a certain percentage of it has failed.

I suspect what makes an “AI datacenter” is actually buzzword bingo, but it could plausibly be about the dominant type of compute it provides: CPU or GPU.