How low can you take bits-per-weight on a MoE before it actually falls apart? (and what does bpw mean exactly)
Two questions in one, and I am not embarrassed about the first. One: what does bpw actually mean? I see it everywhere, 4bpw, 2.5bpw, "stay above 4bpw" and I have been nodding along for months. I assume it is bits per…