I very much do not want to go back to the days of "read up to 4 bytes off the wire to see how big the next chunk is. Oh I only read 3 bytes? Guess I'd better save those 3 + the fact I need to read 1 more, then drop back out to the event loop"
> I very much do not want to go back to the days of ....
Not what I recommend. The C library does much better than that.
In asynchronous programming you can only deal with the data you have, so you buffer it. I have not counted bytes like that - for the purposes of reading from a file, in decades.
But that's where non-blocking leads; it can't block until it reads all the bytes you asked for, that has to be handled somehow.
A fancy C library could buffer the partial read for you rather than you needing to do it, and it could even maybe deal with turning your function into the state machine required to resume the function at the partial read.
But then you look around and realise you've created another async/await.