notsent_lowat and the starved reply
A bulk send will happily fill the socket until the window is huge. A small response that needs to go out on the same connection then sits behind megabytes that have been accepted by write() but not yet put on the wire. From the other end it looks like the server froze.
net.ipv4.tcp_notsent_lowat = 131072 is the cap I use: the socket is not allowed to hide more than 128 KB of unsent data from the application. The bulk flow still runs. The small reply can be queued in a bounded amount of time. I do not raise the TCP buffers to 64 MB on a 1 GB guest to make a single stream look faster in the morning.