congestion recovery for the HA / HTTP service

In most High Availability configurations at the end of the packing processing the response is sent back to the client only after the peer updated the lease so to summarize instead to have only packet processing we have a two stage pipeline with the packet processing at the first stage and the HA processing at the second stage.

The HA processing is sequential at both sides: the HTTP client reuses the same connection for the same destination and queues new requests waiting for the response, and at the peer side the HTTP request is a command which is executed before the answer is sent back and next command or packet is processed. As the packet processing is usually faster (and for sure faster with a multi-threading service) it is possible to accumulate a backlog of packets in the HA / HTTP part i.e. waiting for the peer lease update before being sent.

Packets waiting to be sent are put in a parking lot so it is trivial to count them. This ticket proposes to consider the HA parking lot as a limited size queue. If the parking lot is full no new packet should be added in it. As usual it is only congestion recovery: as the congestion as an external cause there is no way to avoid or control it: the only possible action is to mitigate congestion consequences so at first to limit backlog size.