How to find out when to increase bit rate? (TCP streaming solution)

We have a stream with "frames". each "frame" has a "timestamp" . frames have bit rate property which is actually there size. We generate frames with our app and stream them one by one on to our TCP server socket. At the same time server post replies so when after each sent frame we try to read from socket we receive which timestamp is currently on server. if timestamp is lover than previous frame we lover bit rate 20%. Such scheme seems to work giving me one way vbr (lowering) but I wonder how to implement increase? I mean we can always try to increase 5% each frame until some limited desired value but each time we have delay will lose real-time feature of our stream... Generally such scheme is for finding out how much of network stream is currently used by other user apps and give picture of how much server is loaded at the same time so we can stream just right amount of data for all to receive it in real time. So what shall I do to add increase to my scheme? So having current bit rate of A I thought we could add +7% for 3 frames and than one -20% and than if all that 3 frames with +7% came in time we could add 14% to A and repeat circle and it would hopefully not be really noticeable if 2nd frame wold come to us with delay... probably this one is too localised because it is a requirement for me to use TCP.