I have two servers both network cards speed is 10Gbps

The inbound bandwidth between two servers is 10Gbps , the outbound bandwidth internet bandwidth is 500Mpbs

Both servers using public ip addresses in public and private network

Both servers transfer and connection on nginx port , and the server B used for streaming media , like youtube stream videos

I check the transfer speed using iperf utility From Server A to Server B

# iperf -c 0.0.0.1 -p 8777
------------------------------------------------------------
Client connecting to 0.0.0.1, TCP port 8777
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  3] local 0.0.0.0 port 38895 connected with 0.0.0.1 port 8777
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.8 sec    528 KBytes    399 Kbits/sec

My Current Connections in Server B

# netstat -an|grep ":8777"|awk '/tcp/ {print $6}'|sort -nr| uniq -c
   2072 TIME_WAIT
     28 SYN_RECV
      1 LISTEN
    189 LAST_ACK
    139 FIN_WAIT2
    373 FIN_WAIT1
   3381 ESTABLISHED
     34 CLOSING

Server A Network Card Information

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   100baseT/Full
                                1000baseT/Full
                                10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

Server B Network Card Information

Settings for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

The problem is : as you can see from iperf utility, the transfer speed from server A to server B slow when i restart network service the connection will be ok , after 2 minutes , it's getting slow

How could i troubleshoot slow speed issue and fix it in server B ?

Notice : if there any other commands i should execute in servers for more information, so it might help resolve the problem , let me know in comments