FPrime/GDS Connection failing due to stale port binding

[LornDMiller]

I'm working with an F' topology that's set up to have F' run as the TCP Server and GDS as the TCP Client. Occasionally if I shut down and restart too quickly (on the order of minutes), The new F' instance is unable to bind to port 50000 and as a result F' GDS cannot connect.

I know that this is due to the OS leaving the port from the previous instance "open" for some time after F' shuts down. I've worked around this on previous programs by setting SO_REUSEADDR on that port (insert obvious security implications here).

Is there any way to get F' to set SO_REUSEADDR on this TCP server port so that I don't have to wait several minutes when this happens? I've reviewed the code and don't see a mechanism to insert this short of modifying fprime or replacing the comDriver with my own version, neither of which I really want to do.

Related but much lower priority, when this happens F' GDS spins. I'm assuming it's trying to connect but not sleeping after a failure. Is there a mechanism by which I can force F' GDS to sleep for a second after a connection refused error on that connect attempt? Whenever my processor gets railed like this I immediately start looking for a misbehaving FSW task!

[vincewoo]

Hi @LornDMiller I took a quick look at the F' code and we aren't currently calling setsockopt when we are creating those TCP sockets so you're right, currently there is no way for you to enable that option. We could make a change to F' to expose setting that as an option potentially, and maybe even extend that to allow setting other socket options. Pasting the full set of options here:

SO_ACCEPTCONN
    Socket is accepting connections.
SO_BROADCAST
    Transmission of broadcast messages is supported.
SO_DEBUG
    Debugging information is being recorded.
SO_DONTROUTE
    Bypass normal routing.
SO_ERROR
    Socket error status.
SO_KEEPALIVE
    Connections are kept alive with periodic messages.
SO_LINGER
    Socket lingers on close.
SO_OOBINLINE
    Out-of-band data is transmitted in line.
SO_RCVBUF
    Receive buffer size.
SO_RCVLOWAT
    Receive ``low water mark''.
SO_RCVTIMEO
    Receive timeout.
SO_REUSEADDR
    Reuse of local addresses is supported.
SO_SNDBUF
    Send buffer size.
SO_SNDLOWAT
    Send ``low water mark''.
SO_SNDTIMEO
    Send timeout.
SO_TYPE
    Socket type. 

[LornDMiller]

Hey Vince! Yeah, if this isn't currently possible, it might be a good option to add. We should also be cognizant of the potential security implications of enabling some of these though, and let the user decide their own risk posture.

For example, if we enable S_REUSEADDR and there's a hostile actor on the same machine, they could also open that port and intercept some of our messages.

[ReggieMarr]

I've also run into this issue and found it pretty negatively impacts what I view as the single most important metric feeding into developer productivity (time to iterate).

IMHO on balance the default behaviour of disabling S_REUSEADDR is fine (due to the security implications mentioned above).
But having the option to enable would be a real QOL improvement and since its being done for iterative development I think its fairly safe to say this is not being used in a production environment/somewhere where the machine is exposed (i.e. behind a firewall/on a private network).

[LornDMiller]

@vincewoo - Is there a plan to add this capability? I just got hit by it again, and I'd love to just pull and update to avoid it rather than hack around it inside fprime.

[vincewoo]

@LornDMiller I've created a new issue to track this request #4075

Feel free to upvote and reply to the issue with your interest in the issue to get it up the prioritization list :D

[vincewoo]

Just wanted to update this thread that @m-aleem has added the ability to set socket options to F' in #4146 !