The development team that I’m on has some rather unique challenges when dealing with Eggplant. Most users of Eggplant have a 1 to 1 connection for the SUT. We have some unique connections that we have to traverse that we developed bash code for.
Example:
The bash code that was developed sets us up some SSH tunnels and sets up the VNC server connection for us. We then build these connections in Eggplant’s connection list.
Before we hit the first poxy we are going from one building to another through a firewall, switches, etc… with only a 100mb pipe (not very good).
Once we get to the second proxy, its basically the same scenario minus the building.
Sometimes we can connect and pull up the VNC server windows. However, a vast majority of the time we get connection timeout issues.
Is there a way to increase the amount of seconds that Eggplant is waiting before the connection times out? Again, we are using Linux.
We are also attempting to do this while using 4 SUTs at once and changing between the SUTs to test a large integrated system. In addition to that we are trying to handle the case where multiple users can be running tests at the same time, and same test lines.
Which process is logging the connection timeout issues and is the connection actually being established and then dropped, or is it not even established in most cases? Are you working with tcpdump/wireshark at this point or just looking at server logfiles?
Are you tunneling VNC traffic over the SSH connections?
The text in bold in your question indicates that you believe that the VNC driver used by Eggplant is timing the connection out but can you maybe expand on the evidence for this is and whether you’ve turned up logging on it through config?
We’ve had this problem also and found our way around it by changing sshd configuration values for timeouts on the intermediate hosts using guidance from How to Troubleshoot SSH Connectivity Issues :: DigitalOcean Documentation and other sites. But before spending too much time on implementing the suggestions, we turned up the logging level of the sshd processes on each machine from the default to verbose logging using rsyslog settings to see whether we could establish a consistent pattern for what we were seeing and to determine whether the problem was mostly in BigIP F5 configuration (or our firewalls) or elsewhere.
I’d suggest that the crucial starting question is - does this behaviour occur when you don’t use EPF for the tests but do things by hand?
Which process is logging the connection timeout issues and is the connection actually being established and then dropped, or is it not even established in most cases?
It’s not established in most cases
Are you working with tcpdump/wireshark at this point or just looking at server logfiles?
I’m just looking at log files, but I had the network team look at it and they looked with tcpdump / wireshark. I’m not sure what their results are quite yet, as they have not gotten back to me.
Are you tunneling VNC traffic over the SSH connections?
Yes
The text in bold in your question indicates that you believe that the VNC driver used by Eggplant is timing the connection out but can you maybe expand on the evidence for this is and whether you’ve turned up logging on it through config?
I don’t know what the issue is. I just suspect it is Eggplant, as our bash scripts setup the VNC server connections and SSH tunnels just fine. However, when we ask Eggplant to connect to the VNC server, we receive connection dropped by host or connection not established.
Whose VNC server is being used?
We are using Tiger VNC
I’d suggest that the crucial starting question is - does this behaviour occur when you don’t use EPF for the tests but do things by hand?
It only occurs with Eggplant. Our scripts do what they need to do.
Our network team said that ssh timeout was set to 24 hours years ago. Maybe I just blame it on our 100mb pipe?? I don’t know what the issue could be unless it is firewall related.
>I don’t know what the issue is. I just suspect it is Eggplant, as our bash scripts setup the VNC server connections and SSH tunnels just fine. However, when we ask Eggplant to connect to the VNC server, we receive connection dropped by host or connection not established.
If you get the opportunity to try this with RealVNC in place of TigerVNC you may find, as I did, that the commercial product offers more options for configuration and troubleshooting. I am broadly happy wth RealVNC but as you may know, they have a kind of “embrace and extend” philosophy in terms of product features sold on top of the open-source product, and you may not want to pay the cost for scaling up with that commercial product.
Just to check first though, you can get a VNC remote desktop over the tunnelled connections reliably when you try manually, but not through the VNC software bundled into Eggplant?
I work in defense, so I cannot download software for use off the internet. I have to go through the cyber team. I could probably request real vnc if I can prove that it may work better…maybe.
Anyways, to answer your question, yes. If we use our bash scripts, everything works fine. We have a bash script that starts the VNC server session and we have bash scripts that creates the SSH tunnels (called from Sensetalk). Not all the time, but most of the time we have connection timeout, connection dropped by host, and connection refused issues with Eggplant.
We had some keysight folks come out and give us a sales pitch on Eggplant DAI. At that same time, we asked them questions and also provided recommendations for feature enhancements. When I brought my situation up to the Keysight reps, they really did not know what to do about it, except to tell me to reach out to tech support.
The tech rep that was at the meeting did not seem to think that the new version of Eggplant played nice with VNC and SSH tunnels. She also told me to use VNC Viewer to help troubleshoot. It seems like they know that there is an issue with the new version (23.1) and VNC. The newest version is actually 23.2, but from what we were told it was just release earlier this week. So, we haven’t even thought about asking for it.
I can’t speak to a specific version of Eggplant having this issue, as I’ve seen it across multiple versions of the years. I always upgrade to the latest (on Windows, at least) so it’s not seemed any different from my perspective.
I struggled with this many times myself and never found a technical answer. However, my issues were resolved recently. While I worked onsite before we became fully remote, my laptop was connected via wireless to our LAN. Under the WLAN at work and at home connected by wireless I had this issue. Even with speeds up to 500 mbps.
I upgraded my home network to a 1gb pipe not long ago. I decided to turn off my wireless and connect via Ethernet cable to get the maximum speed out of my new connection. I’ve not had this VPN drop issue since. While I prefer to have a solid technical answer for why something is happening, I’m not complaining.
I don’t know if you are cabled to your LAN or connecting via WLAN, but if you are wireless I highly recommend looking into cabling up to see if this could resolve the problem.
Chris, thanks for your response. I have long thought that our issues were because we are on a 100mb pipe (multiple teams are using this). I just do not have any proof. Im hoping that the network / cyber team can provide some insight with wireshark and tcpdump.
We are supposed to be moving to a 1gb pipe or larger. So hopefuy that will resolve the issues.
Here’s to a bigger pipe! If I had more time, I would set something up to investigate further but I’m happy for the simple solution (for me at least). In the meantime, it would be helpful to get more detail on the “why” regarding:
.