Performance Issue

Post your questions about SoftEther VPN software here. Please answer questions if you can afford.
Post Reply
ingmar
Posts: 4
Joined: Thu Nov 22, 2018 4:37 am

Performance Issue

Post by ingmar » Sun Dec 16, 2018 1:07 pm

Hi all,

I have set up a more complex VPN Setup between three main locations and five branch offices. In this setup I have massive performance drops from about 4.5 MByte to about 100kbyte, when transfering data via the VPN in comparison to direct conncetion via intertnet (I used wget to test).

The Setup is as follows:
Main Office A / DSL 100/40 Mbit (192.168.5.0/24)
Main Office B / DSL 100/40 Mbit (192.168.4.0/24)
Main Office C / Cable 400/40 Mbit (192.168.3.0/24)

In all locations Softether is installed on dedicated AMD APU2D4 Boards, aes-ni is enabled and working, running on Ubuntu 18.04LTS (I also tried with 16.04LTS). I recompiled Softether 4.28 to be sure, the last libssl binary is included and used for AES-NI.
Connection encryption is set to "AES-256", top shows me about 15-20% CPU Usage during tests.
Connection is done using 32 parallel TCP connections (UDP was even slower).

I have set up the VPN Servers behind a NAT firewall, which is forwarding all required ports. So only one nic is used and bridged to the local hub.
As I have the server installed in all three locations, I have set up Layer3 Switching and established dedicated Networks in a separate IP Range (192.168.100.0/252) which are assigend as virtual network adapters to each hub. Reason for this was, that I do not have one centralized instance, where all remote offices connect to. Routing itself works fine between the networks.

In the branch offices, I have set up the same setup on APU2D2 Boards, but also there, the performance via VPN is not acceptable.

I checked the packet and the security logs without identifying any issues.
SecureNAT is disabled in all locations.

Any help, how to identify, what is going wrong, is highly appreciated.

Best Regards
Ingmar

davidebeatrici
Posts: 33
Joined: Tue Aug 28, 2018 6:44 am

Re: Performance Issue

Post by davidebeatrici » Mon Dec 17, 2018 7:18 pm

Hi,

Could you try to compile the latest development version available on GitHub (https://github.com/SoftEtherVPN/SoftEtherVPN) and check whether the issue persists, please?

Best regards,
Davide

ingmar
Posts: 4
Joined: Thu Nov 22, 2018 4:37 am

Re: Performance Issue

Post by ingmar » Thu Dec 20, 2018 8:45 pm

Hi Davide,

thanks for your reply.
I have recompiled with the last git version, but this made things even worse.
Now I get packet drops and ssl disconnects from the clients frequently:

Code: Select all

2018-12-20 21:27:14.543 On the TCP Listener (Port 5555), a Client (IP address XXX.XXX.XXX.XXX, Host name "pXXXXXXXX.dip0.t-ipconnect.de", Port number 57460) has connected.
2018-12-20 21:27:14.543 For the client (IP address: XXX.XXX.XXX.XXX, host name: "pXXXXXXXX.dip0.t-ipconnect.de", port number: 57460), connection "CID-4" has been created.
2018-12-20 21:27:14.614 SSL communication for connection "CID-4" has been started. The encryption algorithm name is "AES256-SHA".
2018-12-20 21:27:14.705 [HUB "XXX"] The connection "CID-4" (IP address: XXX.XXX.XXX.XXX, Host name: pXXXXXXXX.dip0.t-ipconnect.de, Port number: 57460, Client name: "SoftEther VPN Server (Cascade Mode)", Version: 4.20, Build: 9608) is attempting to connect to the Virtual Hub. The auth type provided is "Password authentication" and the user name is "site-2-site-from-xxx".
2018-12-20 21:27:14.705 [HUB "XXX"] Connection "CID-4": Successfully authenticated as user "site-2-site-from-xxx".
2018-12-20 21:27:14.705 [HUB "XXX"] Connection "CID-4": The new session "SID-SITE-2-SITE-FROM-XXX-3" has been created. (IP address: XXX.XXX.XXX.XXX, Port number: 57460, Physical underlying protocol: "Standard TCP/IP (IPv4)")
2018-12-20 21:27:14.705 [HUB "XXX"] Session "SID-SITE-2-SITE-FROM-XXX-3": The parameter has been set. Max number of TCP connections: 32, Use of encryption: Yes, Use of compression: No, Use of Half duplex communication: No, Timeout: 20 seconds.
2018-12-20 21:27:14.715 [HUB "XXX"] Session "SID-SITE-2-SITE-FROM-XXX-3": VPN Client details: (Client product name: "SoftEther VPN Server (Cascade Mode)", Client version: 420, Client build number: 9608, Server product name: "SoftEther VPN Server Developer Edition (64 bit) (Open Source)", Server version: 51, Server build number: 9666, Client OS name: "Linux", Client OS version: "Unknown Linux Version", Client product ID: "--", Client host name: "vpn-gateway", Client IP address: "192.168.4.253", Client port number: 57460, Server host name: "vpn.yyy.net/tcp", Server IP address: "yyy.yyy.yyy.yyy", Server port number: 5555, Proxy host name: "", Proxy IP address: "0.0.0.0", Proxy port number: 0, Virtual Hub name: "XXX", Client unique ID: "BF1F4622F86FF3B226E0A6EE1D350E7E")
2018-12-20 21:27:14.872 Connection "CID-4" has been terminated.

Code: Select all

2018-12-20 21:27:15.978 For the client (IP address:XXX.XXX.XXX.XXX, host name: "pXXXXXXXX.dip0.t-ipconnect.de", port number: 57462), connection "CID-7" has been created.
2018-12-20 21:27:16.059 SSL communication for connection "CID-7" has been started. The encryption algorithm name is "AES256-SHA".
2018-12-20 21:27:16.140 Connection "CID-7" has been terminated.
Is this an issue between the different versions of Softether?

Best Regards
Ingmar

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Performance Issue

Post by thisjun » Thu Jan 24, 2019 7:19 am

How did you measure the throughput with wget?

What is the time span for "4.5 MByte"?

ingmar
Posts: 4
Joined: Thu Nov 22, 2018 4:37 am

Re: Performance Issue

Post by ingmar » Mon Jan 28, 2019 8:06 pm

Hi,

I put some larger files (iso ~700mb; Software binary ~270mb) on the one side on an apache Server and then called wget on the other side once using the VPN tunnel (using the internal ip giving me about 100kb/s) and once without the tunnel (using the dns Name of the server giving me 4.5mb/s).
Wget gives u an actual and total Speed, which fits to the manual stopped seconds/minutes.

Never the less, i‘m wondering if my setup is correct.
On the vpn Server i have three Nic. How should a correct setup look like (ip setup and bridging/local bridging)?
In the manual i only found a variant where one nic is pointing to the Internet, but not both nics in the internal lan.

Best Regards
Ingmar

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Performance Issue

Post by thisjun » Fri Mar 15, 2019 8:13 am

Could you decrease the number of TCP connections?
If packet loss is happening, increasing the number of TCP connections makes a negative effect.

ingmar
Posts: 4
Joined: Thu Nov 22, 2018 4:37 am

Re: Performance Issue

Post by ingmar » Tue Nov 12, 2019 6:08 pm

Hi,

after a long time I wanted to come back with some updates:

1) Changing the Number of TCP connections did not change anything.
2) For testing, I switched one site2site connection from Softether to WireGuard. Interesting enough this did not change anything in speed.
3) I activated SSTP on Site A. When I now connect from a Laptop on site B using the Build in Microsoft Client, i get constant 4MB/s (32 Mbit/s) using SMB copy (up and down). When I deactivate the dial-in connection and use the site2site again, SMB is nailed to 355kb/s. This value you can find easily across the net as indicator for VPN Problems without any solution given. So neither the ISP Speed nor the cpu load on the VPN Gateway is the problem.
4) I noticed the dial-in adapter gets a mtu of 1400, where eth0 of course has a mtu of 1500. I played around with this value without any success.

So i assume its a MTU/MSS issue. The question to the experts now is: Which value needs to be modified and where?

Best regards,
Ingmar

mbuster
Posts: 2
Joined: Fri May 08, 2020 6:28 am

Re: Performance Issue

Post by mbuster » Fri May 08, 2020 12:20 pm

Hi Ingmar,

I am interested to know if you have resolved the issue for your multiple sites locations.
If I could make a suggestion, perhaps you can try a different hardware for example Intel J1900 and Celeron 1037U based hardware.

For SMB performance, it would greatly depends on the latency (ping) response between client and the server, maybe you can try to ping between client and server to see what it is the latency, both ping outside public IP and internal private IP.

Also, Intel J1900 based hardware can do about 100 Mbits (given a right condition) with Softether VPN tunnel via HTTPS/SSL TCP port 443 connection without AES-NI engine.

Let me know if you have any comments, etc.

best regards,

mbuster
Posts: 2
Joined: Fri May 08, 2020 6:28 am

Re: Performance Issue

Post by mbuster » Sat May 09, 2020 5:42 am

Hi,

I just want to add some updates as I have just done some recent tests.

Some specs:
OS: based on Debian 9 Jessie
SE Client and Server Cipher AES128-SHA
maximum concurrent TCP connections: 2 with no UDP acceleration
Hardware: SE Client end: AMD APU G-T40E (no AES-NI engine)

Testing performed:
Windows 10 Client to Windows 2019 server
Ping 10-11ms end to end
SMB transfer 9-10MB/sec or about 80 Mbps to 90 Mbps

As we can see AMD APU can perform about maximum throughput of about 90Mbps with SE AES128-SHA cipher.
As I do not have the AMD APU2 (GX-412TC) hardware, we are not able to perform the throughput test,
but I could imagine that it would be a higher performance, since it has 1 GHZ clock (higher than 800 MHZ of G-T40E)
and with AES-NI built-in engine.

thanks and regards,

Post Reply