FortiGate VPN Speedtests

Lab

My lab consists of the following components:

Both FortiWiFi 90D firewalls had the firmware version v5.2.5, build701. The two notebooks were booted with Knoppix 7.6.1 and used Iperf version 2.0.5. The “left” machine ran as the server with either:

1 2	iperf -s iperf -s -u

while the “right” machine started Iperf with the following commands for different TCP and UDP tests:

1 2 3

iperf -c 192.168.10.10 -r iperf -c 192.168.10.10 -r -P 8 iperf -c 192.168.10.10 -r -u -b 1000M

I tested the throughput without a VPN at all (only routing) and with a few different proposals (see table below). The Diffie-Hellman group for PFS was always set to 14. This is not related to the test results because it is only used for the key establishment and not for the actual symmetric encryption of the traffic.
I also switched the offloading of encryption to “enable” (refer to the Hardware Acceleration Guide), which did not change anything, either.

1 2 3

config system npu     set enc-offload-antireplay enable end

Furthermore, I tested the differences between a normal TCP test and the manual set of the TCP window size and buffer length with “-w 512k -l 512k”, such as shown here or here. But this made no differences, too, since Knoppix Linux seems to auto set the window size pretty optimal.

Results

These are the results. The first four tests are without a VPN. While the first two are without routing (simply plugged in both clients into the same software switch on the FortiGate), tests 3 & 4 are routed through the FortiGates. This was the first time at which I was really shocked about the bad performance of only 180 Mbit/s routing speed. Furthermore, almost all IPsec proposals ran at a speed of 86 MBit/s, which is only 9 % of the IPsec throughput listed in the data sheet.

Proposals	TCP Tx/Rx [MBit/s]	TCP Tx/Rx [MBit/s]	UDP Tx/Rx [MBit/s]
IPerf Options	-r	-r -P 8	-u -r -b 1000M
Same Software Switch H - FGSW - H	942/937	941/936	807/805
Same Software Switch + Hardware Switch H - FGSW - SW - H	942/936	941/936	807/804
No VPN, only Routing FortiGate directly H - FG - FG - H	155/177	151/168	211/206
No VPN, only Routing H - FG - SW - FG - H	155/177	152/168	211/210
DES-MD5	86/86	83/82	93/94
3DES-MD5	86/86	83/83	93/94
3DES-SHA1	86/86	83/83	95/94
AES128-SHA1	86/86	83/83	88/87
AES256-SHA256	86/86	122/133	93/93
AES256-SHA512	85/85	80/80	84/92

The software switch was the problem!

After hours of investigating the slow VPN speed results, I tested the VPN without the software switch on the network ports side, which led to the following results (first column with a “Hardware Switch”, second column with a single interface):

Proposals	Hardware Switch TCP Tx/Rx [MBit/s]	Single Interface TCP Tx/Rx [MBit/s]
Iperf Options	-r	-r
No VPN, only Routing H - FG - SW - FG - H	937/937	933/932
DES-MD5	852/840	845/839
3DES-SHA1	707/642	701/634
AES128-SHA1	825/835	826/830
AES256-SHA1	820/830	816/825
AES256-SHA256	723/819	814/825
AES256-SHA512	637/808	812/810

Now the speed was quite acceptable, for the mere routing as well as for the VPN throughput. 940 MBit/s for routing through both FortiGate is almost realistic for TCP, and about 830 MBit/s for VPN encryption/decryption is realistic, too.
Here are the “single interface” results in a graph. Only the 3DES tests are a bit slower than all the other ones:

Single Interface	No VPN Tx	No VPN Rx	DES-MD5 Tx	DES-MD5 Rx	3DES-SHA1 Tx	3DES-SHA1 Rx	AES128-SHA1 Tx	AES128-SHA1 Rx	AES256-SHA1 Tx	AES256-SHA1 Rx	AES256-SHA256 Tx	AES256-SHA256 Rx	AES256-SHA512 Tx	AES256-SHA512 Rx
Single Interface	933	932	845	839	701	634	826	830	816	825	814	825	812	810

Conclusion

Well, it was my fault that I left the default software switch in place. I should have know better. However, it was the default setting on this FortiWiFi devices.