This software is becoming a royal PITA aint it? lol
Thank you for the ping and the update!
Looks like you're on the right path to chasing the gremlins out. I'm glad iperf3 was helpful to you. It has helped me out tremendously many times.
For the record, you can always ping me anytime. I'm here to help and Lemmy notifications don't work half the time. But direct mentions always work.
Please keep me in the loop with further updates. At this time, nothing further to add from me. You're doing the right things.
MOD APPROVED
Totally understand the security and CAB process. It’s a royal PITA when it comes to troubleshooting.
Mind keeping me in the loop with your findings? I’ll help as much as I can.
Not sure on the logging. I’m a data center guy and would rather see firewalls in the trash lol. They usually just cause problems.
For the WAN, surely there is some way you can reach those sites over the general internet. You have ISP connections.
Are you sharing BGP to the ISP? Maybe make a couple of 1:1 NATs with test boxes not in prod so that you can quickly test pathing outside of the tunnel.
That's because the mods on r/networking aren't actually good at their jobs and neither are most of the commenters, yet they all think otherwise. We're simply better over here B-)
@[email protected] Apologies for the delay. I've been very tired lately. I'm going to most likely repeat some of the things others have mentioned and what you've already noted, but this would be my t/s process. (NOTE: all tests should be ran on the endpoints, not network infra)
-
Traceroute from UK -> Germany and Germany -> UK. Look for latency spikes. The reason I say do both directions is that sometimes there is weird pathing issues present that only show in the opposite direction.
-
iperf 3 from UK -> Germany and Germany -> UK.
- 2a. Clear counters on switches/routers/firewalls.
- 2b. During an extended iperf test, look for interface errors, CPU usage on the devices in path.
- 2c. This is tedious and will take time, but you're dealing with gremlins.
-
TCPdump on both sides during a transaction. Check for re-xmits and window scaling problems. Most likely not the endpoints, but something to rule out.
-
Monitor fortigate logs during all of this
-
Setup test boxes in UK and Germany that are exempt from IPSec tunnels and test throughput again (this should be a clear indicator that the firewalls are fucked if this is good)
-
All else fails, open TAC case with Fortigate.
Performance problems are the hardest problems to solve unfortunately. I've got more thoughts to add to this, but have to get to some commitments today. I'll add more detail either tonight or tomorrow @[email protected]
Wow. I totally don’t see that comment on my end at all. I am so sorry about that.
As for your issue, that is a strange one. Can’t say I’ve ever seen it before. Is this a legacy switch and software version?
Edit: Oh wait, you meant PortFast. So this was a Spanning-Tree issue. Makes sense. Spanning Tree is the devil btw.
You're very welcome. Please, feel free to reach out anytime if you have more questions about networking or the industry. I'm about 9 years in and have a boat load of certs from many vendors.
I may end up closing this community in favor of the networking community [email protected] as that sub/instance is more active and I mod over there too. If you can't reach me on this account, I also have @[email protected] that I'm more active on.
Also, if you would like to keep in contact on LinkedIn, I can PM you my page.
Food for thought :)
Cisco does a decent job at teaching general networking concepts, but most of them are "the Cisco way", which I despise. Juniper is a much more standards-focused vendor vs magical bullshit like Cisco. The downside is that Juniper assumes you know networking already and doesn't do that great of a job at teaching.
Another thing to keep in mind is that Cisco, unfortunately, is still the most prevalent networking vendor. This means the jobs are more plentiful and the skillset is more saturated in the market. Most people are mega fanboys of Cisco and either are too stupid or too ignorant to learn other vendors, which limits their thinking and skillset growth. Cisco is very much a cult.
Get some experience and see what makes you happy, then pursue that knowledge. Ultimately, if you aren't happy with the work then your career will be miserable.
I've heard kbin is a total shit show.