Troubleshooting (Fortigate)

Basic Commands

Ping / Trace Route

execute ping 10.10.1.10
execute traceroute 10.52.56.20

Shutdown / Restart

execute shutdown
execute reboot

Process Management

• Show top processes, refresh every 2 secs
  • diagnose sys top 2
• Kill a process
  • diagnose sys kill 9 <PID>
  • Most processes will restart, but use with caution!
  • diagnose sys kill 11 <PID>
    • kill 11 supposedly restarts a process

Killing off processes can cause unexpected results, so only perform if service through the firewall is being impacted (you may drop all existing connections through the firewall, or cause all IPsec tunnels to drop, if the process you are killing is responsible for those functions, for example). Its common for other processes to briefly show high CPU after you've killed a CPU hogging process, give it 30 secs or so to calm down.

Some discussion groups mention an issue when killing off cmdbsvr as it manages the config, and killing caused config corruption. It can be killed off, but may cause the loss/corruption of any recently edited config.

VPN

IPsec (Site 2 Site)

Show status of a tunnel, using Phase 1 name, sa=0 means no security associations, ie no connection

diagnose vpn tunnel list name P1-NAME

Bring a phase 2 up or down

diag vpn tunnel up P2-NAME P1-NAME
diag vpn tunnel down P2-NAME P1-NAME

Note that the below only debugs Phase 2 problems

diag debug reset
diagnose vpn ike log-filter name VPN-NAME
diagnose debug enable
diagnose debug application ike -1

The diagnose vpn ike log-filter name VPN-NAME line is optional, but if there's more than one VPN in operation the screen will be filled with output. You need a quick copy and paste into notepad when there's something interesting whizzing up the screen! Replace VPN-NAME with the name of the Phase 1.

Run diag debug reset once complete

SSL (User)

diag debug reset
diag debug enable
diag debug application fnbamd -1
diag debug appl sslvpn -1 

Run diag debug reset once complete

High Availability

Command	Action
show system ha	Show HA config
get system ha status	Show master/slave status
execute ha manage 1	Change console to master/slave
diagnose sys ha dump-by all-vcluster	Show HA cluster status
diagnose sys ha reset-uptime	Reset HA uptime timer - causes failover to standby as that will have the higher uptime

Console to Slave

Once SSH'ed to the master firewall, its possible to jump onto the slave (to run diag stats, reboot etc)

1. get system ha status
   • Gives output as shown below
2. execute ha manage 0
   • Normally 0 or 1, 0 in example below to switch to primary

Model: 200
Mode: a-p
Group: 40
Debug: 0
ses_pickup: enable, ses_pickup_delay=disable
Master: 90 PROD-FW01    FG200C4151018656 1           
Slave : 50 PROD-FW02    FG200C4151018713 0           <-- ID to use in "execute ha manage"
number of vcluster: 1
vcluster 1: work 169.254.0.2
Master:0 FG200C4151018656
Slave :1 FG200C4151018713

Change Heartbeat ports

This needs to be done on both firewalls! The command sets the ports and their priorities.

config system ha
set hbdev "port1" 50 "port2" 60
end

Change Priority

By default a FortiGate only fails over during a problem, but you can override so that the higher priority is always active. Therefore you can force fail-overs

You need to override on both master and slave firewalls...

PROD-FW01 # config system ha
PROD-FW01 (ha) # set override enable
PROD-FW01 (ha) # end

Check Master/Slave Config Replication

The configuration should replicate from the master to the slave, to ensure this is running correctly, run the following command and ensure the checksum's match between the units

diagnose sys ha cluster-csum

Force Failover

HA will put the unit with longest uptime live, therefore if you reset the timer on the master unit, it will failover to the standby

diagnose sys ha reset-uptime