BGP Archives

Short answer: When the BGP session between a Tier-0 gateway uplink and the top-of-rack switch does not come up, the cause is almost always one of two mismatches: the VLAN ID of the Edge uplink does not match the VLAN on the ToR interface, so there is no layer 2 path, or the BGP settings (neighbor IP, AS numbers, password, timers) do not match on both sides. Check the session state with get bgp neighbor summary in the Tier-0 SR context on the Edge, then read /var/log/frr/frr.log for the exact reason.

Start with the BGP state

get logical-routers
vrf <vrf-id-of-the-TIER0-SR>
get bgp neighbor summary
get bgp neighbor <peer-ip>
get interfaces

The state in the summary tells you where to look:

State	What it means	Look at
Idle, Connect, Active	The TCP session on port 179 never comes up.	Layer 2 and IP reachability: uplink VLAN, IP and subnet, firewall or ACL on the ToR.
OpenSent, OpenConfirm	TCP works, but the peers reject each other’s OPEN message.	BGP parameters: AS numbers, neighbor IP, password, capabilities.
Established, then drops	The session flaps.	Hold timer, BFD, MTU, physical link or Edge resources.

Two more commands do much of the analysis for you: get bgp neighbor <peer-ip> session-diagnostics and get bgp-bfd diagnostics report.

Cause 1: VLAN ID mismatch on the uplink

The Tier-0 uplink interface sits on a VLAN segment, and that VLAN has to arrive tagged at the ToR interface where the BGP peer IP lives. If the VLAN ID differs anywhere on the path, the two sides never see each other and the session stays in Idle or Active. Check every hop:

The VLAN ID of the NSX uplink segment used by the Tier-0 interface.
The trunk port group or VLAN range on the vSphere Distributed Switch that carries the Edge uplinks (for an Edge VM), or the Edge uplink profile and teaming for a bare-metal Edge.
The physical switch port toward the ESX host or Edge, which must allow the VLAN, and the SVI or routed sub-interface on the ToR that holds the peer IP.

Then test reachability from the Tier-0 SR context, sourced from the uplink IP:

ping <peer-ip> source <tier0-uplink-ip> repeat 5

No reply means the problem is below BGP; fix the VLAN, IP or subnet first. Add size 1472 dfbit enable to the ping to check that a full 1500-byte packet passes; an MTU mismatch typically lets the session come up and then drop when larger UPDATE messages are sent.

Cause 2: BGP configuration mismatch

If ping works but the session does not establish, compare the Tier-0 BGP configuration with the ToR line by line:

Local AS on the Tier-0 and remote AS on the ToR neighbor, and the other way round.
Neighbor IP addresses: the ToR must peer with the Tier-0 uplink IP, and the Tier-0 with the ToR interface IP (or loopback, with multihop set on both sides).
MD5 password: set on both sides or on neither.
Keepalive and hold timers, and BFD: enabled on both sides with compatible intervals.
Address families (IPv4, IPv6, EVPN) enabled on both sides.

Logs on the NSX Edge

The Edge keeps two logs that explain BGP state changes. Read them from the root shell of the Edge node.

/var/log/frr/frr.log

The routing daemon log. It records every FSM transition of a neighbor and the reason for it:

grep -E "bgp_fsm_change_status|ADJCHANGE" /var/log/frr/frr.log | tail -30
grep -i "notification" /var/log/frr/frr.log | tail -30

A good session ends with the peer going from OpenConfirm to Established and an %ADJCHANGE: neighbor <peer-ip> in vrf default Up line. When it goes down, look at the reason next to it:

Message in the log	Meaning
Network or config error	General failure; check connectivity, the VLAN path, firewall rules and the configuration.
NOTIFICATION code 4 (Hold Timer Expired)	No keepalive or update within the hold time: packet loss, congestion, MTU, or a peer that stopped answering.
NOTIFICATION code 2 (OPEN Message Error)	The OPEN was rejected, for example a wrong peer AS (subcode 2).
Administratively down (NSXA_OP_STATE_DOWN)	The neighbor is disabled in the NSX configuration.
BFD down	BFD detected the path failure first and took BGP down with it; check `get bfd-sessions`.

/var/log/syslog

The NSX routing service raises an alarm for each BGP state change of a Tier-0 SR. Filter it to get a clean timeline per peer:

grep "state=BGP" /var/log/syslog

Each line names the peer IP, the SR and the new state: BGP_UP, BGP_DOWN or BGP_DELETE. Match the timestamps with frr.log and with the ToR log to see which side dropped first.

Tag: BGP

NSX BGP Neighbor Not Established: Troubleshooting Tier-0 BGP on the NSX Edge