EXALOGIC: INFINIBAND: How to check if all Exalogic Compute Nodes are connected correctly to both the IB Switches

For checking if all Exalogic Compute Nodes are connected correctly to both the IB Switches, run below command on one of the IB Switches in the Fabric.

ibnetdiscover | grep -A 2 "Ca" | egrep -i -A 2 "<node 1 >|<node 2>|.....|<node N>"

For e.g. if you have four compute nodes in the rack with names el01cn01, el01cn02, el01cn03 and el01cn04 your command will look as follows:

ibnetdiscover | grep -A 2 "Ca" | egrep -i -A 2 "el01cn01|el01cn02|el01cn03|el01cn04" 

Output will be in below format

Ca 2 "H-00XXXXXXX" # "< CN/SN Hostname> EL-C <CN/SN IP> HCA-1" 
[1](XXXXXX) "S-00XXXXXX"[<Switch Port>] # lid <lid #> lmc 0 "SUN IB QDR GW switch <Switch 1> <Switch 1 IP>" lid <lid #> 4xQDR 
[2](XXXXXX) "S-00XXXXXX"[<Switch Port>] # lid <lid #> lmc 0 "SUN IB QDR GW switch <Switch 2> <Switch 2 IP>" lid <lid #> 4xQDR  

Below is sample output which shows el01cn01 connected to two IB Switches el01gw01 & el01gw02 correctly.

Ca 2 "H-00XXXXXX128bf08" # "el01cn01 EL-C XX.XX.XX.XX HCA-1" 
[1](XXXXXX128bf09) "S-00XXXXXXXXbac0a0"[6] # lid 5 lmc 0 "SUN IB QDR GW switch el01gw01 XX.XX.XX.XX" lid 41 4xQDR 
[2](XXXXXX128bf0a) "S-00XXXXXXX728c0a0"[6] # lid 6 lmc 0 "SUN IB QDR GW switch el01gw02 XX.XX.XX.XX" lid 11 4xQDR   

Products to which Article Applies
Exalogic racks using Infiniband Switches

Article Author: Tarun Boyella