I know many of you have been working on Oracle's Exalogic and other Engineered Systems. With partitions enabled now, things have gone multi dimension. But its fun. Isn't it ?

While you have some EoIB configurations together with InfiniBand partitions, the VNICs are not coming up and staying in WAIT-VHUB state ? 

Chances are that you have forgot to add InfiniBand Gateway switches' Bridge-X port GUIDs to your partition. These must be added as FULL members for EoIB to work properly. VHUB means a virtual hub in EoIB. Bridge-x is the access point for hosts to work over EoIB so thats why it must be a full member in partition.

Step 1: Find out the port GUIDs of your bridge-x devices in IB Gateway switch.

# showgwports
INTERNAL PORTS:
---------------
Device   Port Portname  PeerPort PortGUID           LID    IBState  GWState
---------------------------------------------------------------------------
Bridge-0  1   Bridge-0-1    4    0x0010e00c1b60c001 0x0002 Active   Up
Bridge-0  2   Bridge-0-2    3    0x0010e00c1b60c002 0x0006 Active   Up
Bridge-1  1   Bridge-1-1    2    0x0010e00c1b60c041 0x0026 Active   Up
Bridge-1  2   Bridge-1-2    1    0x0010e00c1b60c042 0x002a Active   Up

Step 2: Add these port GUIDs to the IB partition associated with EoIB. Login to master SM switch for this task.

# smpartition start
# smpartition add -pkey <PKey> -port <port GUID> -m full
# smpartition commit

Enjoy !