Datacenter Concepts: Cisco UCS - vNIC/vHBA Allocation Scheme

vNIC/vHBA Allocation Scheme

Vnic to mezz card placement happens in two stages:

1. vCon to Adaptor Placement (Round Robin & Linear)
2. vNIC to vCon Placement

1. vCon to Adaptor Placement (Round Robin & Linear)

Round-Robin (the only option with UCSM version 1.4.1 and earlier)

No of Mezz Cards	VCon-1	VCon-2	VCon-3	VCon-4
1	1	1	1	1
2	1	2	1	2
3	1	2	3	2
4	1	2	3	4

In a server with 1 pcie adaptor card, all the 4 VCons point to the same adaptor card.
In a server with 2 pcie adaptor cards, VCon-1 and VCon-3 point to adaptor-1, VCon-2 and VCon-4 point to adaptor-2.
With 3 pcie cards, VCon-1, VCon-2 and VCon-3 points to the corresponding adaptors 1, 2 and 3. VCon-4 points to adaptor-2.
With 4 pcie cards, VCon-1, 2, 3, and 4 points to adaptors 1, 2, 3 and 4 in order.

In the above example, the adaptor ids are not the pcie slot numbers of the adaptors cards, but the id assigned to them during server discovery. VCon to adaptor placement is not dependent on the pcie slot number of the adaptor card.

Note that in case of 3 adaptors, 4^th VCon gets mapped to adaptor-2 instead of adaptor-1. This will change with future implementations of VCons.

Linear (available with UCSM 1.4.2 onwards)

Round Robin causes problems with vnic pcie orders as newer adapter are added to the server on a later stage. For solving this ordering issue, placement algorithm needs to get changed from round-robin to linear. Example,

No. of vnics placed per VCon

VCon-1 vnic-1, vnic-2

VCon-2 vnic-3, vnic-4

VCon-3 vnic-5, vnic-6

VCon-4 vnic-7, vnic-8

In order for the host to see the vnics in expected orders i.e. vnic-1, 2, 3, 4, 5, 6, 7, 8 on the pcie bus and for VCons to get properly distributed across adaptor cards, placement would happen like this.

VCon-1, VCon-2 goes to adaptor-1.

VCon-3, VCon-4 goes to adaptor-2.

This algorithmic change poses some backward compatibility issues. Upgrading from 1.4.1 to the release which will address this problem will cause host reboots because vnic/vhba placements will change. In order to minimize the impact the behavior would be made like the following:

Service profiles created before the release which will address this issue will keep using the old vcon-mezz placement algorithm (Round Robin) for their lifetime. This will preserve vnic placements on upgrade hence their servers will NOT reboot on upgrade to the newer release. New service profiles will use the new algorithm.

All the new service profiles created in the newer release will cause server reboots on downgrading to lower releases which have the old vcon-mezz placement algorithm.

On upgrade to the newer release as the service profiles created in lower releases will keep using the old algorithm, vnic placement would be preserved for them hence will NOT cause server reboots on downgrade.

2. vNIC to vCon Placement

Explicit assignment by the user.
Implicit assignment i.e. Depending on system to do the assignment.

Issues with using Explicit VCon Assignment

Explicit assignments of vnic/vhba to VCon-3 and VCon-4 results in pcie ordering issue. In UCSM 1.4.1 & earlier releases, VCon to adaptor placement happens in a round-robin order (See Below Table ).

Example: In a server with two adaptors VCon-1 and VCon-3 will point to adaptor-1 and VCon-2 and VCon-4 will point to adaptor-2.

	Round-Robin		Linear
vNic	Desired VCon	Actual VCon	Desired Order	Actual Order
vnic-1	1	1	1	1
vnic-2	2	2	2	1
vnic-3	3	1	3	2
vnic-4	4	2	4	2

Assuming host scans first adaptor-1 then adaptor-2, the vNics will get placed on the pcie bus in the order vnic-1, vnic-3, vnic-2, vnic-4. Hence pcie orders are different then what user requested. This case is rare though and can occur only in the following scenarios:

1. With 2 adaptors, User explicitly assigns VNics to 4 different VCons.
2. If a service profile is migrated from a blade with higher number of adaptors to a blade with lower number of adaptors.

The workaround is to let system do the placement. System has the intelligence to do the placement only on those VCons which have a corresponding adaptor with the same id. Hence in a server with 2 adaptors system will not place VNics on VCon-3 and VCon-4. In the same way on a server with 3 adaptors, system will not place VNics on VCon-4.

References:- VCon Specs and http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-1/b_UCSM_GUI_Configuration_Guide_2_1/b_UCSM_GUI_Configuration_Guide_2_1_chapter_011110.html#concept_FA2F616B0B9A47DB9FEEAE63D5995CDE