One of the important new features of the AK8 code
is the ability to use multiple IP addresses on the same physical network port. This feature
is called VNICs, or Virtual NICs. This allows us to no longer "burn" a whole port in a cluster when one cluster peer owns a network port. Traditionally,
we have had to leave Net0 empty on controller 2, because it was used for managing controller 1. Vise-versa for Net1 on Controller 1. Then, if you have data going over 10GigE ports, you probably only had half of your ports running at any given time, and the partner 10GigE port on the other controller just sat
there, doing
nothing, unless the first controller went down.
What a waste. Those days are over.
I want to thank and give a big shout-out to our good partner, OnX Enterprise Solutions, for allowing me to come into their lab and play around with their 7320 to
do this demo. They let me make a big mess of their lab for the day as I played around with VNICs. If you're looking for a partner who knows Oracle well and
can also piece together a solution from multiple vendors to get you what you need, OnX
is a good choice. If you would like to talk to your local OnX rep, you
can contact Scott Gill at
[email protected] and he
can point you in the right direction for your area.
Here
we go:
Here
is what your Datalinks window looks like BEFORE you upgrade to AK8.
Here's what the same screen looks like after you upgrade. See the new box?
So here
is my current network setup. I have my 4 physical interfaces setup each with an IP address. If I ping them, no problems.
So I
can ping 180, 181, 251, and 252. However, if I try to ping 240, it does not work, as the 240 address
is not being used by any of these interfaces, right?Let's change that.
Here, I'm going to make a new Datalink by clicking the Datalink "Plus sign" button. I will check the VNIC box and tell it to use igb2, even though another interface
is already using it.
Now, I will create a new Interface, and choose "v_dl2" for it's datalink.
My new network screen looks like this. A few things to take note of here. First, when I click the "igb2" device, it only highlights dl2 and int2. It does not highlight v_dl2 or v_int2.I think it should, but OK, it looks like VNICs don't highlight when you click the device. Second, note how the underscore character in v_dl2 and v_int2
do not seem to show on this screen. You
can see it plainly if you go in and edit them, but from here it looks like a space instead of an underscore. Just a cosmetic bug, but something to be aware of.
Now, if I click the VNIC datalink "v_dl2", on the other hand, it DOES highlight the device it belongs to, as it should. Seen here: Note that it did not, however, highlight int2 with it, even though int2
is connected to igb2. That's because
we clicked v_dl2, which int2 has
nothing to
do with. So I'm OK with that.
So let's try pinging 240 now. Of course, it works great.
So I now make another VNIC, and call it v_dl3 using igb3, and v_int3 with an address of 241. I then setup three shares, using ports 251, 240, and 241.Remember that IP 251 and 240 both are using the same physical port of igb2, and IP 241
is using port igb3.
Next, I copy a folder full of stuff over to all three shares at the same time. I have analytics going so I
can see the traffic. My top chart
is showing the logical interfaces, and the bottom chart
is showing the physical ports.Sure enough, look at the igb2 and vnic1 interfaces. They equal the traffic going over the igb2 physical port on the second chart. VNIC2, on the other hand, gets igb3 all to itself.
This would work the same way with 10Gig or Infiniband ports. You
can now have multiple IP addresses and even completely different subnets sharing the same physical ports. You may need to make route table entries for that. This allows us to use all of the ports you paid for with no more waste.
Very, very cool.
One small "bug" I found when doing this. It's really not a bug, it was designed to
do this when VNICs were not around. But now that
we have NVIC capability, they should probably change this. I've alerted the engineering team about this and they're looking into it, so perhaps it will be fixed in a later code.
Here it
is. Remember when
we made the new VNIC datalink, I specifically said to click on the "Plus Sign" button to create it? I don't always
do that. I really like to use the drag-and-drop method to create my datalinks in the network screen.HOWEVER, if you were to
do that for building a VNIC, it will mess you up a little. Watch this.
Here, I'm dragging igb3 over to make a new datalink. igb3
is already being used by dl3, but I'm going to make this a VNIC, so who cares, right?
Well, the ZFSSA does not KNOW you are going to make it a VNIC, now does it? So... it works as designed and REMOVES the igb3 device from the current dl3 datalink in the background. See how it's now missing? At the same time, the dl3 datalink choice
is missing from my list of possible VNICs for me to choose from!!!! Hey!!! I wanted to pick dl3. Why isn't it on the list??? Well, it
can't be on this list because dl3 no longer has a device associated with it. Bummer for you.
When you click cancel, the device
is still missing from dl3.
The fix
is easy. Just edit dl3 by clicking the pencil button,
do absolutely
nothing, and click "Apply". The device will magically come back.
Now, make the VNIC datalink by clicking the "Plus Sign" button. Sure enough, once you check the VNIC box, dl3
is a valid choice. No problem.
That's it for now. Have fun with VNICs.
