One of the important new features of the AK8 code is the ability
to use multiple IP addresses on the same physical network port. This feature is called VNICs, or Virtual NICs. This allows us
to no longer "burn" a whole port in a cluster when one cluster peer owns a network port. Traditionally, we have had
to leave Net0 empty on controller 2, because it was used for managing controller 1. Vise-versa for Net1 on Controller 1. Then, if you have data going over 10GigE ports, you probably only had half of your ports running at any given time, and the partner 10GigE port on the other controller just sat there, doing nothing, unless the first controller went down.
What a waste. Those days are over.
I want
to thank and give a big shout-out
to our good partner, OnX Enterprise Solutions, for allowing me
to come into their lab and play around with their 7320
to do this demo. They let me make a big mess of their lab for the day as I played around with VNICs. If you're looking for a partner who knows Oracle well and can also piece together a solution from multiple vendors
to get you what you need, OnX is a good choice. If you would like
to talk
to your local OnX rep, you can contact Scott Gill at
[email protected] and he can point you in the right direction for your area.
Here we go:
Here is what your Datalinks window looks like BEFORE you upgrade
to AK8.
Here's what the same screen looks like after you upgrade. See the new box?
So here is my current network setup. I have my 4 physical interfaces setup each with an IP address. If I ping them, no problems.
So I can ping 180, 181, 251, and 252. However, if I try
to ping 240, it does not work, as the 240 address is not being used by any of these interfaces, right?Let's change that.
Here, I'm going
to make a new Datalink by clicking the Datalink "Plus sign" button. I will check the VNIC box and tell it
to use igb2, even though another interface is already using it.
Now, I will
create a new Interface, and choose "v_dl2" for it's datalink.
My new network screen looks like this. A few things
to take note of here. First, when I click the "igb2" device, it only highlights dl2 and int2. It does not highlight v_dl2 or v_int2.I think it should, but OK, it looks like VNICs don't highlight when you click the device. Second, note
how the underscore character in v_dl2 and v_int2 do not seem
to show on this screen. You can see it plainly if you go in and edit them, but from here it looks like a space instead of an underscore. Just a cosmetic bug, but something
to be aware of.
Now, if I click the VNIC datalink "v_dl2", on the other hand, it DOES highlight the device it belongs
to, as it should. Seen here: Note that it did not, however, highlight int2 with it, even though int2 is connected
to igb2. That's because we clicked v_dl2, which int2 has nothing
to do with. So I'm OK with that.
So let's try pinging 240 now. Of course, it works great.
So I now make another VNIC, and call it v_dl3 using igb3, and v_int3 with an address of 241. I then setup three shares, using ports 251, 240, and 241.Remember that IP 251 and 240 both are using the same physical port of igb2, and IP 241 is using port igb3.
Next, I copy a folder full of stuff over
to all three shares at the same time. I have analytics going so I can see the traffic. My top chart is showing the logical interfaces, and the bottom chart is showing the physical ports.Sure enough, look at the igb2 and vnic1 interfaces. They equal the traffic going over the igb2 physical port on the second chart. VNIC2, on the other hand, gets igb3 all
to itself.
This would work the same way with 10Gig or Infiniband ports. You can now have multiple IP addresses and even completely different subnets sharing the same physical ports. You may need
to make route table entries for that. This allows us
to use all of the ports you paid for with no more waste.
Very, very cool.
One small "bug" I found when doing this. It's really not a bug, it was designed
to do this when VNICs were not around. But now that we have NVIC capability, they should probably change this. I've alerted the engineering team about this and they're looking into it, so perhaps it will be fixed in a later code.
Here it is. Remember when we made the new VNIC datalink, I specifically said
to click on the "Plus Sign" button
to create it? I don't always do that. I really like
to use the drag-and-drop method
to create my datalinks in the network screen.HOWEVER, if you were
to do that for building a VNIC, it will mess you up a little. Watch this.
Here, I'm dragging igb3 over
to make a new datalink. igb3 is already being used by dl3, but I'm going
to make this a VNIC, so who cares, right?
Well, the ZFSSA does not KNOW you are going
to make it a VNIC, now does it? So... it works as designed and REMOVES the igb3 device from the current dl3 datalink in the background. See
how it's now missing? At the same time, the dl3 datalink choice is missing from my list of possible VNICs for me
to choose from!!!! Hey!!! I wanted
to pick dl3. Why isn't it on the list??? Well, it can't be on this list because dl3 no longer has a device associated with it. Bummer for you.
When you click cancel, the device is still missing from dl3.
The fix is easy. Just edit dl3 by clicking the pencil button, do absolutely nothing, and click "Apply". The device will magically come back.
Now, make the VNIC datalink by clicking the "Plus Sign" button. Sure enough, once you check the VNIC box, dl3 is a valid choice. No problem.
That's it for now. Have fun with VNICs.
