M. Sean McGee - Latest Comments

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

David Owen — Mon, 30 Apr 2012 05:56:10 -0000

Sean,

Do you know if there is a slide (3rd image down) that includes the 6296UP comparison in existance?

Re: The “Mini-Rack” Approach To Blade Server Design

generalnetworkerror — Tue, 24 Apr 2012 03:53:14 -0000

Too bad allegiances and politics in favor of one vendor, specifically one beginning with H and ending in P, trumps design and performance superiority. System Engineers these days tend to look down upon the Network Engineers, so how the heck can some network vendor come along and do computing? Mind you, they see no issue with HP dominating in networking especially with 3Com in the mix (which is a bit of a joke to me). This reminds me of how scared our SA's are from doing iSCSI, so they'll go to their grave with FC.

Great post. Cleared up the fundamental differences. Looking to find posts on perf comparisons next.

Re: The “Mini-Rack” Approach To Blade Server Design

Xeon Blade Server — Thu, 15 Mar 2012 08:40:01 -0000

It is a great approach to explain on the blade server. It helps admin to understand the differences IP addresses for blade servers.

Re: The Tale of Two Cisco UCS Predictions

Ricardo Cabeza — Sat, 18 Feb 2012 10:57:01 -0000

I wouldn't implement a solution unless it had an IBM tape library. You don't have anything unless you can hold it in your hand.

Re: The Cisco UCS B230 – the Goldilocks Blade Server

Frank — Thu, 16 Feb 2012 12:15:45 -0000

Hey Sean,

With the new B230 M2 E7 chip and the ability to run 1333 DIMMS, have you seen any good results in the field using this for VDI?

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Berend Schotanus — Fri, 20 Jan 2012 08:09:06 -0000

Hi Sean,

Very interesting stuff. I have a question about maximums. Can you tell me anything about the total number of vNIC's that is supported by UCS Manager 2.0? I have read the Cisco VM-FEX Best Practices for VMware ESX Environment Deployment Guide, but is their a way to calculate this.

For instance when I have 1 5108 chassis with 2 uplinks per 2208 to the 6248 using 4 B200M2 blades with M81KR VIC using the normal VM-FEX mode.

We will be using vSphere 5.0

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

salit — Thu, 15 Dec 2011 08:47:42 -0000

Hi Sean,

I need a clarification regarding bandwidth.
Each server has 80 Gbps (64Gbps because of PCIe bus limitation), that is a very high value. The total downlink capability of IOM is so 640Gbps.
The two IOM has 160Gbps uplink capability, but if you consider that 0% of the traffic is resolved internally because of FEX way of working (it seems that switching is done at FI level) the 160Gbps is a limitation. Considering that all traffic must exit and re-enter in the IOM if all server are using network the real troughtput is 20Gbps for each server.
This is to the large due btween uplink and downlink plus the FEX passthru way of working.

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Azzafir — Tue, 22 Nov 2011 06:42:24 -0000

Hi Guys,
I was wondering if i can do away with FC ports on the UCS Fabric Interconnect and use Pure FCoE(10G cable) to support my SAN and LAN needs. Right now I am having 2 connections to my Nexus 5548UP-L3 with Storage License.
1 connection is the 10G ethernet for my LAN and the other is an 8Gb FC which i also connect to the Nexus 5548UP for my SAN. I was wondering if I could consolidate and just use a single FCoE 10Gbps to support my LAN and SAN needs.

Thanks guys. Went thru many documents but it seems that they use separate FC and 10G links on for the SAN and LAN when it comes to the UCS. I am not going for iSCSI solution :)

Thanks all.
Azzafir Patel.

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Jason — Wed, 14 Sep 2011 14:48:41 -0000

Hi Sean,
I love the blog and am happy to hear about the UCS advancements. I'm not a vendor, but a customer, not a Server guy, but a Network guy. I was, however, interested in an answer to Cam from xsigo's (albeit loaded) question about FCoE throughput across a VIC 1280 port-channel. Does the VIC similarly port-channel the FCoE traffic from each server so that the practical FC throughput from each server will exceed the 4 Gbps/fabric available in the Palo adapter? (that would be very advantageous in my opinion)

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Bill Graham — Fri, 19 Aug 2011 23:50:06 -0000

Hello Sean, Full disclosure, I'm a former Juniper SE in the service provider space but now a potential UCS customer. I'm serious about finding the best of breed elements to build a private cloud computing solution. I've inherited a small investment in an IBM BladeCenter/BNT solution and was pinning my hopes on swapping out BNT and FC pass-through w/Brocade FC ToR for the BladeCenter integrated Brocade Converged Switch. Unfortunately, the thing runs two different CLIs despite being branded a "converged switch". Need I say more? Just a little background on why I'm down the UCS path.

What are your thoughts about UCS 2.0 for a Q1 2012 greenfield deployment? Are there going to be any advantages to using the 1st generation hardware and 1.4 software given my timeline? I'm not familiar with the UCS software development cycle. I've seen scenarios, unrelated to UCS, where the hardware starts development based on v1.3 of software (current at dev time) but then 1.4 is released while the hardware is still in development or systest. So, the new hardware ends up at FRS/FCS running 2.0 with all features up to v1.3, new features specific to the new hardware, but must wait for a later 2.x release to catch-up with 1.4 features. I'm sorry for such a long wind-up to ask: Any feature parity gaps when using UCS 2.0 hardware at FRS? Thanks in advance. Bill

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Difernan — Fri, 29 Jul 2011 15:07:26 -0000

Hey Sean, if you need help let me know, AS UCA practice difernan at cisco dot com

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Ian Erikson — Thu, 28 Jul 2011 11:44:24 -0000

Support for Layer 2 Disjoint Networks in End Host Mode....this is great for our setup- I don't have to use two different chassis to combine our DMZ into one UCS domain- or change to switching mode and lose my bandwidth.

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Jeff Allen — Mon, 25 Jul 2011 22:28:42 -0000

I think it's safe to say I enjoyed the comments almost as much as the article. The Xsigo/Egenera gang certainly keep life interesting. It's got to be tough having to teach people how to pronounce your company name at the start of each preso...

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Sun, 24 Jul 2011 00:21:07 -0000

Hi Chris,
You sound just like many of our biggest UCS customers before they had a chance to get into the details of UCS and see it in action. :) I highly recommend a face-to-face meeting with Cisco - and I'd be more than glad to help arrange it if you'd be open to it. I won't promise that UCS will solve all of your problems or that it's the best solution for you. I will promise that a face-to-face meeting will help you understand UCS much better than any blog or online research can.

If UCS was just another blade offering, I don't think we'd see the kind of success that it's had in the industry. Data center teams (server/lan/san teams) aren't adopting UCS just because it's a Cisco product. In fact, many times we have to help server teams get past that issue. :) They find out that UCS is a solution like no other and most find that it helps them solve real problems in their environment.

I'd love to at least help you understand UCS a little better so that you can make an informed decision as to whether it's the right solution for you or not. Please reach out if you'd like to chat (seanmcge at cisco dot com).

Best regards,
-sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Sun, 24 Jul 2011 00:04:36 -0000

Now those are good questions, Cam. Glad to see we're finally on the same page.

The power and price questions will be answered publicly closer to product launch. The bandwidth question was answered in detail in my response to Egenera below.

Best regards,
Sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Chris_Donohoe — Fri, 22 Jul 2011 09:19:43 -0000

Nevermind. I answered my own question with some research. I know it will look like I'm throwing rocks, but Cisco UCS looks like it's just another blade offering that doesn't supply much additional functionality (if any). Considering that, I don't know why anyone would want to consider Cisco UCS over HP Bladesystem unless it was for a very niche purpose at a great price. Even then, I'm risking my environment on a first generation product as opposed to a seventh generation product. I've used Cisco as a network solution for a very long time, but I don't know if I'll ever be ready to consider them for a server solution. It's like going to McDonalds and asking for a chicken sandwich.

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Cford — Fri, 22 Jul 2011 08:53:49 -0000

Ok, as you suggest....lets uplevel it and actually answer the questions.....

What is the power?
What is the price?
What is the bandwidth?

Just saying no doesnt add any credibility to your story either.....

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Thu, 21 Jul 2011 22:11:26 -0000

Hi Chatinbox,
UCS Manager added a couple of features also - iSCSI Boot, disjoint L2 connectivity, and VM-FEX for RedHat KVM.

Thanks for stopping by!
-sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Thu, 21 Jul 2011 22:00:34 -0000

Hi Cam,

Wow, you've really outdone yourself this time. This year I think you'll definitely have a good shot at the Oscar award for Best Competitive FUD Rant. Only one problem is that competitive Fear, Uncertainty and Doubt (FUD) is usually based on a least mostly CORRECT information. So, that might hurt your chances. You REALLY need to consider attending a UCS Training class if you're going to keep coming back to "educate the masses" on my blog. As Xsigo's Director of Product Management, it's exposing how much you guys truly don't understand about Cisco's solution or our capabilities. That has to impact your credibility with a would-be customers. Let me know if you're interested in a UCS class and I can help find one for you. It would really save me a lot of time in responses. ;)

For example:
- no, the 6248 doesn't "drop the bandwidth" (since its comparable to the 6120 as the article plainly states)
- no, Palo is not limited to 4G
- no, PCIe gen 2 x16's limit is not 52g
- no, we don't have thermal issues
- no, the 2104/2208 is not a switch
- no, our adapter doesn't work in "active passive mode"
- no, we're not "reordering FCoE" frames
- your power numbers are way off
- your $$ estimates are ever farther off
- no, there's nothing wrong with being open and honest with the customer about today's software capabilities vs. future software capabilities (based on hardware limits or being "ready" for future capabilities)
- etc.

In all seriousness, it's really unfortunate when a conversation so rapidly deviates from healthy competitive discourse and heads headlong into disingenuous competitive mudslinging. I honestly think it doesn't do anyone any good - not for either the vendors or the customers. Any chance we can bring it up a level? Maybe not all the way up to the "high road" (I realize that would be a stretch) but at least up to the feeder road and out of the gutter? I really don't mind answering serious questions, like the post from Egenera, but your post is in a completely different category.

Sincerely and Respectfully,
Sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Chatinbox — Thu, 21 Jul 2011 10:08:36 -0000

Do you have any information on management capabilities & design of ucs manager.looking at the buzz for ucs,wanted to learn if there are any innovations in ucsm also

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Wed, 20 Jul 2011 22:33:25 -0000

Hi Buzz@Egenera.com,
Thanks for stopping by.

1. In the article, I specifically said "Either side A or side B (diagram below) can burst up to 40 Gbps each, but, simultaneously, cannot exceed the PCIe Gen 2 x16 bus limitation of 64 Gbps.". Did you miss that section? I tried to make that abundantly clear - I'm an engineer, not a marketeer. I have to stand in front of customers and defend what I write, so, it serves me no purpose to over-promise and under-deliver. I even included a comment about anticipating competitive FUD. With my obvious ability to predict the future, my stock portfolio should look better than it does... :)

2. Our testing shows that a customer can achieve up to 39.6 Gbps on a single fabric (side A or side B). That means that while a single VIC's total throughput is limited by the PCI Gen 2 x16 bandwidth capability of 64 Gbps (after accounting for 8b/10b encoding overhead), either side can burst up to almost the full 40 Gbps.

3. The PCIe 2.0 x16 speed is 64 Gbps... *per direction*. (http://www.pcisig.com/news_roo....

4. Don't make the mistake of assuming that the total bus bandwidth is equally divided across all 8 ports and no port can ever exceed it's share of the 64 Gbps per direction. In data center networking, hardly any high throughput traffic pattern is sustained. As such, our design provides lots of throughput for sustained flows across all 8 ports (subject to PCIe 2's limitation) or periodic bursting on a single port or single fabric up to the interface's theoretical maximum.

5. Yes, both Fabrics (both 4x 10 Gbps paths) can be active at the same time, subject to the total throughput limitation plainly called out in the article and in 1 & 2 above.

6. If you want more than 64 Gbps on a single server, deploy a full width UCS server (B250 or B440) that can accommodate up to 2 x VICs. Let me know if you have an interested customer and I'll discuss it with'em. =]

7. It's obvious that Cisco has engineered an adapter ASIC in anticipation of PCIe 3.0 (http://www.pcisig.com/news_roo..., which doubles gen 2's throughput. Don't hold it against us for being prepared early...

8. When the specs on the B230 were released, the only available adapters were dual 10 Gbps. Therefore, the Product Manager said it was limited to 20 Gbps. That limit was caused by the adapter, not the B230's capabilities. We believe in "truth in advertising". :) If the PM had said the B230 was capable of up to 64 Gbps but we only had a 2x 10 Gbps card available at the time, we'd have been called out on that instead.

Once it ships, installing a 1280 VIC in a B230 would obviously provide more than 20 Gbps of throughput. See 1 & 2 above.

9. Do you know of any customer pushing more than 64 Gbps of sustained bandwidth on a 2 proc Nehalem platform? Most likely not. Like I said in the article, let me know and I'll buy them a beer. In other words, this whole conversation regarding 64 Gbps vs. 80 Gbps from a single adapter is really a moot point for customers. Let's call it "NIC-picking (sic) from UCS competitors". ;)

10. I really want to have a tenth point. I hate ending on an odd number. However, I already feel like we've beat this horse to death and I'm at risk of triggering some folk's narcolepsy.

Bottom line is that the 1280 VIC provides the most throughput of any single x86 blade mezzanine Ethernet adapter. The 1280 VIC eliminates the need to deploy a ton of mini-rack (http://bit.ly/chQYnw) switch modules in a single blade chassis just to get more 10 Gbps lanes. Cisco UCS customers now have it as an option in addition to all the other options in the UCS CNA portfolio.

Again, I do sincerely appreciate you taking the time to post. It helps educate our current and future UCS customers on misunderstandings caused by my poor writing skills. ;)

Sincerely,
-sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Cford — Wed, 20 Jul 2011 10:52:01 -0000

Hi Sean, cam from xsigo here....disclosure out of the way.

Seems like there are always at least 2 numbers for everything cisco announces. There are usually the numbers they announce....and the numbers that ship.....and there are rarely in the ballpark of each other. Here are a few areas I came across purusing your docs....maybe you can clear this up for all of is.

The pics above say that the 6248 doubles the bandwidth. The 6140 already has over 1 tb of bw, and the new 6248 actually drops the bandwidth down to 960 gb.....seems like you are going the wrong direction. You do gain 1u per box,.....though for those of you in need of an extra 2u in your rack.

A picutr above shows the new 6249 as supporting 4096 vlans, but the spec sheet only states 1024....I am assuming this is standard cisco marketing......hw supports what the software cannot deliver...correct?.....kind of like the Vic numbers.

I didn't see the power specs for the 2208... Can you help? Seems to me that with the new 4x10g lag, thebinterior ports will need to be 32x10g with another 8 ports for uplink....so this is a 40 port 10g switch.....pluss all of the chassis management hardware. So it is roughly equivalent to the 6248 switch from a hardware perspective. The 6248 is showing peak peer at 600w. Are these blades also 600w? What will the price be for this 40 port 10 g switch? Gonna need to ne around 25k or so to make those 70% cisco margins.

Let's talk about your new Vic. Sounds like an impressive engineering feat to squeeze 8 ports of cna into a single asic. Bet that puppy is pretty hot. The old palo adapter was listed at 18w.....and this new adapter is listed at 18w "typical"......I find that hard to believe. Your 6248 switch says 600 w peak and 350w typical.....that comes out to about 7w per port for typical and 12 w peak. Nics and especially cnas are usually hotter than switch ports....but even at these numbers, they don't add up. Even at 7 w per port, this sucker should be over 50 w.

Let's also nip this 80 g in the bud. I believe your adapter typically works in active passive mode with hardware failover....correct? So you are really talking about 40g of throughput per your numbers. The pci bus is x16 whic really can only support about 52g of actual packet throughput.....so your 40g adapter should be fine as demonstrated. Also, with the lag, any single stream can only use 10g.....but if you have multiple flows per nic, they can spread across the multiple interfaces...is this correct?

What about storage bandwidth? The old palo adapter couldnonly push about 4g of fcoe.....what will this new adapter support? Does the lag spread the fcoe traffic as well or treat it as a single flow?...gotta thing reordering fcoe is not a great idea...so I imagine a single hba is always a single flow....besides it will be pinned to an outbound 8 g port.

Switching bw.....all switching between servers happens in the 6248.... Correct? So I can cab switch full 40g across any 2 servers on the same chassis....but that's about it. Also, in order to use that bandwidth, I would need all 8 uplinks available....that means 16 cables for each blade chassi....or 2 cables per server.....how is that cable consolidation......might as well use rack servers and save a lot of money.

With 8 uplinks to a 6248 per chassis....looksnlike I top out at 4 chassis.....not very scalable is it? Speaking of scalability.....the old 6140 claimed support for 40 chassis....but the new 6248 only claims 20 chassis.....you guys going backwards......or is this just another case of the actualnproduct not living up to the marketing hype.....pernmost cisco products.

Lat one....seems like this stuff is "ready" for a lot of stuff.....l3 ready, fabric path ready, 40g ready....so maybe you can clarify for us exactly what this stuff will actually do when it actually ships....and when that might actually be.

Thanks...looking forward to the straight facts.....or maybe just some more marketing hype....your blog....your choice

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Buzz — Wed, 20 Jul 2011 00:08:44 -0000

This doesn't add up......8 x 10Gb downlink ports on each VIC 1280?
The VIC 1280 adaptor is specified as a PCIe 2.1 16x interface. This is the interface to the motherboard and processor/memory complex. This PCIe 2.1 16x interface has a maximum throughput of 64Gb/s. ( encoded it is 80Gb/s but typically un-encoded is used). The VIC 1280 is also specified as having 8 x 10Gb ports which results in 160Gb/s of bi-directional throughput. The interface to the motherboard would need to support 160Gb/s to service the 8 x 10Gb ports at full throughput. So, it seems that although the VIC 1280 has 80Gb/s of potential throughput, the PCIe x16 would limit the total for the VIC 1280 to 64Gb/s - across 8 ports...that's actually 8x 4Gb/s ports ( 8Gb/s bi-directional time 8 = 64Gb/s).And, the Cisco UCS B230 blade specification says "One dual-port mezzanine slot for up to 20 Gbps of redundant I/O throughput"....optimistically, this may mean 40Gb/s .....if the motherboard interface is limited to 40Gb/s, then the VIC1280 increased throughput does not seem like it can be realized ?Maybe there are new blades coming that will allow tapping this performance, but it doesn't seem that the current crop of blades will allow tapping this new throughput..

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

M. Sean McGee — Tue, 19 Jul 2011 18:28:06 -0000

Hi Dan,
Great question and I'll get to it in a sec. First, I want to say that it's ok that you're asking questions about UCS even though you seem to be from HP (posting from IP address 15.195.201.91/zswa01cs006-da01.atlanta.hp.co...). I don't mind answering anyone's questions - even questions from a competitor. I do prefer that everyone is upfront about being a customer, a partner, or a vendor, though. Transparency keeps everyone honest in this open forum.

In regards to your question... There are two numbers: A)the number of logical interfaces supported by the VIC hardware (256) B)the number of logical interfaces the Fabric Interconnect will allow the VIC to instantiate (116 at FCS). Since the VIC is effectively a remote line card to the Fabric Interconnect (FI), the FI 'owns' the virtual port once it's created and controls the number of interfaces that each VIC can create.

At FCS, UCS Manager will limit (in software) the number of usable/definable virtual interfaces per VIC to 116. As customer demand requires that number to grow, we'll increase the supported number of interfaces up to the VIC hardware limit of 256.

Again, great question and I hope I was able to clearly answer it for you.

Best regards,
-sean

Re: UCS 2.0: Cisco Stacks the Deck in Las Vegas

Dan — Tue, 19 Jul 2011 13:55:57 -0000

Confused on the VIC stats.
All the Sldieware says 256 but your text says 116.
Am I missing something?