Quicksearch
Disclaimer
The individual owning this blog works for Oracle in Germany. The opinions expressed here are his own, are not necessarily reviewed in advance by anyone but the individual author, and neither Oracle nor any other party necessarily agrees with them.
< XEN-Windowsvirtualisierung mit ZFS-snapshotbasierter Instanzprovisionierung ;) | What have we learned from history? >
SAN is dead
Wednesday, March 7. 2007
SAN is dead. Okay, itīs a bold statement, but i think there are several factors, that will lead to an shift in storage architecture within the next few years: Most of the times you need SAN to present a block device to a system that isnīt directly attached to the server. Instead you consolidate the storage at a cental place. Okay, but do you really need a different technology for this. Perhaps the different technology is a future obstacle itself. I want to make some points.
1. You have already a network in your datacenter, itīs the local area network. Itīs questionale, why you need to build up an expensive fibre-optical network to yield meager 4 GBit/s when your networking deparment starts to deploy 10 GBit/s via Copper with 10GBASE-T respectivve 10GBASE-CX4.
2. The development of ethernet is much fast than the development of Fibrechannel. While development in FC is looking towards 8 GBit/s, the specifcation comitees in Ethernet look at 100 GBit/s right now. The speed gap will rise from 6 GBit/s right now to 92 GBits/s in the not so far feature.
3. Often people will tell you that iSCSI needs more processing power to be as fast as FC. But: With modern multicore processors like N2, Victoria Falls or Rock you have than enough cores to throw some to the task of doing the TCP/IP respective iSCSI stuff. Itīs a minimal ssue today and it getīs a non issue real soon. Take developments like Neptune (our new networking chip) into consideration, and you get the whole picture.
4. We will run into the need for higher storage speeds real soon now: Imagine a multi processor multi core system: Not the coarse multiproc systems like an Operon. I talk about systems with 16 processors with 32 Threads each. 512 Threads to feed. Each running for example at 1 GHz. (Dear press: No real system, i didnīt used Rock as an example. I know by my logs that you read here) How you want to feed some many hungry mouth ? Plugging large amounts of FC cards at 4 GBitīs into the system. Or plugging a smaller or the same amount of 10 GBit/s cards into the systems. Put 4 FC cards in it and you yield 16 GBit/s, put four 10 GBit/s cards into the system and you yield 40 GBit/s
5. It think the future massive multicore/multiprocessor systems will be surrounded by X4500 like systems coupled with iSCSI to it. Because: Condense the computing power to a few chip is only half of the story, feeding it with data is the second half.
6. Security with FC is a major pain in the a... Encryption? No generally implemented standard. Authentication? No generally implemented standard. Now take into consideration that IP has the well established standard IPsec for this task and more and more new processor designs have dedicated computing resources for encryption? Imagine a certificate based authentication of servers to the storage devices. Imagine that you can only see storage devices when you have an unrevoked and signed certificate. You can leverage this technologies from IPsec without further thinking about it.
I donīt think that the demise of FC will be a fast death. Not a one or two year horizon. By the sheer installed mass of FC it will be a long one. But changing paragims in building systems will demand a much faster development of new levels of speed than in FC technology. This is the chance of Ethernet and iSCSI. So think about a 5 year horizon.
I would like to hear you comments about my thoughts.
Trackbacks
Trackback specific URI for this entry
No Trackbacks
Comments
Display comments as
(Linear | Threaded)
Very interesting theory!
I agree in most of the points. Some years ago (at the beginning of the SAN) the idea was a "dedicated" network for the storage. In the future, the network is no longer a bottleneck- so why not use the LAN for the storage? But I think this is a long process - the whole SAN-stuff is at the top of itīs developement and of itīs spread.
I agree in most of the points. Some years ago (at the beginning of the SAN) the idea was a "dedicated" network for the storage. In the future, the network is no longer a bottleneck- so why not use the LAN for the storage? But I think this is a long process - the whole SAN-stuff is at the top of itīs developement and of itīs spread.
At the end, there is one fact, that can slow down this development: In the most SANs and LANs are administrated by different departments. The organisational change will be tricky.
Well... maybe..
Some points where I at least partially disagree (my numbers, not references to yours):
1. Some terminology nitpicking... In my book, a SAN is a concept that can be implemented using iSCSI, or FC, or both, or something else. So, I guess, what you mostly mean is: FC is dead. The physical separation of storage and regular traffic is a separate issue, I think.
2. As soon as you start running your SAN traffic over the regular LAN, maybe you have better security mechanisms in place. However: when you do that, you really need these mechanisms. If you just use FC to hook a few servers to some local storage units, you don't really need security beyond what the FC switches provide. If you use your LAN... well, then you're in trouble.
There's a reason why you separate your console and iLOM/ALOM traffic from your regular LAN, right?
3. In my mind, iSCSI and FC are conceptually almost the same, except for the variation in physical media. Seen from that conceptual viewpoint, replacing FC by iSCSI makes sense, since overall less specific hardware and software needs to be maintained.
4. If you really need to bring lots of I/O to a massively multi-core system: build an X4500 with a Niagara-II board and put some key application logic on that system (like a database server for instance).
Do you really want to throw all the raw data around? Maybe for video streaming and maybe for massively parallel processing of huge amounts of raw scientific data.
My take on this:
Maybe we shall see iSCSI replacing FC step by step. First for routed traffic, later perhaps local storage traffic.
But do we really want to mix regular and storage traffic?
I don't think so.
Expect perhaps where you want to get rid of local disks in client PCs. But that's a place where FC can't realistically go, anyway.
Some points where I at least partially disagree (my numbers, not references to yours):
1. Some terminology nitpicking... In my book, a SAN is a concept that can be implemented using iSCSI, or FC, or both, or something else. So, I guess, what you mostly mean is: FC is dead. The physical separation of storage and regular traffic is a separate issue, I think.
2. As soon as you start running your SAN traffic over the regular LAN, maybe you have better security mechanisms in place. However: when you do that, you really need these mechanisms. If you just use FC to hook a few servers to some local storage units, you don't really need security beyond what the FC switches provide. If you use your LAN... well, then you're in trouble.
There's a reason why you separate your console and iLOM/ALOM traffic from your regular LAN, right?
3. In my mind, iSCSI and FC are conceptually almost the same, except for the variation in physical media. Seen from that conceptual viewpoint, replacing FC by iSCSI makes sense, since overall less specific hardware and software needs to be maintained.
4. If you really need to bring lots of I/O to a massively multi-core system: build an X4500 with a Niagara-II board and put some key application logic on that system (like a database server for instance).
Do you really want to throw all the raw data around? Maybe for video streaming and maybe for massively parallel processing of huge amounts of raw scientific data.
My take on this:
Maybe we shall see iSCSI replacing FC step by step. First for routed traffic, later perhaps local storage traffic.
But do we really want to mix regular and storage traffic?
I don't think so.
Expect perhaps where you want to get rid of local disks in client PCs. But that's a place where FC can't realistically go, anyway.
I strongly believe that SAN and LAN will converge into a single connection into the server. With the availability of massive amounts of bandwidth itīs no problem to mix regular and storage traffic. So "SAN is dead" is a bold, but not incorrect statement. Having a single connection would ease the cabling, would ease the administration and would ease the handling of systems.
Of course you need the stronger security when you use IP. But think the way we trust SANs is equally false like trusting rotating rust as an persistent way to store data without further protection mechanisms. Perhaps it isnīt such an issue for the company with five or so servers, but most of this companies use direct attached storage anyway. We are in need of more secure storage attach. E.g: WWN zones or port based zones are technolgies we used in Ethernet years ago and dismissed them as insecure.
At the end, iSCSI has a nice advantage: I donīt have to throw money to companies that sell me extremly expensive gigabit-switches
Of course you need the stronger security when you use IP. But think the way we trust SANs is equally false like trusting rotating rust as an persistent way to store data without further protection mechanisms. Perhaps it isnīt such an issue for the company with five or so servers, but most of this companies use direct attached storage anyway. We are in need of more secure storage attach. E.g: WWN zones or port based zones are technolgies we used in Ethernet years ago and dismissed them as insecure.
At the end, iSCSI has a nice advantage: I donīt have to throw money to companies that sell me extremly expensive gigabit-switches
"I donīt have to throw money to companies that sell me extremly expensive gigabit-switches"
Well... instead you are going to throw money to companies that sell you extremely expensive managed 10GBit Switches with Security and Traffic shaping as well as some extremely expensive uplink bandwidth of... what?
Also, let's see the cost of the 10GB infrastructure beyound the edge switches. 10GB edge switches call for an uplink capacity well in excess of that, and very quickly you are talking about some very real money.
"We are in need of more secure storage attach."
OK, sure, the established FC mechanisms are not that great. On the other hand, for many uses, they don't have to be, they just are not nearly as exposed as regular LAN traffic is.
My personal no. 1 learning regarding security is: even if all the technology is perfectly secure and everything is great: you can't do anything against human failures. And since you have to cope with these: nothing beats physical separation...
Let's see how this works out in reality.
Well... instead you are going to throw money to companies that sell you extremely expensive managed 10GBit Switches with Security and Traffic shaping as well as some extremely expensive uplink bandwidth of... what?
Also, let's see the cost of the 10GB infrastructure beyound the edge switches. 10GB edge switches call for an uplink capacity well in excess of that, and very quickly you are talking about some very real money.
"We are in need of more secure storage attach."
OK, sure, the established FC mechanisms are not that great. On the other hand, for many uses, they don't have to be, they just are not nearly as exposed as regular LAN traffic is.
My personal no. 1 learning regarding security is: even if all the technology is perfectly secure and everything is great: you can't do anything against human failures. And since you have to cope with these: nothing beats physical separation...
Let's see how this works out in reality.
+1
The LKSF book
The book with the consolidated Less known Solaris Tutorials is available for download here
Web 2.0
Contact
Networking
xing.com
LinkedIn
Facebook
My photos
- mail: joerg@c0t0d0s0.org
- icq: 144000169
- Me, myself and I
- openBC
- My del.icio.us
- c0t0d0s0@twitter
- c0t0d0s0@last.fm
- wiki@c0t0d0s0
- Amazon Wishlist
Networking
xing.com
My photos
Comments
about Nanosecond
Wed, 23.05.2012 00:11
I remember this being drummed
into us during Digital Design
at Uni. It's important to cons
ider it when laying out [...]
Mon, 21.05.2012 18:04
Hello Kevin, Im not surprised
with what you are seeing or ha
ve seen when attaching a SSD t
o a USB2.0. USB3.0 helps [...]
Mon, 21.05.2012 04:44
Hi Greg,
With regards to IO
PS I have seen terrible result
s using a 60GB SATA2 SSD with
USB2.0 - USB2 really cho [...]
about ZFS Dedup Internals
Sat, 19.05.2012 09:50
There is no impact to boot/imp
ort times, as the DDT is loade
d as needed ... so the pool is
imported as fast as wit [...]
Buttons
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Germany License
