We currently get most of our Internet content, especially video, from large data centers. The high cost of these data centers, and their data comm lines, is a huge barrier to entry to new content providers. This is why 20% of the Internet’s traffic today comes from a single source. So what options to we network architects have to bring about a shift in the Internet’s content architecture such that a few large companies don’t monopolize content?
One is the approach taken by NADA in Europe to create a universal network of P2P-enabled Nano Data Centers:
NADA is seeking to leverage advancements in Peer-to-Peer technology to connect the Nano Data Centers to enable them to work together to provide services to end users.
The set top box would essentially be split in two – one half facing the end user with all the typical functionality and services, while the other half acts as the Peer, or Nano Data Center.
“They isolate it using virtualization technologies, and that secure compartment is now talking to all the other set top boxes, co-ordinating and shifting stuff around. Each of the set top boxes has plenty of storage in it so we can put them together and build a massive data store for all those YouTube videos, Flickr pictures or whatever. We’re using Peer-to-Peer under the hood to provide a service,” Dr Ott said.
This approach, or something like it, has tremendous promise.
The server farm replacement needs to be an always-on device, separate from display machines like PCs and TV sets, inexpensive, easily expandable, and easily manageable. The devices that most resemble it today are home gateways and set top boxes, and the home gateway is actually a better leverage point than the set top box we have today.
I think I’ll build a prototype and see what happens.
This isn’t a good idea. Internet bandwidth is most expensive at the edges, and latency from there to other users at other edges is the longest. It’s the worst place from which to serve up data. You want to do that from servers in the “middle” of the Net. What’s more, the most scarce and valuable resource of the Internet is bandwidth near the edges. Building bandwidth out to those edges is MUCH more expensive than building a server farm… by many orders of magnitude.
Put the servers out there, and you’ll raise the cost of broadband deployment and exhaust the resources that are already there. Anyone can buy space on a fast, cheap server at a server farm for far less than it costs to serve data from the edge. So, why don’t the people who are running this project just do that? There’s only one possible reason: they want to get users and ISPs to give them these resources for free. Which just doesn’t wash. If use of these devices became widespread it would either drive up the cost of broadband tremendously or be banned from networks outright by businesses and ISPs. And deservedly so. It’s a bad idea.
Abundant first-mile bandwidth is one of the key assumptions, for sure, so this is more a futures thing than a present-day practical exercise. But one of the questions that keeps coming up about FiOS is what to do with all the bandwidth, and NDCs are one of the answers.
Another exercise would be a Micro Data Center than could be employed by small ISPs to cache content closer to their customers, and I assume that would be of greater interest.
Maybe with a scavenger class you could ensure that these boxes only use “sunk bandwidth” and thus don’t impose extra costs on customers or ISPs.
There you go, a key part of the system.
Do you think for a minute that the makers of these boxes, or the content providers behind them, would settle for “scavenger class?” Especially when one of the things that motivates them to use P2P is that it hacks the net so as to take PRIORITY over other traffic? I don’t think so. I think that their intent is to try to take as much bandwidth as they can, at the expense of ISPs, without paying for it.
Actually I do think the ISPs who buy home gateways from suppliers like the company I work for would go for something like this. They want consumers to use large amounts of bandwidth for low-priority applications, and want to break Google’s strangle-hold. I agree that the piracy companies., such as Vuze, aren’t interested in slow downloads, but they’re not the target.
And the NDC would be more than a P2P box, it will be your gateway to the Internet for mobile devices and what-not.
Richard, I do not think that the telcos — which are [your company’s] largest customers — really care whether Google is the king of search or not. Yes, they could conceivably have been Google’s competitors, but they’ve decided to sit that dance out. Nor do they really want their consumers to be using large amounts of bandwidth even if it IS low priority, because it still costs. And because, if history is any guide, many users won’t accept the idea that the applications are low priority, will get impatient with them (because they want absolutely EVERYTHING to be instant), and want them to be high priority.
In any event, the bottom line is that shifting servers to the edge of the network is always a bad idea. It’s not any more or less “democratic,” because you can always feed the raw data to a server from the edge. But it doesn’t make sense to distribute from the edges.
I don’t think search is the issue, Brett, content delivery is. And it’s pretty clear that triple-play and VoD are major interests of telcos and cablecos. It’s also clear that the telcos that have high-capacity cable plants are interested in leveraging the investment.
There are many applications that can’t be centralized in server farms at all, because the content originates at the edge, such as personal communications in all of its many forms. And there are other forms of content that warrant caching inside the ISP network for various reasons. So it follows that personal data of most kinds lends itself to caching inside the home network, with a fast pipe to the outside world. That’s one reason we have remote desktop software, isn’t it?
I hope you document your prototype. I’ve been fooling around with edge/home/media servers for the last 12 months in off-hours. I haven’t gotten anywhere with it, because I’ve got higher-priority things to do, but the potential is easy to see. I can see plenty of things that I’d rather keep on a mirrored edge server. Google Docs for my biz, for one tiny example.
Any content that originates at the edge can (and should be) sent to the middle for widespread distribution. It costs far too much, and involves too much latency and other overhead, to send it from the edge repeatedly.
How often is repeatedly? If the content only goes to a few people, most of them friends and family, it’s not worth the trouble to warehouse it remotely.
Brett, why would I want to send my stuff to California or North Carolina for use by people who are exclusively in Texas? Seriously, why waste the bandwidth? It can be used for something else, like Euro-porn, or people in North Carolina uploading family photos.
There is no repeatedly here, either. None. I’m not running a global enterprise with 25/8 worker drones, I’m showing my mom her great-grandkids playing in the sprinkler Or showing my primary customer the layout of his network. All of whom are right here in 200 or 300 fiber miles (and often way less), not thousands.
BTW, where is the middle? It’s a cloud, or pretty dang close to what I always imagined.
Richard: Actually, it pays to “warehouse” it briefly even when it goes to a few people. When we send e-mail, for example, an upstream server “warehouses” the data long enough to distribute the copies to the individual recipients. It also protects against spam. (We all know what “any to any” connectivity did to the medium of e-mail until we got sensible and insisted that users transmit their mail through servers. If we hadn’t done that, e-mail would already be completely useless.)
Scott: The fact is that a server across the country is almost certain to be “closer,” network-wise, to someone on a different ISP in Texas than you are. And if you are only sending a few copies of something to a relative (Interesting how often P2Pers claim that this is what they’re doing!), the impact of one file traveling a few extra miles is negligible compared to the impact of P2P. In fact, without the congestion caused by P2P, it’ll get through a lot faster locally.
As for the location of the “middle”: it consists of major Internet hubs, peering points, co-location centers, and backbone facilities. The Palo Alto Internet Exchange, located in an unassuming former telephone building on Ramona Street in downtown Palo Alto, California, is probably the most important of these.