PPSP Seok-Kap Ko Internet Draft Seung-Hun Oh Intended status: Informational Byung-Tak Lee Expires: April 18, 2010 ETRI October 18, 2009 Introduction of Olive and Grid Delivery draft-softgear-ppsp-olive-griddelivery-intro-00.txt Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 18, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract Seokkap, et al. Expires April 18, 2010 [Page 1] Internet-Draft Olive and Grid Delivery October 2009 This draft briefly introduces Olive distributed media platform and Grid Delivery solution. Olive platform, which is built by ETRI, provides live peer-to-peer streaming service. Grid Delivery solution, which is built by Peering Portal, is one of the most popular Peer-to- Peer solutions for VoD, AoD, live streaming, and file download service in Korea. Table of Contents 1. Introduction ................................................ 3 2. P2P Streaming Architectures ................................. 3 2.1. Simple tracker based architecture ...................... 3 2.2. Server controlled architecture ......................... 4 2.3. Fully distributed architecture ......................... 5 3. ETRI Olive platform ......................................... 6 3.1. Overview ............................................... 6 3.2. Basic Operation......................................... 7 3.2.1. Channel Creation .................................. 7 3.2.2. Channel Watching .................................. 8 3.3. Messages .............................................. 10 4. Peering Portal Grid Delivery solution ...................... 14 4.1. Overview .............................................. 14 4.2. Features .............................................. 14 4.2.1. Caching .......................................... 14 4.2.2. Indexing ......................................... 15 4.2.3. Cache state update ............................... 15 4.2.4. Segmentation ..................................... 15 4.2.5. Meta information ................................. 16 4.2.6. Protocol ......................................... 16 4.3. Operation ............................................. 16 5. Security Considerations .................................... 17 6. Acknowledgments ............................................ 18 7. References ................................................. 18 7.1. Normative References .................................. 18 7.2. Informative References ................................ 18 Seokkap, et al. Expires April 18, 2010 [Page 2] Internet-Draft Olive and Grid Delivery October 2009 1. Introduction This document introduces two existing P2P streaming architectures. ETRI Olive distributed media platform is developed by ETRI as an experimental P2P streaming system. ETRI is Electronics and Telecommunications Research Institute, Korea. This platform has been deployed to several enterprise sites. Grid Delivery is one of the most popular P2P solutions for VoD, AoD, live streaming, and download service in Korea. Grid Delivery solution is developed by Peering Portal[PeeringPortal]. Their solution have been deployed to 14 major famous video streaming services. This document shares their experiences of running commercial P2P service. Grid Delivery solution reduced 80% server network traffic and improved overall system performance with several times. This introduction may help to understand how P2P streaming service works and to understand various P2P streaming architectures. This draft can be a "architectural survey" document of PPSP working group in the future. 2. P2P Streaming Architectures P2P IPTV architectures is categorized according to how much a central server work. Terminologies in this document may not be the same to PPSP BOF's[PPSP.WG]. 2.1. Simple tracker based architecture Simple tracker based P2P streaming architecture is a simple and basic architecture. The following figure shows this type of architecture. Seokkap, et al. Expires April 18, 2010 [Page 3] Internet-Draft Olive and Grid Delivery October 2009 +--------+ | Server | +--------+ ^ | +----------+ | +------->| Tracker | (peer list) | | +----------+ | | all/random peer list | | +--------+ v v +-------------> | Peer A | +--------+ | +--------+ | Peer J |<--+ +--------+ Gossip +--------+ +--------+ +-------------> | Peer B |-------->| Peer D | | +--------+ +--------+ | +--------+ +-------------> | Peer C | +--------+ There is the server which supports channel information. This server gives tracker information of a certain channel to new joining peer. Tracker information includes tracker's IP address. The new joining peer (Peer J in the figure) connects to the tracker and asks a peer list. Tracker stores all peer list. This peer list includes IP address and port number of all peers which attends in this channel. A Content Provider of this channel is also included in this peer list. After the joining peer (Peer J) receives all peer list or random peer list from the tracker, this peer checks which peer is the best peer among peers (Peer A,B,C in the figure) in the peer list. There are some methods to find the best peer among peers. And, the joining peer can get more peer list from the initial peers using Gossiping. if Peer J finds the best peer among peers in the list, Peer J connects Peer A and asks Peer A to send media. After Peer J receives media stream from peer A, Peer J registers itself to the tracker. Peer J is added into the peer list in the tracker. This architecture's major character is that a tracker stores a peer list in the channel only. This peer list includes only coarse information about peer such as IP address and port. The initial setup time of this architecture is long because of initial optimization and gossiping. 2.2. Server controlled architecture In contrast to the tracker based architecture, the server controlled P2P streaming architecture uses a central control server. This server stores all information about the channel. This information includes a peer list and virtual link information of the overlay. This Seokkap, et al. Expires April 18, 2010 [Page 4] Internet-Draft Olive and Grid Delivery October 2009 architecture supports Tree-Push model normally. The following figure shows this architecture. . +----------------+ +------->| Control Server | (overlay topology) | +----------------+ |best candidate peer | +--------+ v +-------------> | Peer A | +--------+ | +--------+ | Peer J | --+ +--------+ +--------+ +-------------> | Peer B | +--------+ A new joining peer (Peer J) connects the control server. Peer J finds the channel from the server and asks the peer list to the server. The control server calculates virtual distances from Peer J to the existing peers in overlay topology and finds the best peer candidate for Peer J in the peer list. In contrast to the tracker based architecture, the control server does not give all peer list to the joining peer (Peer J). Instead of all peer list, the control server gives very small number of peer list which are the best candidates (Peer A,B, in figure) for the joining peer. Peer J selects the best peer among the candidates(Peer A,B). If Peer J selects Peer A as the best peer, Peer J notifies the control server that Peer J selects to Peer A. The control server makes Peer A send the media to Peer J. The control server knows everything in this channel. i.e. The server knows who sends to who and who receives from who. The setup delay of this architecture may be very small because the central control server gives optimal initial information to the joining peer. This architecture is also good for CDN-style P2P streaming. However, if there are too many peers in one channel, it could be a burden for the central server. 2.3. Fully distributed architecture Fully distributed architecture replaces the control server with DHT(distributed hash table). P2PSIP is one of DHT implementation[I- D.ietf-p2psip-base]. PPSP Chunk discovery[I-D.chunk-discovery] and IPTV usage for RELOAD[I-D.p2psip-iptv] belong to this architecture. This architecture supports Mesh-Pull model normally. The following figure shows this architecture. Seokkap, et al. Expires April 18, 2010 [Page 5] Internet-Draft Olive and Grid Delivery October 2009 . +----------------+ +------->| DHT (P2PSIP) | (peer list) | +----------------+ |Who has chunk#11 in channel#9? | +--------+ v +-------------> | Peer A | +--------+ | +--------+ | Peer J | --+ +--------+ +--------+ +-------------> | Peer B | +--------+ DHT stores peer list for each channel. Similar to server controlled architecture, DHT gives the peer list and additional information to the joining peer. The joining peer can use additional information to select the best peer to connect with. For a example scenario, Peer J sends channel ID to DHT. DHT gives the peer list to Peer J. Peer J selects the best peer among the peer list. Peer J asks Peer A to send media. After that, Peer J registers its information to DHT as a peer in the channel. This architecture supports serverless service. This architecture can be very scalable. The frequent query may cause overlay load. And, One responsible peer cannot handle or store all peer list if there are too many peer in one channel. IPTV usage for RELOAD[I-D.p2psip-iptv] gives one solution about this problem. 3. ETRI Olive platform 3.1. Overview Olive platform is a peer-to-peer live streaming service platform built by ETRI. Olive platform saves CAPEX and OPEX of streaming service using CDN and P2P technology. Olive platform has been deployed for some enterprise site. The following figure shows Olive platform architecture. Olive platform consists of a central control server, relays, channel providers, and viewers. Olive software allows a user to select its operating mode. Olive is a server controlled architecture which is described in section 2. All other nodes works as clients in control plane. In the control plane, clients communicate with the central server through HTTP and TCP based proprietary protocol. Normal nodes Seokkap, et al. Expires April 18, 2010 [Page 6] Internet-Draft Olive and Grid Delivery October 2009 do not communicate with each other except the central server. They send or receive the real media with each other in data plane. A central server controls all nodes and interworks with EPG(electronic program guide)servers. The central server collects configuration and bandwidth of all nodes. All other nodes reports their capacity, bandwidth usage, and configuration about P2P streaming to the central server. The central server has enough memory and computing power to handle all nodes. One logical central server may consists of multiple servers for performance and availability. +---------+ | Central | +--------+ | Server |---------| EPG | +---------+ | Server | | +--------+ +-----------+-----+--+--------+-------+ | | | | | +--------+ | | | +------+ |Channel | +-----+ +-----+ +-----+ | | |Provider|===|Relay|==|Relay|==|Relay|==|Viewer| +--------+ +-----+ +-----+ +-----+ +------+ 3.2. Basic Operation This section describes an example operation of Olive. The peer selection algorithm can be replaced by administrative configuration. 3.2.1. Channel Creation The following figure shows channel creation process. First, a channel provider connects EPG server through HTTP. The EPG server is a Web server which interwork with the central server. The EPG server provides the channel provider a connection information to establish control connection to the central server. This connection information consists of protocol version, IP address and port number of the central server. Second, The channel provider connects to the central server after login to EPG server. The channel provider sends INIT message which includes operating mode, capacity, and bandwidth. The capacity describes how many stream can be accepted in this node per input and output. Seokkap, et al. Expires April 18, 2010 [Page 7] Internet-Draft Olive and Grid Delivery October 2009 Third, The channel provider gets new media ID from the web server. The channel provider upload media description to the web server before getting media ID. Fourth, The channel provider sends REG message to the central server. This message consists of media ID, sender's IP address and port, and stream bandwidth. When the central server accepts this REG message, it updates EPG with inserting new channel. Channel Central EPG Provider Server Server | | |----(HTTP) Login ------------------->| |<---(HTTP) response /con.info --------| |--(TCP establish)---->| |---INIT(mode)-------->| | [channel creation] |----(HTTP) GET new media ID --------->| |<---(HTTP) Response ------------------| |---CTRL:REG(mediaID)->| | |------>[insert program] | | | 3.2.2. Channel Watching The following figure shows channel watching process. The relay selection strategy may be changed according to the situation. Seokkap, et al. Expires April 18, 2010 [Page 8] Internet-Draft Olive and Grid Delivery October 2009 Channel EPG Central Relay Relay Provider Server Server (R1) (R2) Viewer | | | | | | | |<--- (HTTP) Login ---------------------| | |---- (HTTP) response/con.info -------->| | | |<-- (TCP estabilish) --------| | | |<-- INIT(mode) --------------| | |<--- (HTTP) GET EPG -------------------| | |---- (HTTP) response/EPG-------------->| | | [channel select] | |<--- (HTTP) GET channel info ----------| | |---- (HTTP) response/relay candidates->| | | |<--(RTT measure) --| | | | |<--(RTT m.)-| | | |<-- REQ(media, R2) ----------| | |--RTT(R1,CP)--->| | |<--(RTT measure)-------------------| | | |<-(R.m)| | | |<--RRTT---------| | |<-- OUT+(R2)------| | | |--INP+(CP)----->| | | |--OUT+(Viewer)->| | |==================================>|===========>| | | First, A viewer logins EPG server and connects to the central server. The viewer sends INIT message with viewer mode. Second, the viewer gets program(channel) list from EPG with HTTP. Each channel has media ID as an identification. Third, when the viewer selects a channel, the viewer sends GET request for the media ID to the EPG server. The viewer receives initial relay candidates. Fourth, the viewer performs RTT measurement to the each relay candidate. Currently, the viewer use the Ping(IGMP Echo request) to measure RTT. The viewer selects the best relay from the result of the RTT measurement. Fifth, the viewer sends REQ message to the server. This message includes media ID, selected relay IP address and port, and the viewer's IP address and port. Sixth, the central server try to form better media delivery tree. The central server orders a relay to measure RTT to other relay or Seokkap, et al. Expires April 18, 2010 [Page 9] Internet-Draft Olive and Grid Delivery October 2009 channel provider. In this example, relay R2 receives RTT request from the server. RTT request message includes other relay set(R1) and the channel provider IP address. The relay R2 performs RTT measurement to the relay R1 and the channel provider CP. If RTT between R2 and CP, the relay reply with RRTT(Response of RTT request) to the server. Seventh, the central server asks the channel provider to send the media to the relay R2 using OUT+ message. The central sever also asks the relay R2 to receive the media from the channel provider and to send the media to the viewer using IN+ and OUT+ message. Finally, the viewer can watch/receive the media. 3.3. Messages This section describes proprietary messages of ETRI Olive platform. All messages here are passed over TCP or TLS. The Olive message is based on text form originally. Each element in a message is separated by colon. The message is encrypted when it is transmitted. Initialization - This message is used to register the node to the central server. CTRL:INIT:bandwidth:name:mode:maxin:maxout o bandwidth : total bandwidth of this node. It uses minimum bandwidth if asymmetric network. o name : node name. Olive node ID. This name must be authorized in the web server. o mode : node's operating mode. 0:channel provider, 1:client(viewer), 2:relayer, 3:relayer+viewer. o maxin : maximum number of incoming media stream. o maxout : maximum number of outgoing media stream Channel Registration - This message is used for a channel provider to register its own channel to the central server. Seokkap, et al. Expires April 18, 2010 [Page 10] Internet-Draft Olive and Grid Delivery October 2009 CTRL:REG:mediaId:sourceIP:sourcePort:senderIP:senderPort:cast:bandwid th o mediaId: the media ID. this value is allocated by central server. and it can be received from EPG server. o sourceIP: original source IP address. The relay can register the media as another channel. In this case, sourceIP is not the same to senderIP. o sourcePort: source UDP port number. o senderIP: this node's IP address for this media. o senderPort: this node's UDP port number for this media. o cast: original media source may be multicast or unicast. This must be "UCAST" or "MCAST". This supports hybrid multicast, for example, IP multicast - overlay multicast - IP multicast. o bandwidth: this media's bandwidth (kbps). Channel Request - This message is used for a viewer to watch a certain channel. CTRL:REQ:mediaId:relayIP:relayPort:viewerIP:viewerPort:bandwidth o mediaId: the media ID, which the viewer want to watch. o relayIP: the neighboring relay's IP address. o relayPort: the neighboring relay's IP address. o viewerIP: this node's IP address for this media. o viewerPort: this node's UDP port number for this media. o bandwidth: the bandwidth which the viewer is allowed. RTT Measure Request Seokkap, et al. Expires April 18, 2010 [Page 11] Internet-Draft Olive and Grid Delivery October 2009 - This message is used for the other node to perform RTT measurement. The central server sends to a client node which is a relay, a viewer, or, a channel provider. RTT:numCandidate:candidate1IP:candidate1Port:candidate2IP:candidate2P ort:...:mediaId:clientIP:clientPort:bandwidth o numCandidate: number of candidates in this message. A candidate may be a relay or a channel provider. o candidate*IP: candidate's IP address. o candidate*Port: candidate's IP port. o mediaId: RTT measure request for this media. o clientIP: the node which perform RTT measurement. The receiver IP of this request. o clientPort: The receiver port of this request. o bandwidth: the bandwidth for this media. RTT Measure Response - This message is the result response of RTT measure request. RRTT:candidateIP:candidatePort: mediaId: bandwidth o candidateIP: the best candidate's IP address which has smallest RTT to this node. o candidatePort: the best candidate's Port. o mediaId: RTT measure response for this media. o bandwidth: the bandwidth for this media. Media Output Start - This message is used for the node to send the media to another. OUT+:cast:meidaId:receiverIP:receiverPort Seokkap, et al. Expires April 18, 2010 [Page 12] Internet-Draft Olive and Grid Delivery October 2009 o cast: This must be "UCAST" for unicast, or "MCAST" for multicast. The node sends the media with this way. o mediaId: the media ID which this node should send. o receiverIP: The destination IP address for this media at this node. o receiverPort: The destination IP address for this media at this node. Media Input Start - This message is used for the node to receive the media from another. IN+:cast:meidaId:senderIP:senderPort o cast: This must be "UCAST" for unicast, or "MCAST" for multicast. o mediaId: the media ID which this node will receive. o senderIP: The sender IP address for this media. o senderPort: The destination IP address for this media. Media Output Stop - This message is used to close a outgoing stream. OUT-:mediaId:receiverIP:receiverPort Media Input Stop - This message is used to close a incoming stream. IN-:mediaId:senderIP:senderPort Seokkap, et al. Expires April 18, 2010 [Page 13] Internet-Draft Olive and Grid Delivery October 2009 4. Peering Portal Grid Delivery solution 4.1. Overview Peering portal is one of the biggest companies for providing p2p- based streaming solution to the multimedia portal providers in Korea. Its solution, which is named as 'Grid Delivery', has the following key ideas. In the media streaming service its contents normally stored in servers are very frequently reused. Therefore under Grid Deliver the contents that users request once are stored in somewhere except servers. When another user requests that contents again, they are served to him/her not by servers but the node who stored it temporally. As a result, the heavy load on the servers can be released. It can reduce the cost of providing media streaming service much more than the server-client solutions do, while providing the same service. The following figure shows the node architecture of Grid Delivery solution. This node architecture supports multiple application using client interface layer. +----------++-----------++----------++-----------+ Presentation |Web based ||Stand-alone||AdobeFlash||Silverlight| Layer |player ||Application||player ||player | +----------++-----------++----------++-----------+ Client |ActiveX ||DirectShow ||AdobeFlash||Silverlight| Interface |plug-in ||COM+ intfce||Component ||Component | Layer +----------++-----------++----------++-----------+ Grid Delivery +----------------+ +---------+ +-------+ Peer | Peer selection | | Cache |-----| Cache | Communication | & scheduling | | Manager | +-------+ Layer +----------------+ +---------+ +----------------+ +----------------+ | Network client | | Network server | +----------------+ +----------------+ 4.2. Features 4.2.1. Caching There are two types of caching. - To temporally store in advance the contents of high potential that the clients (peers) may request. Seokkap, et al. Expires April 18, 2010 [Page 14] Internet-Draft Olive and Grid Delivery October 2009 - To temporally store the contents that any user requests and uses once. Grid Delivery adopts the second approach. Because it stores them in caches only when the user's request the contests, there is not addition redundant traffic, which the first approach can make, on both the server side and the network. Also the more the content are requested, the more it caches them. It improves the efficiency of the system naturally. 4.2.2. Indexing To receive the media data from the client, not the server, we need to know which client is able to provide the content that we need. Indexing is a process that makes necessary information to know the list of the clients having the needed data. Indexing can be managed in either central server or the distributed peers. Grid delivery adopts the server-based indexing. It receives the information directly from the server about the clients that can provide the request data, and thus it can make decision easily and quickly. When one peer requests data, the server gives the list of candidate peers who cache the request data using the indexing. That makes it possible for the request peer to receive the media stream from the selected peer among the candidates. 4.2.3. Cache state update For the server to perform indexing, the state of the peers must be updated periodically. In order to find out the appropriate peers that can provide the necessary information to the request peers, the indexing server needs to monitor whether each peer is on-line or not. Therefore the on-line peer must send the information about the cached data and the necessary additional data for indexing to the server. 4.2.4. Segmentation For the more efficient reception of the widely distributed multiple data sources, it necessary to dynamically change the data source according the situation. For this, the client must check the situation not on a entire file basis not on a partial data basis (we henceforth call it segment), and then can request the appropriate data source for the segment. The process of dividing a file into several segments is called as segmentation. Seokkap, et al. Expires April 18, 2010 [Page 15] Internet-Draft Olive and Grid Delivery October 2009 The size of the segment can be variable, but in Grid deliver it is fixed. Therefore it can reduce the overhead of managing variable size of the segments. Actually, it can adopt one from 64KB to 512KB according to the features of the target services and applications. 4.2.5. Meta information Under Grid Delivery, it needs to check the validity of the received segments because the segments is not the one transmitted from the original server but the one stored in cache of a peer. Meta information is used for this purpose. Grid Deliver based these meta information has the following advantages : - to prohibit the spread of the infected data in the cache, - to prevent from distributing the data malformed by the malicious user - to control the term of data validity - to even revoke the data which has been already stored in the cache but is the wrong data inserted by mistake. 4.2.6. Protocol In order to request the wanted file and transmit the requested file, we need a protocol between peers and between a peer and a server. The proprietary protocol designed by Peering Portal based on HTTP, to make it works well in the network where data source is dynamically changed. 4.3. Operation The following figure shows Grid Delivery operation briefly. Seokkap, et al. Expires April 18, 2010 [Page 16] Internet-Draft Olive and Grid Delivery October 2009 Client Server Peer |---- 1. Server Configuration Req ----->| | |<--- 2. Server Configuration Rsp ------| | |---- 3. Media Meta Info Req ---------->| | |<--- 4. Media Meta Info Rsp -----------| | |---- 5. Media File Request ----------->| | |<--- 6. Peer List ---------------------| | |---- 7. Media Chunk Request ---------->|------->| |<--- 8. Media Chunk -------------------|--------| |---- 9. Statistics ------------------->| | Step 1-2. A client gets P2P network configuration information from server. This configuration consists of property parameters, encryption keys, server list, and so on. Step 3-4. The client choose a media/content which the client want to get. And then, the client gets meta information for the content. This meta information consists of validation data for integrity of each segment(chunk), time expires, segment size, permission, and so on. Step 5-6. The client sends File Request message to the server. The server gives the client the peer list back. This peer list consists other peer's IP address and port. This peer list may include additional information such as its bandwidth or its capacity. Step 7-8. The client sends Media Chunk Request message to the server or peers in the peer list. The peer selection algorithm is based on IP location. The peer selection algorithm considers network bandwidth or peer's capability also. Because P2P streaming service is a time- critical service, the response time of Media Chunk Request is very time critical. If the response comes too late, the client tries to another peer or the server. Step 9. The client sends statistics information to the server periodically. 5. Security Considerations TODO - fill in Seokkap, et al. Expires April 18, 2010 [Page 17] Internet-Draft Olive and Grid Delivery October 2009 6. Acknowledgments TODO - fill in 7. References 7.1. Normative References [I-D.ietf-p2psip-base] Jennings, C., Lowekamp, B., Rescorla, E., Baset, S., and H. Schulzrinne, "REsource LOcation And Discovery (RELOAD) Base Protocol", draft-ietf-p2psip-base- 04, Oct 2009. [I-D.chunk-discovery] N. Zong, "Chunk Discovery for P2P Streaming", draft-zong-ppsp-chunk-discovery-00, June 2009. [I-D.p2psip-iptv] S. Ko, "IPTV usage for RELOAD", draft-softgear- p2psip-iptv-01, July 2009. 7.2. Informative References [PPSP.WG] https://www.ietf.org/mailman/listinfo/ppsp, http://trac.tools.ietf.org/area/tsv/trac/wiki/PPSP [PeeringPortal] http://www.peeringportal.com Author's Addresses Seok-Kap Ko ETRI 1000-6 Oryong-dong, Buk-gu, Gwangju, 500-480, Korea Phone: +82-62-970-6677 Email: softgear@etri.re.kr Seung-Hun Oh ETRI 1000-6 Oryong-dong, Buk-gu, Gwangju, 500-480, Korea Phone: +82-62-970-6655 Email: osh93@etri.re.kr Seokkap, et al. Expires April 18, 2010 [Page 18] Internet-Draft Olive and Grid Delivery October 2009 Byung-Tak Lee ETRI 1000-6 Oryong-dong, Buk-gu, Gwangju, 500-480, Korea Phone: +82-62-970-6624 Email: bytelee@etri.re.kr Seokkap, et al. Expires April 18, 2010 [Page 19]