Changes between Version 8 and Version 9 of ContainmentInSeattle

Changes between Version 8 and Version 9 of ContainmentInSeattle

Please note that these Trac pages are no longer being updated. Wiki contents/documentation have moved to GitHub.

Changes between Version 8 and Version 9 of ContainmentInSeattle

Please note that these Trac pages are no longer being updated. Wiki contents/documentation have moved to GitHub.

Changes between Version 8 and Version 9 of ContainmentInSeattle

Show
Ignore:
Timestamp:
12/17/09 15:31:59 (10 years ago)
Author:
cosminb
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ContainmentInSeattle

    v8 v9  
    11== Containing Node Communication In Seattle == 
     2Please see attached paper for information about Containment in the Seattle Testbed. 
    23 
    3 The Seattle project allows researchers to remotely run programs in parallel on donated resources. Each computer that is donating resources is referred to as a node, and the resources on each node are split across several vessels (a vessel is similar to a !PlanetLab sliver). Each vessel is capable of running independently from the other vessels on the node. In this project we consider two classes of machines: machines that opt-in to receive Seattle traffic, and machines that do not opt-in to receive traffic generated by Seattle nodes. All nodes opt-in to receive Seattle traffic. Non-nodes may also choose to opt-in to receive Seattle traffic, perhaps to provide a service to nodes. 
     4Module descriptions and server startup details coming soon... 
    45 
    5  
    6 The primary goal of this work is to ensure that nodes can only send packets to opt-in machines. This is important because nodes could otherwise be used to generate SPAM, participate in !BitTorrent swarms with illegal content, or participate in DDOS attacks. There are also several secondary goals: 
    7  
    8  * It is important that there is no hidden delay in allowing or denying traffic as this may interfere with measurements or performance of applications on nodes.  
    9  
    10  * The implementation must be secure against most forms of attack. For example, it should not be possible for a machine to opt-in another machine.   
    11  
    12  * The system must scale to over one million nodes (the expected size of Seattle). 
    13  
    14 Our solution is to use a set of trusted central servers to host a service to manage addresses of machines that have opted in to receiving Seattle traffic. We call this service the Seattle Node Directory Service.  The opt-in machines are responsible for registering (and periodically re-registering) to continue receiving Seattle traffic. In order to prevent spoofing, when a machine wishes to register with the Seattle Node Directory Service, it must first do so through TCP. In response to each registration the Seattle Node Directory Service generates a random renewal key and sends it to the registering machine. The machine can then use these keys to renew its registration via UDP. This is done to improve performance, as TCP is expensive and a TCP-only registration system will not scale adequately. 
    15  
    16 Seattle nodes will make decisions about what traffic to allow or deny locally. To do this, nodes cache the relevant parts of the node directory. In most cases, two vessels will communicate only if they share one or more user or owner keys (i.e. if they host vessels that have an overlapping set of users). However, exceptions such as vessels that provide public services may also be handled. 
    17  
    18 The Seattle Node Directory Service is a distributed system, where each server manages a different set of user keys. A client may have multiple userkeys, meaning it may need to register with more than one server. Before it can register, a client must first be able to determine which servers in the Seattle Node Directory Service to contact for each user key. To do so, the client sends a request to any of the Seattle Node Directory servers to receive information about what range of keys each server supports. We refer to this as 'keyrange' information. The client then locally determines which servers it needs to register with given the local set of keys. 
    19  
    20 In addition to userkey registration, clients also perform a 'query' registration. This allows other nodes in Seattle to receive accurate replies from the server on requests concerning the membership of the client in question in the event of a cache miss. Though it has a similar interface, query registration is different than regular userkey registration. The difference is that the client registers with a special key. On reading this special key, the server substitutes it with the source ip of the registering client, and saves the key in a special table, different than that used for userkey registration. When a client determines which server it needs to register for query, it looks at which server keyranges match the string local ip of the client. 
    21  
    22 If a program attempts to send a packet to an address not in the node's cache, the node sends a packet to the directory service asking if the address is valid. If the directory service replies positively, the address is added to the node's cache. The directory service will reply negatively if the client with the address in question did not perform a query registration. 
    23   
    24 To maintain the node's cache of addresses as current as possible, the Seattle Node Directory Service sends out update packets describing changes to the list of addresses registered for each user key. The update mechanism uses a gossip protocol, with each node forwarding the update packet to a random set of other nodes. For each user key the server will generate a separate update packet containing address changes relevant to just that user key.  Update packets may contain multiple changes, and are sent at short intervals (less than 1 minute apart) to ensure that changes are disseminated quickly. Update packets are not sent out for clients that only perform a query registration. 
    25  
    26  
    27 Unless traffic between the nodes and the Seattle Node Directory Service is protected, an attacker may be able to send false updates to nodes. It is therefore necessary to guarantee integrity of the packets exchanged between the directory service and nodes to protect against man-in-the-middle attacks. This is achieved by including a signed hash and a timestamp with each packet. 
    28  
    29 The Seattle Node Directory Service is a distributed system, so it is possible that a server may fail at any time. For this reason, each server is assigned a backup server. Note, the backup servers are not idly standing by waiting for failures, but rather they are responsible for a keyrange themselves. Every time a server sends out an update packet to clients, it also sends a copy to its assigned backup server. The backup server applies the changes from the update packet to its table. In the event a client is unable to register with the main server, it automatically switches and tries to register with the backup server. The backup server processes client registrations and sends out updates just like the primary server.