Musings on the leecher problem. - The leecher problem: - Many users only download files, they don't upload them. - This makes things slower for everyone, since the network-wide total download bandwidth has to equal the total upload bandwidth. - Can't rely on the client to enforce restrictions, people will hack the client. - Calculating client upload/download (share) ratio - Kazaa apparently has clients report their own share ratio (!) - Apparently someone actually thought this was a good idea - Report uploaded/downloaded blocks somewhere - But where? - A node tracking ratios for downloaders of a block/file? - A node tracking each client? - But malicious nodes can collaborate and pump up their share ratio. - Making sure the data is valid is hard. - The BitTorrent approach: - http://www.bitconjurer.org/BitTorrent/protocol.html - Prefer uploading to users you're downloading fastest from. - Unchoke top 4 interested peers by download rate - Opportunistic unchoking - one node (rotated) is unchoked regardless of download rate to try unused connections. - If not downloading (seeding - complete file obtained), use upload rate instead of download rate - Depends only on local transfer rate - Can't be manipulated by scheming leechers. - Download rarest blocks first - Adapting this for Progression - There's no central tracker. - This doesn't change things much, we can use the DHT. - But DHT lookups are more expensive. - Progression chunks aren't strictly associated with one file - Can't just get a list of all peers for a file - But peers with some chunks are likely to have at least some other related chunks of interest. - Proposed algorithm for finding peers: - Pull all the chunk blocks from the DHT. - Or just some? A random sampling? - Build indices of peers and chunks - Periodically send a list of chunks we're interested in to our peers and see if they have them available. - Resort to the DHT when necessary to find a chunk, and/or periodically - Would it help to gossip with peers about other peers's available chunks? - Can we achieve rarest-first? - Rarest-first makes a lot of sense. - Without the tracker, we don't know which chunks are rarest without looking up all the chunk blocks - We can approximate it using our peer/chunk indices. - Does this relate to the idea about pushing rare blocks into the DHT?