Berkeley CSUA MOTD:Entry 41576
Berkeley CSUA MOTD
 
WIKI | FAQ | Tech FAQ
http://csua.com/feed/
2006/1/27-29 [Computer/SW/P2P, Transportation] UID:41576 Activity:nil
1/27    Anyone know of a effective way to block all BitTorrent traffic?
        I'm using pf on OpenBSD and I've tried STFW but as near as I can
        tell, blocking the default ports is useless b/c most BT clients
        are using non-standard high number ports (and I can't block all
        of these for other reasons).
        \_ packet payload inspection.
            According to a recently published paper by AT&T Labs
            1, inspection of the data packets that are transmitting
            between clients is a good way to detect BitTorrent
            traffic. The communication between BitTorrent clients
            starts with a handshake followed by a never-ending
            stream of length-prefixed messages. The header of the
            BitTorrent handshake message uses the following format:

            <a character (1 byte)><a string (19 byte)>

            The first byte is a fixed character with value '19',
            and the string value is 'BitTorrent protocol'. Based
            on this common header, you can use the following
            signatures to identify BitTorrent traffic:

            * The first byte in the TCP payload is the character 19 (0x13)
            * The next 19 bytes match the string 'BitTorrent protocol'
           http://www.f5.com/solutions/technology/rateshaping_wp.html
           brought to you by google and "bittorrent traffic signature"
           \- i'm a little busy to go into depth right now [generic
              p2p detection and killing off the sessions is something we
              are trying to do at industrial strength levels] but the
              approach that makes the most sense sort of depends on your
              relationship with the other users, number of users [can you
              respond via a phone call and telling them to stop, or by
              quashing the traffic [batch vs. realtime detection], will
              you have 0 tolerance or allow some legitmate use], whether
              you want to kill off "most" of the traffic [low hanging
              fruit] or trying to aim for 100%, and related to that,
              your data volume sizes and how crafty the people you are
              trying to catch are ... like say you are ignoring http
              due to data volume and they use http port for the traffic].
              there was a reasonable presentation on this at the "hot-p2p"
              event last year. how c and the above advice is good too.
              SNORT may be simpler for you to use than BRO and i would be
              surprised if there wasnt a BT sig for SNORT.
2017/09/20 [General] UID:1000 Activity:popular
9/20    

You may also be interested in these entries...
2008/12/15-29 [Computer/Networking] UID:52254 Activity:kinda low
12/15   ausman, maybe this can help form your views on net neutrality:
        http://lessig.org/blog/2008/12/the_madeup_dramas_of_the_wall.html
        if you're bored, skip down to the iFilm example.
        \_ jim: consider the case of a private electrical utility:
           do you believe in "current neurality"? i agree if say GOOG
           were to put in a giant server complex creating a sudden local
	...
2008/11/29-12/4 [Computer/Networking] UID:52128 Activity:moderate
11/28   So you know, I am going to see if the collective motd.wisdom has
        anything to add to my own research. I would think that given my
        political leanings in most areas, I would be a big fan of "net
        neutrality" but I am not, at least not so far. What is wrong with
        someone like AT&T charging more for premium internet service? Don't
        they do that already? Is there something I am missing here? -ausman
	...
2007/10/24-25 [Computer/SW/Mail, Computer/SW/P2P] UID:48429 Activity:kinda low
10/24   is there a command line bittorrent client that lets me download
        selective files out of a torrent, instead of the entire torrent?
        I have this torrent "Busty Conquests of Wendy Whoppers" that contains
        10+ hardcore clips of Wendy Whoppers and her 32GGG breasts, but I
        think I have 3 of these already.  I don't want to destroy my ratio
        downloading gigantic breast porn I already have.  thanks.
	...
2007/7/11-12 [Computer/SW/P2P] UID:47263 Activity:low
7/11    So what p2p systems are there besides bittorrent and soulseek?
        \_ Lots. I like eMule for when I'm looking for something random
           and hard to find. BitTorrent for everything else.
        \_ Lots. eMule is best for things that are random and hard to
           find. BitTorrent for everything else.
	...
2006/11/12 [Computer/SW/P2P, Politics/Domestic/Immigration, Politics/Foreign/Europe] UID:45332 Activity:nil
11/12   Anyone know of a way to watch the rights restricted content on the
        BBC' s Torchwood website. (UK only). The episodes that area already on
        BitTorrent are HIGHLY  reccomended but definitely not for the kiddies.
        Think of X-files and CSI crossed with a healthy dose of Sopranos and
        you will get my drift. With lots and lots of DR Who tie-ins along with
        some really really good eye candy.
	...
2010/8/23-9/7 [Transportation/Car] UID:53931 Activity:nil
8/23    "China's nine-day traffic jam stretches 100km"
        http://news.yahoo.com/s/afp/20100823/sc_afp/chinaroadtraffic
        "... the jam between Beijing and Jining city had given birth to a
        mini-economy ..."
        And we think traffic in L.A. is bad.
        \_ Actually those of us who have travelled don't.
	...
2010/2/10-3/9 [Transportation/PublicTransit] UID:53700 Activity:nil
2/10    Does anyone have an authoritative URL that shows the % of people
        in the Bay Area who commute via foot, bike, car, BART, and Caltrains?
        In particular I'd like to look at trend as well.
        \_ http://www.sfced.org/about-the-city/urban-data-and-statistics/commute-patterns has some.  -tom
        \_ Guys, guys, guys, I asked a simple question. What % of Bay Area
           traffic goes to autos, bikes, foot, BART, and Caltrain? I'm
	...
2009/7/21-24 [Transportation/Car/RoadHogs, Transportation/Car/Hybrid] UID:53167 Activity:low
7/20    Do people not know that the only place where there is no speed
        limit is on a freeway onramp?  Which means that it is the entrant
        driver's job to speed up and get in past the existing traffic?
        \_ The ones who can't accelerate are in SUVs
           \_ True.  My 2nd-gen Prius (not the 2010) accelerates on the
              on-ramps fine.  -- !OP
	...
2009/4/6-13 [Reference/Tax, Transportation/PublicTransit] UID:52808 Activity:high
4/6     Alameda sales tax is now 9.75%. that's pretty rough. sales
        tax is regressive.  Some boneheaded Oakland city council member
        wants to raise Oakland sales tax even more, in this
        recession. - motd liberal
        \_ Yes, the sales tax, car tax, and income tax increases enacted by the
           state legislature are the largest in history, and massively
	...
Cache (8192 bytes)
www.f5.com/solutions/technology/rateshaping_wp.html
Download PDF Version With the increasing proliferation of broadband, more and more users are using Peer-to-Peer (P2P) protocols to share very large files, including software, multi-media files, and applications. This trend has exponentially increased traffic flows across a very wide area network. A more powerful technique based on application signature identification via packet inspection may be needed. Traditional rate shaping techniques may not be sufficient to control new breeds of applications. For example, BitTorrent is a protocol that is typically used by simple desktops to transfer user files via broadband connections. However, using BitTorrent to transfer high volumes of data puts huge pressures on the broadband operators' network. Unfortunately, prohibiting BitTorrent traffic has become routine for some broadband operators and is now a key area of contention between users and broadband operators. This White Paper describes how you can use F5 BIG-IP iRules and the Rate Shaping feature of the BIG-IP Local Traffic Management device to identify different types of traffic for individualized control that can return double-digit capacity without spending a dime on additional bandwidth. Through the combination of iRules and Rate Shaping, you can: * Ensure that critical applications are not impacted by non-priority traffic. F5's BIG-IP Cuts Application Delivery Time and Optimizes Bandwidth Rather than using a one-size-fits-all approach to controlling network traffic, network managers need a more application-oriented way to transmit and distribute network data. In the case of BitTorrent traffic, F5 suggests: Step 1 - Identifying BitTorrent traffic via packet inspection Step 2 - Implementing a rule to isolate BitTorrent traffic Step 3 - Assigning a rate shaping policy that only applies to BitTorrent traffic With BIG-IP iRules and the Rate Shaping feature in the BIG-IP Local Traffic Management system, you can control the bandwidth usage of any type of traffic. Figure 1 shows how Rate Shaping can control the bandwidth usage of just BitTorrent traffic. Controlling BitTorrent Traffic Figure 1: Controlling BitTorrent Traffic The following sections describe each step of the process, provide a sample iRule to identify the BitTorrent application signature, and describe your options for controlling virtually any type of traffic. The communication between BitTorrent clients starts with a handshake followed by a never-ending stream of length-prefixed messages. The header of the BitTorrent handshake message uses the following format: <a character (1 byte)><a string (19 byte)> The first byte is a fixed character with value '19', and the string value is 'BitTorrent protocol'. Based on this common header, you can use the following signatures to identify BitTorrent traffic: * The first byte in the TCP payload is the character 19 (0x13) * The next 19 bytes match the string 'BitTorrent protocol' Using BIG-IP iRules to Detect BitTorrent Traffic BIG-IP iRules is a powerful yet simple tool you can use to identify and isolate the application traffic you want to direct, filter, or persist on. BIG-IP iRules gives you the ability to customize application switching based on business needs, optimizing the handling of traffic - where and when to send it for the fastest response based on application type, category, and priority. The following example uses an iRule to intercept traffic and pinpoint when a TCP connection has initiated BitTorrent communication and manage only that traffic without affecting any other type of traffic. iRule example Once a TCP client is accepted, BIG-IP inspects the first packet's payload of a TCP connection and looks for a match with the BitTorrent protocol signature. Using the BIG-IP Rate Shaping feature, you can assign a Rate Class that corresponds to the policy you define to control traffic with the BitTorrent protocol signature. In this example, if the TCP payload is a BitTorrent payload type, it is assigned to the Rate Class "p2p_bt". You can also target BitTorrent traffic for special processing, isolating it from all other traffic on the network including routing all BitTorrent traffic through a separate WAN link, limiting the amount of bandwidth devoted to BitTorrent traffic, or any combination of bandwidth control techniques described in this paper. Once the connection is built, you can designate all the subsequent packets in the same client session as "p2p_bt", using the BIG-IP session persistence feature. BIG-IP minimizes the degradation of switching efficiencies due to packet inspection because it doesn't need to process every packet of a session beyond the first few bytes of the first payload packet. By combining bandwidth control functionality with an iRule that identifies and isolates specific types of traffic, you can control traffic in the following ways: * Base throughput rate * Absolute limit on the rate at which traffic is allowed to flow when bursting or borrowing * Maximum number of bytes that traffic is allowed to burst beyond the base rate, before needing to borrow bandwidth * Direction of traffic (any, client, server) to which the Rate Class is applied * Rate class from which this class can borrow bandwidth * Method that the Rate Class uses to queue and dequeue traffic You can also define policies in each Rate Class for traffic flowing through any single or group of virtual servers and/or pools. The following example shows the interface and properties for a basic rate class. Properties for a basic rate class Figure 2: Properties for a basic rate class Limit Excessive, Non-critical Traffic The typical service provider environment that facilitates P2P conversations includes a complex network of connections. These connections start from one end-user to an access network through the backbone of the core network to another access network, and then to the destination end-users at a distant location. A key junction point in these P2P connections, like those used with BitTorrent, is the junction point between the Metropolitan Area Network (MAN) router and the router connecting to the service provider's network backbone. From a traffic management perspective, these junction points are high-impact traffic management locations, as thousands of users transverse this junction to access the service provider backbone to complete P2P file transfers. F5 traffic control at this junction point enables the network operator to manage thousands of users from one network device that is physically located at this junction point. The customer referenced in this paper used an in-line deployment of BIG-IP as a bridge between the MAN router and backbone router to manage BitTorrent traffic. Network Topology Structure Figure 3: Network Topology Structure Measuring Performance Improvements Baselining Implementing traffic policies starts with measuring and documenting the baseline performance of your "untuned" network. You can use any monitoring solution (MRTG, Cacti, Cricket) to baseline the performance of your network. Bandwidth Consumption Baseline Prior to creating policies to manage specific types of traffic, measure the traffic load passing through the BIG-IP switch to baseline performance. You can configure a simple monitoring solution to draw curve diagrams of input/output traffic change through the system. Figure 4 shows an example of a data traffic diagram collected by a typical traffic performance monitor system. Input/Output Peak of General Bandwidth Figure 4: Input/Output Peak of General Bandwidth Note that the traffic through this network junction is high from 8:00 to 24:00, and during this time period, traffic exceeds 60MB/s quite often. Bandwidth Consumption of Specific Applications Baselining traffic across the network also involves the graphing of traffic throughput by application. Using monitoring software, analyze typical Internet applications to determine what type of applications merit their own Rate Class. Figure 5 shows the traffic consumption of FTP and WWW traffic that you can use to determine if these types of applications are using a disproportionate amount of bandwidth. Bandwidth Proportion of WWW and FTP Figure 5:...