Berkeley CSUA MOTD:Entry 35877
Berkeley CSUA MOTD
 
WIKI | FAQ | Tech FAQ
http://csua.com/feed/
2025/05/25 [General] UID:1000 Activity:popular
5/25    

2005/1/24-26 [Computer/SW/P2P, Computer/SW/Unix] UID:35877 Activity:kinda low
1/24    Of all the p2p software, which is the fastest at transferring large
        (>1GB) files from one person to another?  Some are premised on
        multiple concurrent uploads to speed up the download (e.g.,
        BitTorrent), which is great for popular files, but I have a large
        set of data that would only be interesting to one other person.  Is
        FTP still the best way to go?
        \_ Split up the file and use multiple FTP connections.  The improvement
           over single FTP connection is large if the distance is great (e.g.
           between California and Japan).
           between California and Japan), where the bottleneck is the roundtrip
           time instead of bandwidth.
        \_ p2p isn't about moving files from one user to the next.  In your
           example above, justputting the files on a website would be as fast
           ast.
           \_ One caveat with just putting files on a website, most Apache
              builds can't send files larger than 2 gigs. -dans
        \_ Also, how much data are you talking about?  Tens of gigs, or
           terabytes?  Once you get into the terabyte range you're probably
           better off just yanking the hard disks, and fedexing them.
           Sneakernet is still the bandwidth king. -dans
           \_ "Never underestimate the bandwidth of a station wagon full of
               tapes." -some guy in the fortune file
        \_ I have files around 2-5GB in size, so FTP sounds like the way to go.
           Any reccommendations for free, secure FTP servers for Windows XP?
           Going back to the p2p model, at what point does it become efficient?
           That is, how many people does it take to share a file such that it
           gets distributed fairly quickly?  --op.  (ps - these are VMWare
           images)
           \_ If you can initiate the transfer you might consider scp or rsync
               with the appropriate flag to compress on the fly.  As to your
               p2p question, I think your understanding is a little flawed.
               What p2p lets you do is aggregate bandwidth from more than one
               source.  Simplifying a lot and ignoring overhead, it's going to
               take the same amount of time to transfer a file from a single
               server connected via a T1 (1.5 megabits per second) as it would
               to transfer a file via a p2p network where six sites with
               256 kilobit per second connections are hosting the file.  Add
               more users and you add more bandwidth.
               -dans
2025/05/25 [General] UID:1000 Activity:popular
5/25    

You may also be interested in these entries...
2009/8/21-9/1 [Computer/SW/Unix] UID:53297 Activity:nil
8/20    When I use rsync to backup, it's pretty cool except in cases where
        I rename a directory name from the source. Rsync will just do
        a plain copy. Is there a program that'll detect renaming of
        directories (by checking for children files), or at least
        move them to a dated directory?
        \_ Not related but beware of using rsync as a backup tool.
	...
2009/7/17-24 [Computer/SW/OS/OsX] UID:53156 Activity:kinda low
7/17    -rw-r--r--@
        What does the "at sign" mean? This is on Mac OS. VMWare disk file.
        \_ The file has metadata attributes
           \_ How do I add/delete attributes to files? What about
              -rw-r--r--+ <-- what is the "+" sign? Also how do you make
              tar preserve these attributes?
	...
2009/2/26-3/5 [Computer/SW/Unix] UID:52653 Activity:nil
2/26    If I want to rsync a perforce or svn directory while users
        are checking things in, could I wind up with a corrupted copy?
        \_ sure why not.  if you are really cool, your copy of the svn
           repo is on LVM, and you snap the LVM to copy it.
           \_ It'll seem less cool when you realize you've taken a snapshot
              of a block level device, which only guarantees that this set of
	...
2008/8/27-9/3 [Computer/HW/Laptop, Computer/SW/OS/Linux] UID:50980 Activity:nil
8/27    I have a rather large linux partition.  I just got a new laptop
        and want to move all my settings and customization to that new
        computer.  how to do this?  I tried remastersys but it seems that it
        get stuck somewhere, and I am hoping it is not really trying to create
        a 26GB iso file.
        any ideas?  is there anyway i can back up my debian package database
	...
2008/6/10-13 [Computer/SW/OS/VM, Computer/SW/Unix] UID:50210 Activity:nil
6/10    Is there a handy guide to virtualizing an already running
        physical linux box into an instance of Vmware?
        \_ this probably isn't the "right" way, but I have many times
        just run rsync. ("rsync -vpa root@oldbox:/ /")  on a fresh virtual
        image.  Just make sure the partitions are the same on the virt disk
        as on the real disk and if you are using a new udev, kill the info
	...
2008/5/29-31 [Computer/HW/Drives] UID:50086 Activity:nil
5/29    Considering abandoning tape backups in favor of something like this:
http://www.thermaltakeusa.com/product/Storage/hdd_station/blacx/st0005u.asp
        Please convince me that I'm crazy.
        \_ why?
           \_ Why do I want you to convince me I'm crazy?
              \_ what are you trying to accomplish?  What is your context?
	...
2007/11/12-16 [Computer/SW/OS/Linux, Computer/SW/Unix] UID:48623 Activity:nil
11/12   how do i make a fail safe magical backup for my debian box
        that i can quickly boot from if the box explodes?
        \_ keep a linux live boot cd around for just such an emergency
           \_ And learn about 'dd'
        \_ I was hoping there was something as slick as CCC, for unix.
           \_ You can first duplicate the disk offline with dd, then just
	...
2007/11/12-16 [Computer/SW/Unix, Computer/SW/OS/OsX] UID:48607 Activity:high
11/12   Server nerds, CCC for mac is totally awesome.  is there somethng
        as cool and friendly for PC or unix?
        \_ Hi.  "CCC" is a free mac program known as "Carbon Copy Cloner".
           If you use a Mac, and have some sort of external usb or firewire
           drive, I recommend you spend 15 minutes downloading it and
           setting it up.  You can easily make a backup set up your data,
	...
2007/8/31-9/3 [Computer/SW/Unix] UID:47860 Activity:nil
8/31    Pretend I am going to be on a boat in the arctic.
        I want to upload whale song sound data while I am
        on my terrible internet connection in the middle of
        the arctic.  What would I use?  FTP seems so 80s.
        \_ FTP is a tool that works and does exactly the job you want.
           What's wrong with FTP?
	...
2007/5/5-7 [Computer/SW/Unix] UID:46533 Activity:low Cat_by:auto
5/4     Question. Say I have dir1/A and dir2/A. How do I collapse them
        together? I'm unable to do "mv dir1/A dir2" or "mv dir1/A/* dir2/A"
        \- cd /foo/.../source;tar cf - . |(cd /bar/.../dest;tar xfBp -);rm -rf .
        \_ cp dir1/A/* dir2/A; rm -rf dir1/A?  What exactly are you trying to
           do and why can't you use mv?
           \- * globbing is risky with unknown inputs. when you want to
	...
2006/11/2-3 [Computer/SW/Unix] UID:45114 Activity:nil
11/02   I have a bunch of directories with files (and other directories)
        in them. I have a mirrored copy that contains many of the same
        files, but with different dates. Is there an easy way (short of
        writing s/w to check each file against the other) to do something
        like : If date on mirror < 10/31, then sync date with other dir?
        I don't think rsync can help with this. I just need the dates
	...