Berkeley CSUA MOTD:Entry 29505
Berkeley CSUA MOTD
 
WIKI | FAQ | Tech FAQ
http://csua.com/feed/
2025/05/28 [General] UID:1000 Activity:popular
5/28    

2003/8/28-29 [Computer/SW/OS/Linux, Computer/SW/OS/Solaris] UID:29505 Activity:high
8/28    Followup on a previous post about network transfer of large files
        and checksums:  I have compared the 2nd download which passed the
        md5sum with the first one which didn't.  They have identical sizes
        but differ in content on about 200 bytes out of about 640MB.  Is
        there a way to estimate the likelihood that this is the result of bad
        transmission or a malicious substitution?  I am asking both for
        theoretical curiousity and practical interest.  So besides some
        high brow math. argument, is there some obvious indication like
        whether the differences are concentrated, continuous, etc to check?
        \_ Mount the iso file (assuming it's on a linux box) and poke around.
           mount -o loop -t iso9660 filename.iso /mnt/tmp
           \- yes there is a way to guess whether it is random or malicious
              depending on what the contents are [probably], but it is a lot
              of work, so i wouldnt bother. 200bytes is a hell of a lot.
              that is a little strange. my guess is linux -> ass. --psb
        \_ Have you determined what the differences are?
           \_ All I did was was comparing the two images byte by byte with
              a simple c program.  Of course one could recursively look into
              each volumes, and to be comprensive one has to look at
              the partition map, catalog file, and auxillary partitions.
              But as the posters above wrote, it is way TOO MUCH work for a
              mild curiosity.  I was asking if some statistical/probabilistic
              analysis is possible (in theory) and some rule-of-thumb
              available exists in practice.  The transport was thru ftp, btw.
              \- tcp checksum is not going to miss 200bytes in a <1gig xfer.
                 what you should do is do the xfer 100 times [or whatever]
                 and see how many times a strong checksum fails. if you do
                 that, i'd appreciate it if you would send me the info.
                 linux has a history of flailing on large data. --psb
                 \_ I transfer 8GB disk images and 600MB iso's between
                    my linux boxes. I've never had any problems. what do
                    you mean linux "flailing on large data"?
        \_ I throw around 2 terabytes of data with linux every other day
           and I haven't noticed any data loss yet but I have not
           conducted an exhaustive statistical study. - danh
           \- do you guys actually check the data or do you cross your
              fingers? obviously if you dont look, you wont find.
              also it may not manifest itself withing a certain range
              of behavior/configurations.
              anyway, first hand, i have had linux system writing
              corrupted packets on the the net [went away when ethernet
              driver was changed]. when we changed various things in
              bpf and syskonnect ethernet driver fleebsd was fine
              with our hacks, linux occasionally had issues (we didnt do
              too much research on what the problem was ... we just abandoned
              it ... and the problems seem to in part go away when we had
              faster processors and faster disk bus). i dont remember which
              file system it was, but one of them lost us some data and it
              didnt appear to be a hardware problem [was a while ago also...
              lately i havent been looking but havent casually noticed
              data loss at fs level]. i dont need to say anything about
              linux nfs server. admittedly these are rare, but they are
              in areas you expect perfection. a bigger problem is just
              general "weird behavior" under load [or sometimes even
              not under load]. linux does too many short cut things for
              "typical case" speed hacks. this can lead to your being out
              to sea when something goes wrong [e.g. when you look at a
              solaris crash dump, you have much better info than trying
              to figure out what happened in the linux case. this might
              partly be my better knowledge of solaris but in some cases
              the relevant info about the thread state, locks, watchdogs
              simply were not there] and also the system behavior often is
              sort of unusual under load [e.g. low free memory + high io,
              compared to FreeBSD and solaris (although when various large
              changes were made in solaris kernel algorithms for short
              periods i did see some performace issues)]. finally i dont like
              the way the memory-file system subsystem has been evolving.
              recently seen some problems in work environments with lots of
              (tcp) connections ... you get weird hangs on clients when the
              server drops packets ... admittedly this might have been fixable
              by throwing hardware at the problem or tweaking various para-
              meters (and this was on some HPC enviornments were we could not
              compare against solaris/bsd).
              YMWTGF: andrew hume HotOS linux suspect   --psb
              \_ Our answer was much simpler than yours.  After too many
                 lost files, NFS problems, dropped packets, etc, etc, we
                 simply stopped using Linux because it sucks.  We didn't
                 have the time to get into this driver vs that driver or
                 what kernel patch might have helped or which NIC, etc.
                 Linux = not ready for enterprise = out the fucking window.
                 Staff time is more expensive than the value of possibly
                 finding a solution to kludge Linux into working.  The
                 moment we switched to real OS's our problems just magically
                 went away without hiring a team of Linux kernel developers.
                 Linux is cute but their development philosophy precludes
                 it's use in enterprise environments.  Just FYI, I'm tossing
                 around 20-30TB/month between various hosts.
                 \_ Which OS did you switch to?  FreeBSD?  Solaris?
                 \_ Yeah, especially now that Sun sells the X1 for under $1k.
        \_ I should have added: The system from which I run the ftp was
           OS X, which is a (free)bsd derivative.  And I also noticed that
           the bad download had wrong modification time.  It was set to be
           the day of the download, even though I have "preserve" on.
2025/05/28 [General] UID:1000 Activity:popular
5/28    

You may also be interested in these entries...
2013/2/19-3/26 [Computer/SW/OS/OsX] UID:54611 Activity:nil
2/19    I program a lot by sshing to a Linux cluster.  So I'm used to using
        Xemacs to code.  This works fine from a Linux or Windows workstation,
        but sometimes I have to use a Mac.  On Mac, the meta is usually
        bound to option, but that often doesn't work over ssh for some reason.
        This makes using emacs a real pain.  Any suggestions on how to fix it?
        (Other than "use vi")
	...
2012/8/28-11/7 [Computer/HW/Memory] UID:54466 Activity:nil
8/26    Amazon medium instances (3.75GB RAM): 0.160/hour = $1382/year
        Generic standard Linux VPS (4GB RAM): $480/year
        Amazon costs more (but does offer superior scaling options).
        \_ Amazon is $670 if you buy a year's usage up front (heavy util).
           Why is heavy util less expensive than light util?
	...
2012/1/4-2/6 [Computer/HW/Drives] UID:54281 Activity:nil
1/4     I want to test how my servers behave during a disk failure and
        a RAID reconstruction so I want to simulate a hardware failure.
        How can I do this in Linux without having to physically pull
        a drive? These disks are behind a RAID card and run Linux. -ausman
        \_ According to the Linux RAID wiki, you might be able to use mdadm
           to do this with something like the following:
	...
2010/7/21-8/9 [Computer/SW/OS/FreeBSD] UID:53890 Activity:nil
7/21    Can I just use ifconfig to expand my netmask on a FreeBSD box?
        Are there any gotchas here? Linux forces me to restart my network
        to expand my netmask.
        \_ yes... and no, you don't have to restart your network on linux either
           \_ Rebooting is the Ubootntoo way!
              \_ Oooboot'n'tootin!
	...
2010/7/22-8/9 [Computer/SW/OS/FreeBSD, Computer/HW/Drives] UID:53893 Activity:nil
7/22    Playing with dd if=/dev/random of=/dev/<disk> on linux and bsd:
        2 questions, on linux when <disk>==hda it always gives me this off
        by one report i.e. Records out == records in-1 and says there is an
        error. Has anyone else seen this?  Second, when trying to repeat this
        on bsd, <disk>==rwd0 now, to my surprise, using the install disk and
        selecting (S)hell, when I try to dd a 40 gig disk it says "409 records
	...
2010/5/26-6/30 [Computer/SW/Unix/WindowManager, Computer/SW/OS/OsX] UID:53844 Activity:nil
5/26    anyone use lxde?  supposedly it is less stupid than xfce and
        less bloated than gnome.  thoughts?
        \_ lol, does anyone still use desktop linux?  Get with the times
           buy a mac.  Now.  DO IT.  Go NOW.
           \_ but we prefer herring to Kool-Aid
              \_ "you have to yell, he's hard of herring"
	...
2010/5/6-26 [Computer/SW/OS/Windows] UID:53818 Activity:low
5/5     Does anyone know how to do custom install of Ubuntu 10.04? I can't
        even boot it up to give me the menu to custom install and it
        keeps installing a bunch of crap I don't need. It's getting
        just as slow and bloated as Winblows install. Dear lord,
        I miss the old Ubuntu.
5/5=1.0 Numerology FTW.    5+5=10
	...
2010/4/22-5/10 [Computer/SW/Languages/Misc] UID:53797 Activity:nil
4/22    In Linux is there an easy way to rename the scripts in /etc/rc?.d ?
        For example I want to set all the /etc/rc?.d/S91apache to S100apache
        so that it'll run the ramdisk BEFORE going to apache.
        \_ Sure, just move them.
           \_ I mean is there a script that will rename all of them
              for me? Like: setrc apache2 0 0 1 1 1 1
	...
2010/2/8-18 [Computer/SW/Apps/Media, Computer/SW/Apps] UID:53695 Activity:kinda low
2/5     I like Adobe Flash. When written correctly, it scales along
        with your browser size. It looks consistent on every single
        browser. It is predictable. On the other hand, I'm not a big
        fan of CSS/HTML, which for the most part, look wildly different
        between browsers, and don't even work consistently or
        correctly at times. So why do so many people (like Steve Jobs)
	...
2009/10/27-11/3 [Computer/SW/Unix] UID:53475 Activity:nil
10/27   http://www.maxgames.com/play/flash-mind-reader.html
        how does this work?
        \_ sh -c 'for ((i=0;i<10;i++)); do for ((j=0;j<10;j++)); do echo "$i$j-(\
$i+$j)" | bc; done ; done' | uniq
        \_ bash -c 'for ((i=0;i<10;i++)); do for ((j=0;j<10;j++)); do echo "$i$j\
-($i+$j)" | bc; done ; done' | uniq
	...
2009/4/20-23 [Computer/SW/Database] UID:52876 Activity:nil
4/19    ORCL u SUNW = ORCL.
        What is Larry Ellison thinking? What is he going to do with a bunch of
        legacy Sun hardware that no one uses anymore, its fading workstation
        customer base, and open source Sun MySQL that doesn't even generate
        revenue? I really don't get all this acquisition business.
        \_ A lot of big companies still use big, fat Sun hardware. Or use
	...
2009/1/15-23 [Computer/SW/OS/OsX] UID:52398 Activity:nil
1/15    can any serious development be done on OSX that is not *for OSX*.
        i'll grant that ruby on rails has excellent tutorials for the mac.
        discuss:
        \_ What kind of serious development?  If you want to use the standard
           OSX ui then your ui code will be pretty much useless elsewhere,
           but that's why concepts like MVC are so important.  Otherwise
	...
2008/11/29-12/6 [Computer/SW/OS/FreeBSD, Computer/SW/OS/VM] UID:52129 Activity:moderate
11/29   I'm experimenting with virtualization, and as a poor college student
        I'm wondering what the best alternatives for virtualization are, and
        how best to cut my teeth on messing with non-linux platforms (or I
        guess interesting stuff on Linux would work too). Right now I've got
        FreeBSD7 running on KVM on my home computer (on a Core 2 Quad), and am
        somewhat at a loss as to how to use it. (More details: bridged
	...
2008/11/14-26 [Computer/SW/Languages/Java, Computer/SW/OS/Solaris] UID:51970 Activity:moderate
11/13   http://sfgate.com/cgi-bin/article.cgi?f=/n/a/2008/11/14/financial/f051352S72.DTL
        http://preview.tinyurl.com/6nngpm
        Sun Microsystems Inc. plans to cut up to 6,000 jobs, or 18 percent of
        its global work force, as sales of its high-end computer servers have
        collapsed.  The drastic move announced Friday highlights Sun's
        desperation to cut costs and survive as an independent company. Sun's
	...
2008/11/14-26 [Computer/SW/OS/Linux, Computer/SW/OS/Solaris] UID:51989 Activity:moderate
11/14   lulz why doesn't GOOG buy JAVA i mean SUN i mean whatever the hell they
        are these days.
        \_ Even GOOG isn't THAT stupid
           \_ Sorry, but WHY would Google do something like that? They
              run 99.2% Linux servers on the backend. They don't use
              Solaris for development. I mean, what does Sun have to
	...
2008/9/24-29 [Computer/SW/OS/Windows, Computer/SW/OS/Solaris] UID:51283 Activity:nil
9/24    Why is nscd going crazy?  DoS?
        \- back in the solaris say 2.5-2.6 era, it had both some bugs
           (some malformed nis maps made it go crazy) and architectural
           flaws in the IPC/door+threading mechanism. if you are running
           OS-recent, dunno, but you can trace it.
           \_ Yeah, I think it's just buggy.  I've restarted it, and it seems
	...
2008/4/3-9 [Computer/SW/Languages/Misc, Computer/SW/OS/Solaris] UID:49658 Activity:nil
4/3     Solaris experts: I've never played with ZFS. Does it have a native
        dump command a la ufsdump?
        \_ This might be what you are looking for:
           http://preview.tinyurl.com/2xqkda [sun - bigadmin]
	...
2008/3/30-4/6 [Computer/SW/OS/Solaris] UID:49614 Activity:nil
3/30    Question: I just deleted 60 GB of files from an 80 GB disk. The
        disk activity lights were blinking like crazy and I could hear the
        drive crunch while the data was deleted. This is under Solaris.
        Anyway, I think UNIX uses unlink() when files are deleted. Shouldn't
        it just update the free list on the superblock and call it a day?
        What is all the crunching about?
	...
2007/11/27-30 [Computer/SW/Languages/C_Cplusplus, Computer/SW/OS/Solaris] UID:48701 Activity:high
11/27   I'm using select to do a nonblocking check to see if a single socket
        has anything to read off it.  Problem is, I can have up to 12228
        file descriptors, and Linux fd_set only supports up to 4096.  Any idea
        what I can do about this?  (Or a better solution?) -jrleek
        \- 1. who are you
           2. i am busy this week and you didnt mention language
	...