Berkeley CSUA MOTD:Entry 49622
Berkeley CSUA MOTD
 
WIKI | FAQ | Tech FAQ
http://csua.com/feed/
2025/04/03 [General] UID:1000 Activity:popular
4/3     

2008/3/31-4/6 [Computer/SW/Unix] UID:49622 Activity:nil
3/31    There's this website that I'd like to perform an asshole
        wget operation on it (wget -r --level=3 -k http://site.com
        However, it's got rate throttling and denies me if I fetch
        too quickly. Is there like a "--wait 10" command or something
        I can use with wget? ok thx.
        \_ man wget, look for "--random-wait".
        \_ man wget, look for "--wait".
           \_ Oh man, I can't believe I missed it. Thanks motd!   -op
              \_ Different versions have different features.  Don't do this
                 from soda.
                 \_ I just did a bunch of wgets on soda. Now what?  -op
                    \_ This could be considered a DOS attack.  It's not nice to
                       do that from a shared machine, because you could be
                       prosecuted.
        \_ Look up DOS attack.
2025/04/03 [General] UID:1000 Activity:popular
4/3     

You may also be interested in these entries...
2009/5/5-6 [Computer/HW/Laptop] UID:52950 Activity:moderate
5/5     Is there a good (or standard) way to make an offline copy of a w
        ordpress blog (mine, not someone else's)? tia.
        \_ oh man.
           \_ I could cobble something together with curl / wget, but I'd
              rather not if there is a standard way of doing this.  I'm
              pretty new to wordpress / blogging and I just want to keep
	...
2009/4/22-28 [Computer/SW/Security] UID:52894 Activity:nil
4/22    ok, here's a little networking puzzler. I haven't been able to access
        youtube for a couple weeks. Couldn't figure out why. Happened on all
        browsers. traceroute did weird stuff and then timed out. Finally I
        got so frustrated I setup firefox to ssh tunnel through soda.csua,
        which worked great. Then, I kill the ssh proc, quit FF, and now,
        I can access youtube just fine from any browser. wtf? any
	...
2007/8/7-13 [Computer/SW/Languages/Perl, Computer/SW/Unix] UID:47555 Activity:nil
8/7     I'm trying to figure out whether it's possible for get wget to
        authenticate to a web page protected by a login form ('post'
        method to send username/pass, and cookies.)  Cookies don't seem
        to be the problem, with --save-cookies and --keep-session-cookies
        but getting the username/password submitted isn't doing it.  On
        this particular page, getting to a link, let's say 'example.aspx',
	...
2006/1/26-29 [Computer/SW/Languages/Web] UID:41548 Activity:nil
1/26    In PHP how do you do a regular http get? I'd like to avoid
        using system("wget ...") mechanism. Thanks.
        \_ $html = file_get_contents('<DEAD>...');<DEAD>    --dbushong
           For more details, read:
           http://php.net/manual/en/ref.filesystem.php#ini.allow-url-fopen
           http://www.php.net/manual/en/features.remote-files.php
	...
2006/1/17-18 [Computer/SW/Mail] UID:41396 Activity:nil
1/17    So, the outstanding 'issues' with new Soda are:
        * X11 forwarding
        * SMTP AUTH
        are these problems being worked on?
        \_ smtp auth is being worked on, whats the issue with x11?
        \_ i just noticed that wget doesn't work.
	...
2005/10/24-25 [Computer/SW/WWW/Browsers, Computer/SW/OS/OsX] UID:40238 Activity:moderate
10/23   Guys, I would like to fetch the 'opensolaris' logo from the website
        http://www.opensolaris.org
        What tools can I use to do that?  I am on WiFi.  And I want to fetch
        the logo just because they are making it hard to do so.
        \_ Uh, Firefox?  Any other web browser?  wget?  Why does it matter
           that you're using WiFi?
	...
2005/9/6-7 [Computer/SW/Unix] UID:39515 Activity:nil
9/7     When I used wget -k -r to get stuff recursively, it doesn't translate
        URLS like "http://hello.com/Application?key=val" I think it is
        confused by the ? and other stuff. Also I need to convert them to
        HTML. Do you know how to make wget translate properly? ok thx.
	...
2005/7/21-23 [Computer/SW/WWW/Browsers] UID:38762 Activity:nil
7/21    http://www.apple.com/trailers
        Does anyone knows how to download these trailers to my home computer?
        \_ 1) mozilla/firefox with mplayer plugin
           2) get the html for the page, read the EMBED directives
           \_ Also, if the URL for the large trailer is:  moviexyz_480.mov, the
              real download is typically moviexyz_m480.mov
	...
2005/2/3 [Computer/SW/WWW/Browsers] UID:36053 Activity:moderate
2/2     What is the exact user-agent string I shouold give lynx, wget, curl and
        the like to make them indistinguishable from say, MSIE browser running
        on a wintel?  Googling led me to to things as simple as MSIE or
        something with parenthesized list and I am confused.  Thanks.
        \_ tail -f /var/log/httpd/access.log
        \_ somebox% nc -l -p 8888
	...
2005/1/5-7 [Computer/SW/OS/Linux, Computer/SW/OS/Solaris] UID:35563 Activity:very high
1/5     So who else thinks that Linux Kernel Development has gone haywire?
        WTF is up with this movement from an 8k to a 4k stack in the kernel
        that breaks tons of existing drivers that are ported over from
        Windows? And wtf is this crap doing on production distros like
        Fedora? Don't they realize that if you're going to have a large
        install base that you can't arbitrarily do crap like that anymore?
	...
2004/12/23-25 [Computer/SW/WWW/Browsers, Computer/SW/Unix] UID:35424 Activity:nil
12/23   When I used wget to fetch files form a particular site, it immediately
        gave me a 403 forbidden (before it even get to load robots.txt).
        I can view the web page using a browser, and I have set the user
        agent to be 'Internet Explorer.'  So what's wrong?
        \_ You probably neglected to set the referer.
	...
Cache (46 bytes)
site.com
REFRESH(0 sec): http://site.com/main/index.htm