Entry 28777 (Berkeley CSUA MOTD)

Berkeley CSUA MOTD:Entry 28777

WIKI \| FAQ \| Tech FAQ
`http://csua.com/feed/`

2025/07/18 [General] UID:1000 Activity:popular

7/18

2003/6/20 [Computer/SW/WWW/Browsers, Computer/SW/Languages/Perl] UID:28777 Activity:very high

6/19    I want to write a script (or find any workable method) of viewing
        and saving multiple sequential pages on a website.  I can do this
        in perl easily (just extracting data from source) except for cookies.
        I generated frames to let a browser handle the cookie transaction but
        mozilla won't let me save frame source (!?!). Seems like this should
        be easy.  Any advice? -jnat
        \_ have you tried 'wget' or 'curl'?
           \_ wget doesn't seem to work with http://picturetrail.com
        \_ I've thought of doing this too.  Is there a good way to emulate
           a web browser so you don't get in trouble for screen scraping?
              \_ must...supress...urge...to...make...vector...calculus...joke
           \_ curl is what you want.
              \_ must...suppress...urge...to...make...vector...calculus...joke
        \_ for those who are less geeky, there is getleft.
        \_ What do you mean by "multiple sequential pages"?  You can do
           cookies and all other browser emulation in perl.  I'm not sure
           what you're trying to do.  Me and many others have written web
                                        \_ "I"
                                           \_ Nyet, grasshoppa!  In this case
                "Me and many others" is a single plural term.  This is not
                what you were taught but did not learn in 3rd grade about
                "me and jonny were spanked for being bad" vs "I and jonny".
                In the future I'll confine myself to the simpler forms of
                English for you so as not to confuse the masses.
                \_ if you want to be truly pedantic, it's "Johnny and I."
           spidering programs to crawl and rape other people's sites.  It's
           a reasonably well solved problem.
           \_ Okay where do I find out how to do this?  I've been doing
             google searches and reading pages and pages and not seeing
             anything client-side.
             \_ http://cpan.org.  You'll find all the modules for making http
                connections, saving and sending cookies, etc.

2025/07/18 [General] UID:1000 Activity:popular

7/18

You may also be interested in these entries...

2012/2/5-3/26 [Computer/SW/WWW/Browsers] UID:54300 Activity:nil

2/5     How is Firefox on version 10, while I still have 3.6 installed.
        I wait for the X.1 versions and they never come out.
        \_ I'm also on 3.6.26.  It claims that versions 4 - 10 are all faster
           than 3.6.x, but do they use more memory?  Thx.
           \_ Newer Firefox versions use less memory too:
              http://www.maximumpc.com/article/news/mozillas_memshrink_program_brings_big_memory_savings_firefox_7
	...

2010/9/13-30 [Computer/SW/WWW/Browsers] UID:53956 Activity:nil

9/13    Blah blah android blah, ok other than the bootjack stomp of the
        phone marketing crap of this, does anyone know where to find the old
        Android TCL scripting framework that was used for automating
        and controlling desktop apps (like mozilla for example). Thx.
	...

2010/1/11-25 [Computer/SW/WWW/Browsers] UID:53625 Activity:nil

12/9    Does anyone know when Firefox will support Win7?  I can't find a
        roadmap page on http://mozilla.org.  Thx.
	...

2009/10/1-21 [Computer/SW/WWW/Browsers] UID:53417 Activity:moderate

10/1    I am thinking of installing firefox on soda under my home directory.
        Will this make me a hozer?
        \_ Possibly. I wonder if we should have another VM for that...btw,
           I remember someone saying they're glad we're not on FreeBSD
           anymore, but last I checked, a bunch of our stuff is on FreeBSD,
           but our login server is not.
	...

2009/7/16-24 [Computer/SW/WWW/Browsers] UID:53146 Activity:nil

7/15    Any comment on Firefox 3.5?  Better or worse than 3.0?  Thx.
        \_ currently has an unpatched remote code execution vulnerability,
           don't upgrade yet.  -tom
           \_ Ooh, glad that I asked.  Thanks!
              \_ 3.5.1 just released fixes it, supposedly.  Might be worth
                 waiting a few days to see how it shakes out.  -tom
	...

2009/5/5-6 [Computer/HW/Laptop] UID:52950 Activity:moderate

5/5     Is there a good (or standard) way to make an offline copy of a w
        ordpress blog (mine, not someone else's)? tia.
        \_ oh man.
           \_ I could cobble something together with curl / wget, but I'd
              rather not if there is a standard way of doing this.  I'm
              pretty new to wordpress / blogging and I just want to keep
	...

2009/4/22-28 [Computer/SW/Security] UID:52894 Activity:nil

4/22    ok, here's a little networking puzzler. I haven't been able to access
        youtube for a couple weeks. Couldn't figure out why. Happened on all
        browsers. traceroute did weird stuff and then timed out. Finally I
        got so frustrated I setup firefox to ssh tunnel through soda.csua,
        which worked great. Then, I kill the ssh proc, quit FF, and now,
        I can access youtube just fine from any browser. wtf? any
	...

2008/12/18-2009/1/7 [Computer/SW/Mail] UID:52279 Activity:nil

12/18   Campus USENET service will be terminated on 12/31.
        http://ls.berkeley.edu/mail/micronet/2008/1608.html
        \_ I emailed RobR to tell him. -ausman
        \_ The CSUA is considering asking campus to allow us to run NNTP for
           ucb.class.*, as bSpace sucks major major ass. Thoughts? --t
           \_ That's noble, but maybe the effort would be better spent
	...

2008/11/14-26 [Computer/SW/WWW/Browsers] UID:51987 Activity:nil

11/14   When does support for Firefox 2.0.0.x end?  http://www.mozilla.com used to
        mention this, and I forgot what it said.
	...

2008/9/1-3 [Computer/Companies/Google] UID:51015 Activity:moderate

9/1     THE GOOG had Scott McCloud do a comic explaining why THE GOOG Chrome
        (their open-source webbrowser) is cool.  I don't really think it worked
        http://blogoscoped.com/google-chrome
        \_ Oh boy, it comes with porn mode!
        \_ Oh boy, it comes with a porn hider feature!
           http://blogoscoped.com/google-chrome/22
	...

2008/8/18-21 [Computer/SW/WWW/Browsers] UID:50893 Activity:nil

8/18    so i have a bunch of tabs open in one firefox window.  i have a bunch
        of tabs open in another firefox window.  how do i combine them all
        into one window ?
        \_ Why would anyone use tabs? I don't use tabs, and I don't
           understand why anyone else would either.     -dim #1 fan
        \_ Install the Duplicate Tab plugin, and hit Ctrl-Shift-M.
	...

Cache (863 bytes)

picturetrail.com -> www.picturetrail.com/View PictureTrail is a great medium to chronicle family memories. This impressive collection of albums is an excellent example of a job well done. View This longtime PictureTrail member returns again with a wide selection of photos including a tribute to John Denver. View This member's wonderful albums should remind us of how important friends and family are to us. View Excellent images captured from around the world by this very well travelled PictureTrail member. Uploader software and use it to upload pics to your account. You will find uploading pics to be fun and easy with this application. This is a great way to present the details and background about each album. You can also add a cool java slideshow to each intro page. Contact Us Photo Copyrights belong to photographer of origin. These photos may not be copied without their express permission.

Cache (132 bytes)

cpan.orgComprehensive Perl Archive Network 2004-05-14 online since 1995-10-26 2357 MB 243 mirrors 3623 authors 6384 modules Welcome to CPAN!