Berkeley CSUA MOTD:Entry 45090
Berkeley CSUA MOTD
 
WIKI | FAQ | Tech FAQ
http://csua.com/feed/
2025/07/08 [General] UID:1000 Activity:popular
7/8     

2006/11/2 [Computer/SW/Languages/Misc, Computer/Theory] UID:45090 Activity:high
11/02   What is the best way to implement this?  I have a queue where jobs
        are submitted for processing, and multiple worker threads
        asynchronously pull the front job from the queue to process it.
        I also have N (N=3)
        priority classes.  Normally I would just implement this as N FIFO
        queues and that would be fine.  But wait, there's more!  Some jobs
        cannot be processed until other jobs (their dependancies) have been
        completed.  A job and its dependancy may have different priorities.
        Then, certain jobs may require the exclusive use of a resource being
        used to process a different job in a different worker thread being
        fed from the same pool of jobs (think dining philosophers problem).
        Any ideas?
        \_ Classic priority inversion problem.
        \_ Is this your 162 homework assignment?
           \_ Nah, extending an existing system for work. -OP
             \_ This sounds amazingly similar to some of the job queueing stuff
                handled by Sun's N1 Grid Engine.  You might be able to adapt it
                to your use, or if not, grab the source -- Sun publishes this --
                and see how they solved the same problems. (We use N1GE and
                our users are pretty happy with it) -ERic
                \_ Grid computing? NOOOOOOOO!!!!
                   \_ and in spite of the name, it is not really Grid
                      computing,  it is more of a batch job submission /
                      processing engine.  A great way to distribute tasks
                      across many systems.  SUN Marketing FTL -ERicx
                      across many systems.  SUN Marketing FTL -ERic
                \_ Here is a link to the source:
                   http://gridengine.sunsource.net/servlets/ProjectSource
                   GridEngine is a decent piece of code written by fairly
                   good engineers. I worked on a interdepartment project
                   w/ the N1GE team while at Sun and they were very nice
                   responsive people.
                   I used to work w/ the N1GE group when I was at Sun, if
                   you need a contact, I can find out who is currently
                   working on queue mgmt.
                   \_ and in spite of the name, it is not really Grid
                      computing,  it is more of a batch job submission /
                      processing engine.  A great way to distribute tasks
                      across many systems.  SUN Marketing FTL -ERic
        \_ Would it be an acceptable solution to have multiple threads
           processing the jobs that have no dependencies, but have only
           one thread process jobs with dependencies? That thread could
           then do a topological sort on the elements in its queue when
           it goes to read from the queue. I guess after sorting, the
           job entire dependency graph could be handed to a worker thread...
             - ciyer
           \_ There may be a high fraction of jobs with (relatively simple)
              dependancy graphs.  I'm thinking when a job with dependancies
              gets queued, it's dependancies can get (upwards only) priority
              inheritance.
2025/07/08 [General] UID:1000 Activity:popular
7/8     

You may also be interested in these entries...
2006/11/7-8 [Recreation/Computer/Games] UID:45214 Activity:nil
11/7    Next-gen console predictions anyone?  I'll start:
        PS3s sells out instantly, eBay for around $8-900, remain out of stock
        through christmas then just as supply is kicking in, they run out
        of people willing to spend $600 on a console.  Wii will sell out
        initially, but resupply will just barely keep up with demand.
        \_ You do realize that there are almost exactly as many ps3s
	...
2006/8/29-31 [Computer/SW/Unix, Computer/SW/Languages/Misc] UID:44191 Activity:nil
8/29    Any recommendations for windoze software for backing up files to an
        external hard drive? I'd prefer that the backup isn't one huge .bkf
        file a directory? Is this possible with the ntbackup.exe that comes
        with windows XP.. Also I'd like the software to be a GUI that normal
        non-technical people could run. THanks.
        \_ Not for windows but if you're running any flavor of Unix, just
	...
2006/4/24-26 [Computer/SW/Languages/Misc] UID:42812 Activity:nil
4/24    I finally got to scan some of the old photos for archive purpose.
        The problem is these scanned photos carry no EXIF data.  I want
        to add 'Date Picture Taken' information en masse.  I tried several
        tools that claim to do EXIF batch update but none could batch
        insert 'Date Picture Taken' information.  Does anyone have any
        suggestion for a batch EXIF editor? Thx.
	...
2006/3/10-13 [Computer/SW/Languages/Perl] UID:42187 Activity:low
3/10    I wrote a little perl, that had a little curl
        right in the middle of it's call stack.
        And when it crashed, it crashed very very fast,
        but when it was slow, it was working
        \_ LWP::UserAgent!
           \_ Can I use LWP::UserAgent to do multiple concurrent requests,
	...
2006/2/18-20 [Computer/HW/IO] UID:41918 Activity:nil
2/17    What's the best wireless keyboard to get? Looking for something
        that's compact and quiet. I bought the Gyration keyboard/mouse
        combo and it's a total piece of shit. The first package I bought
        was completely broken. The second package had a working mouse
        but the keyboard wouldn't respond. Upon researching I found
        out there's a whole bad batch they shipped out and many people
	...
2012/8/30-11/7 [Computer/SW/Apps, Computer/SW/Unix] UID:54470 Activity:nil
8/30    Is wall just dead? The wallall command dies for me, muttering
        something about /var/wall/ttys not existing.
        \_ its seen a great drop in usage, though it seems mostly functional.
            -ERic
        \_ Couldn't open wall log!: Bad file descriptor
           Could not open wall subscription directory /var/wall/ttys: No such file or directory
	...
2011/11/20-2012/2/6 [Computer/Companies/Apple, Computer/SW/Unix] UID:54237 Activity:nil
11/20   Are there tools that can justify a chunk of plain ASCII text by
        replacing words with words of similar meaning and inserting/removing
        commas into the text?  I received a 40-line plain text mail where
        all the lines are justified on left and right.  Every word and comma
        is followed by only one space, and every period is followed by two
        spaces.  The guy is my kid's karate instructor which I don't think is
	...
2011/10/26-12/6 [Computer/SW/Unix] UID:54202 Activity:nil
10/24  What's an easy way to see if say column 3 of a file matches a list of
       expressions in a file? Basically I want to combine "grep -f <file>"
       to store the patterns and awk's $3 ~ /(AAA|BBB|CCC)/ ... I realize
       I can do this with "egrep -f " and use regexp instead of strings, but
       was wondering if there was some magic way to do this.
       \_ UNIX has no magic. Make a shell script to produce the ask or egrep
	...
2010/11/21-2011/1/13 [Computer/SW/Languages/Web] UID:53988 Activity:moderate
11/21   Lifehacker's recommending Dreamhost as a personal web hosting service.
        Apart from csua, who do you guys use? --erikred
        \_ What do you want to use it for? Do you need CGI or PHP?  My
           brother worked for Dreamhost and said they are unethical. In
           fact, he sued them. This refers to their treatment of customers
           and employees both. I don't know who  or what "Lifehacker" is,
	...
2010/8/29-9/30 [Recreation/Humor, Politics/Foreign/Europe, Computer/Theory] UID:53940 Activity:nil
8/29    http://www.google.nl/trends?q=ramadan,+porno&ctab=0&geo=ma&date=all&sort=0
        Funny graph.  -- linkpusher
	...
Cache (1556 bytes)
gridengine.sunsource.net/servlets/ProjectSource
gridengine CVS client setup Access the source code repository for this project in one of the following ways: * Browse source code online to view this project's directory structure and files. net:/cvs checkout gridengine WinCvs client setup and checkout instructions To use WinCvs to check out your own set of source code files, you must first set up the correct CVS root using the following steps. You should see a scrolling list of filenames as these are created in your folder. WinCvs how-to guide for this site Command line CVS client setup and checkout instructions Before you can do a CVS checkout, you must set the CVSROOT to indicate where the CVS repository for this project is located. net:/cvs checkout gridengine If this is the only project you are working on, you only need to set the CVSROOT once. Thereafter when you log in to this domain, the CVS repository for this project is assumed. If you are working on multiple projects, however, you must specify the CVSROOT each time you log in to ensure that the CVS repository you are accessing is the right one. This command should result in a scrolling list of files being added to the local directory you specified on your machine. Now you are ready to use your file editor to work with individual files and make changes. If you wish to contribute to the project's CVS repository, you must have "write" permissions as a project member. Note: If your site requires an SSH tunnel for security purposes, you will need to set one up before you can commit modified files into the project repository.