Path: chonsp.franklin.ch!usenet From: Neil Franklin Newsgroups: alt.folklore.computers Subject: Unix Batch processing (was Re: Old Computers) Date: 19 Jan 1999 22:32:57 +0100 Organization: My own Private Self Lines: 50 Sender: neil@chonsp.franklin.ch Message-ID: References: <77vnpr$7gh@netaxs.com> <780ge2$i55$1@teabag.demon.co.uk> X-Newsreader: Gnus v5.3/Emacs 19.34 ftit@engin.umich.edu (Sergej Roytman) writes: > > Chris Hedley wrote: > > > >I think what amazes me more is that, after 30 years, Unix still doesn't > >have a proper batch processing system. Okay, there's cron, but that's > >more of a scheduler (although some versions do have a basic form of job- > >queueing) but there's nothing out there that takes much account of > >current system-load and so on. > > Hm, batch==I submit it now, it gets done whenever, and I get my results > ("syntax error on line 2") next morning? maintenance job gets queued to > run at 3:00AM so as not to interfere with user jobs? Would at(1) or > some sort of nice(1)-wrapped construction not fit the bill? Please > forgive the ignorance of one who pretty much grew up on *nix. Auch! Auch! Auch! Auch! That is far too cludgy. Definiton: For me batch = do one job at a time to give each all the processor/memory to reduce paging delays. In the firm I worked for 7 years ago I installed the following for the finite element crunchers: While the day the system (1 big workstation, 4 or 5 small ones) worked as an client-server interactive modeling system. Crunching jobs were spooled with lp (!) to an "printer" called ansys. At 20:00 cron fired an enable at the ansys "printer", where the "cat to /dev/lpx script" (this was HP-UX, SysV derived) was replaced by an "cruch this file" script. All jobs were "printed" sequencially. This converted the server into an batch processor. At 6:00 cron fired an disable, so that the next days lp command got queued for the next night. It worked so good that I never got an change request. Only stories 1 years later that thanks to the power of this system, the average model sizes had grown so large that they were still running the next morning. They solved that with more hardware. -- Neil Franklin, Nerd, Geek, Unix Guru, Hacker, Mystic neil@franklin.ch.remove http://neil.franklin.ch/ Programming: when you stop hammering around on the computer as if it were a piece of dumb matter and instead tell it what to do for you ###### Newsgroups: alt.folklore.computers Subject: Re: Unix Batch processing (was Re: Old Computers) References: <77vnpr$7gh@netaxs.com> <780ge2$i55$1@teabag.demon.co.uk> Organization: U S West X-Newsreader: trn 4.0-test69 (20 September 1998) From: cmikk@tig.oss.uswest.net (Chris Mikkelson) NNTP-Posting-Host: 204.147.85.37 Message-ID: <36a60640.0@news2.uswest.net> Date: 20 Jan 1999 10:37:20 +0600 X-Trace: 20 Jan 1999 10:37:20 +0600, 204.147.85.37 Lines: 29 X-Report: Report abuse to abuse@uswest.net. Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.belnet.be!newsgate.cistron.nl!het.net!news-feed.inet.tele.dk!bofh.vszbr.cz!newsfeed.wli.net!su-news-hub1.bbnplanet.com!su-news-feed4.bbnplanet.com!news.gtei.net!newsfeed1.uswest.net!news1.uswest.net!not-for-mail In article , Neil Franklin wrote: >In the firm I worked for 7 years ago I installed the following for the >finite element crunchers: > >While the day the system (1 big workstation, 4 or 5 small ones) worked >as an client-server interactive modeling system. Crunching jobs were >spooled with lp (!) to an "printer" called ansys. > >At 20:00 cron fired an enable at the ansys "printer", where the "cat >to /dev/lpx script" (this was HP-UX, SysV derived) was replaced by an >"cruch this file" script. All jobs were "printed" sequencially. This >converted the server into an batch processor. At 6:00 cron fired an >disable, so that the next days lp command got queued for the next night. > >It worked so good that I never got an change request. Only stories 1 >years later that thanks to the power of this system, the average model >sizes had grown so large that they were still running the next morning. > >They solved that with more hardware. Bravo! Nice hack... At my old number crunching job, we used some beast called DQS. In theory it was pretty cool. Every workstation had a queue for jobs, and would run them pretty much all day. We hacked the xdm login scripts to suspend the queues when somebody logged in. -Chris ###### From: Stefan Skoglund Newsgroups: alt.folklore.computers Subject: Re: Unix Batch processing (was Re: Old Computers) Date: Fri, 29 Jan 1999 00:13:21 +0100 Organization: Telenordia Lines: 11 Message-ID: <36B0EF11.29350C9B@ebox.tninet.se> References: <77vnpr$7gh@netaxs.com> <780ge2$i55$1@teabag.demon.co.uk> <36a60640.0@news2.uswest.net> NNTP-Posting-Host: du142-243.ppp.algonet.se Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Trace: cubacola.tninet.se 917565080 21125 195.100.243.142 (28 Jan 1999 23:11:20 GMT) X-Complaints-To: abuse@algo.net NNTP-Posting-Date: 28 Jan 1999 23:11:20 GMT X-Mailer: Mozilla 4.06 [en] (Win95; I) Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!newsfeed-zh.ip-plus.net!news.ip-plus.net!news-nyc.telia.net!masternews.telia.net!news-feed.inet.tele.dk!bofh.vszbr.cz!newsfeed1.telenordia.se!pepsi.tninet.se!not-for-mail Chris Mikkelson wrote: > > Bravo! Nice hack... > > At my old number crunching job, we used some beast called DQS. In > theory it was pretty cool. Every workstation had a queue for jobs, > and would run them pretty much all day. We hacked the xdm login > scripts to suspend the queues when somebody logged in. > Ahh, you used the GiveConsole script in an creative way.