From: Brian Inglis Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Sat, 09 Dec 2000 14:08:46 -0700 Organization: Systematic Software Lines: 80 Message-ID: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> References: <90rv6i$6b$1@spies.com> Reply-To: Brian.dot.Inglis@SystematicSw.ab.ca NNTP-Posting-Host: h-207-148-139-207.dial.cadvision.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Trace: news3.cadvision.com 976396127 20949 207.148.139.207 (9 Dec 2000 21:08:47 GMT) X-Complaints-To: news@cadvision.com NNTP-Posting-Date: Sat, 9 Dec 2000 21:08:47 +0000 (UTC) X-Newsreader: Forte Agent 1.8/32.548 Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!sunqbc.risq.qc.ca!nntp.cadvision.com!207.228.64.17.MISMATCH!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1642 alt.folklore.computers:70415 On Fri, 08 Dec 2000 18:07:04 -0800, aek@spies.com (Al Kossow) wrote: >In article , Eric Smith > wrote: > >from a discussion about improvements to the TPC tape image >container format: > >> aek@spies.com (Al Kossow) writes: >> > I'd be up for stealing the high byte as a flag. The other thing I'd >> > like to see is a meta-record right in the front that has 1) An ASCII >> > description of the format the file was written in and 2) An ASCII >> > record that you can put a description of what the tape was (paper >> > label, original density, etc) >> >> My previous proposal had been for using a length field of all-ones as a >> meta-data record, with a real length field inside that: >> >> . >> . >> . >> toward beginning of image file >> -------------------------------- >> | all-ones (0xffffffff) | >> -------------------------------- >> | meta-data length (4 bytes) | >> -------------------------------- >> | meta-data type (4 bytes) | >> -------------------------------- >> | meta-data | >> . >> . >> | | >> -------------------------------- >> | meta-data length (4 bytes) | >> -------------------------------- >> | all-ones (0xffffffff) | >> -------------------------------- >> toward end of image file >> . >> . >> . >> >> >> However, I think I like your proposal of using high bits of the length >> field better, as tape blocks can't be longer than 64K, and it doesn't >> require an extra length field. >> >> I'd propose that the entire most-significant byte be used to indicate >> meta data (if it is non-zero), and thus the length should be determined >> by anding with 0x00ffffff. A program that is not meta-data-aware should >> skip over any records with bits set in the most-significant byte. >> >> Obvious new records types, with some proposed high bytes: >> >> 0x80 image file header - contents need to be defined >> >> 0xc0 single record containing possibly bad data (length assumed accurate) >> 0xc1 single record containing possibly bad data (length assumed >inaccurate) >> 0xc2 single bad record of unknown length - no data stored >> 0xc3 an unknown number of bad records of unknown length > > >I'd like to add a entire archive or sub block CRC as well, so >you can verify the contents > > 0x81 CRC-32 of entire archive, to be placed as the end of the > file. CRC does not include CRC meta-record (obviously) not obvious -- CRC checks include the CRC data -- giving the result zero > 0x82 CRC-32 of archive since the last sub-block CRC Thanks. Take care, Brian Inglis Calgary, Alberta, Canada -- Brian_Inglis@CSi.com (Brian dot Inglis at SystematicSw dot ab dot ca) use address above to reply ###### From: aek@spies.com (Al Kossow) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 9 Dec 2000 13:25:26 -0800 Organization: Spies In The Wire Lines: 17 Message-ID: <90u806$9iu$1@spies.com> References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> NNTP-Posting-Host: spies.com X-Trace: 9 Dec 2000 13:29:53 -0800, spies.com Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!news.kjsl.com!news.spies.com!localhost!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1646 alt.folklore.computers:70416 From article <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com>, by Brian Inglis : >> >> 0x81 CRC-32 of entire archive, to be placed as the end of the >> file. CRC does not include CRC meta-record (obviously) > > not obvious -- CRC checks include the CRC data -- giving the > result zero Sorry, what I was saying was poorly worded. What you're trying to do is create a 'seal' to see if anything inside of a container has changed, so you need to know what is inside and what is outside of the container. There should be a redundant/stronger error check to see if this seal is correct. ###### From: aek@spies.com (Al Kossow) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Sat, 09 Dec 2000 13:52:13 -0800 Organization: Apple Computer, Inc. Lines: 87 Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> NNTP-Posting-Host: haxrus.apple.com X-Trace: news.apple.com 976398732 5279 17.205.21.66 (9 Dec 2000 21:52:12 GMT) X-Complaints-To: usenet@news.apple.com NNTP-Posting-Date: 9 Dec 2000 21:52:12 GMT Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!paloalto-snf1.gtei.net!news.gtei.net!forum.apple.com!news.apple.com!haxrus.apple.com!user Xref: chonsp.franklin.ch alt.sys.pdp10:1627 alt.folklore.computers:70406 In article <90u806$9iu$1@spies.com>, aek@spies.com (Al Kossow) wrote: > From article <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com>, by Brian Inglis : > >> > >> 0x81 CRC-32 of entire archive, to be placed as the end of the > >> file. CRC does not include CRC meta-record (obviously) > > > > not obvious -- CRC checks include the CRC data -- giving the > > result zero > > Sorry, what I was saying was poorly worded. > > What you're trying to do is create a 'seal' to see > if anything inside of a container has changed, so you need to know > what is inside and what is outside of the container. There should > be a redundant/stronger error check to see if this seal is correct. And with further reflection, I realized that by adding AFC to the list, there is no context for the post. -- Something was lost by switching to the 'all the world is a byte stream' Unix model of I/O - tape blocking. In the olde days, when you wrote a 7 or 9 track tape, you separated your records with 'tape marks'. Two tape marks next to each other meant that it was the logical end of tape. When these tapes are read on Unices, this information is lost (assuming you just copy all the records together in a single file). A long time ago, a utility for copying tapes was written for RSX-11 systems called 'TPC'. The intermediate step in making these copies created a 'TPC container file', which is very simple: block length (a 4 byte little-endian value) data If the block length word was zero, it indicated there was a tape mark to be written. -- With the appearance of computer simulators, there was a desire to be able to efficently deal with backspacing these containers, so the 'E11' or 'Modified TPC' container was created, which adds the block length to both sides of the data block length (a 4 byte little-endian value) data block length (a 4 byte little-endian value) -- Problem is, you can't easily tell these two formats apart. The other problem is these formats are a problem if you try to use them for archiving since: 1) You'd like to be able to save what this is an archive of in the archive 2) The format cannot indicate that a block was read incorrectly (which happens on real tapes, esp ones that you'd like to save) 3) There is no way to detect 'bit rot' in the containers after they've been written. Which was why Eric Smith and I have proposed adding meta-records using the 4th byte of the length value as a flag to indicate that the next record contains meta-data. -- Are there already existing formats for 'tapes in disc files' (although it would be useful for disc containers with bad blocks too) that save detected bad block information? -- -- The eBay Curse: "May you find everything you're looking for.." ###### From: peter@taronga.com (Peter da Silva) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 9 Dec 2000 22:57:35 GMT Organization: TSS Inc. Lines: 38 Message-ID: <90udcv$2jo6$1@citadel.in.taronga.com> References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> NNTP-Posting-Host: citadel.in.taronga.com X-Trace: citadel.in.taronga.com 976402655 85766 10.0.0.43 (9 Dec 2000 22:57:35 GMT) X-Complaints-To: usenet@taronga.com NNTP-Posting-Date: 9 Dec 2000 22:57:35 GMT X-Newsreader: trn 4.0-test72 (19 April 1999) Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!news.kjsl.com!news.usenet2.org!citadel.in.taronga.com!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1638 alt.folklore.computers:70409 In article , Al Kossow wrote: >Which was why Eric Smith and I have proposed adding meta-records >using the 4th byte of the length value as a flag to indicate that >the next record contains meta-data. I'd like to suggest that you switch to a structured file format, like Electronic Arts "IFF". Each file consists of a series of tagged records with a type, a length, and a checksum. The first (32-bit) word of an IFF file is "FORM", followed by the length of the file, followed by the type of the file. This is followed by records, each starting off with a header containing the length and type, the data in the record, and then a checksum. The PNG group and the MIDI association each came up with more stream-oriented versins of this, and so for a tape file you would perhaps go with: F O R M length or 0 for a stream T A P E With records consisting of: T Y P E (4 character type of the record, D A T A, M A R K, etc...) length or 0 for a type with no data If length is non-zero, the record content becomes: data CRC length -- Rev. Peter da Silva, ULC. WWFD? "Be conservative in what you generate, and liberal in what you accept" -- Matthew 10:16 (l.trans) ###### From: aek@spies.com (Al Kossow) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Sat, 09 Dec 2000 15:38:37 -0800 Organization: Apple Computer, Inc. Lines: 31 Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> <90udcv$2jo6$1@citadel.in.taronga.com> NNTP-Posting-Host: haxrus.apple.com X-Trace: news.apple.com 976405116 5630 17.205.21.66 (9 Dec 2000 23:38:36 GMT) X-Complaints-To: usenet@news.apple.com NNTP-Posting-Date: 9 Dec 2000 23:38:36 GMT Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!newsfeed-zh.ip-plus.net!news.ip-plus.net!news.tesion.net!news.belwue.de!news-stu1.dfn.de!news-mue1.dfn.de!news.augsburg.net!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!paloalto-snf1.gtei.net!news.gtei.net!forum.apple.com!news.apple.com!haxrus.apple.com!user Xref: chonsp.franklin.ch alt.sys.pdp10:1629 alt.folklore.computers:70407 In article <90udcv$2jo6$1@citadel.in.taronga.com>, peter@taronga.com (Peter da Silva) wrote: > In article , > Al Kossow wrote: > >Which was why Eric Smith and I have proposed adding meta-records > >using the 4th byte of the length value as a flag to indicate that > >the next record contains meta-data. > > I'd like to suggest that you switch to a structured file format, > like Electronic Arts "IFF". Each file consists of a series of tagged > records with a type, a length, and a checksum. > The first problem I see with this is it assumes the file is read in one direction. This is NOT true with a simulator reading a simulated magtape. I have mixed feelings about parsing text strings in meta-data headers. At least in the simple scheme we've proposed, you can easily ignore any of the meta-records simply by noting that the fourth byte is non-zero (since no magtape created 2^24 byte records). I'd still be curious to hear how other people have solved this problem (if they have..) for the tape libraries that have already been dumped. -- The eBay Curse: "May you find everything you're looking for.." ###### From: peter@taronga.com (Peter da Silva) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 10 Dec 2000 02:58:38 GMT Organization: TSS Inc. Lines: 22 Message-ID: <90urgu$2qvv$1@citadel.in.taronga.com> References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90udcv$2jo6$1@citadel.in.taronga.com> NNTP-Posting-Host: citadel.in.taronga.com X-Trace: citadel.in.taronga.com 976417118 93183 10.0.0.43 (10 Dec 2000 02:58:38 GMT) X-Complaints-To: usenet@taronga.com NNTP-Posting-Date: 10 Dec 2000 02:58:38 GMT X-Newsreader: trn 4.0-test72 (19 April 1999) Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!news-proxy.baileynm.com!citadel.in.taronga.com!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1640 alt.folklore.computers:70411 In article , Al Kossow wrote: >The first problem I see with this is it assumes the file is read >in one direction. This is NOT true with a simulator reading a >simulated magtape. That's why the length is at both ends of the record. >I have mixed feelings about parsing text strings in meta-data >headers. You don't. It's just a 32-bit magic number that happens to be easy for people to grok when looking at the file in a dump. The IFF format had a lot of thought put into it, don't dismiss it out of hand. -- Rev. Peter da Silva, ULC. WWFD? "Be conservative in what you generate, and liberal in what you accept" -- Matthew 10:16 (l.trans) ###### From: Kevin Ashley Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Mon, 11 Dec 2000 02:16:45 +0000 Organization: Posted via ULCC Internet Services Lines: 33 Message-ID: <3A34390D.BACC980D@ulcc.ac.uk> References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> NNTP-Posting-Host: cziwkga-pc1.fds.ulcc.ac.uk Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 4.61 [en] (X11; I; Linux 2.0.34 i686) X-Accept-Language: en Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!uni-erlangen.de!newsfeeds.belnet.be!news.belnet.be!newsfeed.icl.net!newspeer.clara.net!news.clara.net!server3.netnews.ja.net!ulcc.ac.uk!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1657 alt.folklore.computers:70436 Al Kossow wrote: [lots snipped] > > Are there already existing formats for 'tapes in disc files' > (although it would be useful for disc containers with bad > blocks too) that save detected bad block information? > I recall that the E-Systems offshoot, EMASS, did produce a format for doing this as part of some software they supplied with their FileServ HSM. I think it was originally created for Mobil, who were using the HSM to load up a warehouse or two full of seismic survey tapes. I still have the manuals at work somewhere with a description of the format. What I can remember was that it was nested: a wrapper for the whole tape, containing wrappers for files, containing records. Each had length words. It did cope with bad blocks, but I doubt it handled the other requirement your post mentioned (easy backspacing for emulators.) I missed the beginning of this thread but would be interested to see what you come up with. My group does a fair amount of this stuff as part of our digital preservation work. We have a hastily-designed in-house format which I'm not happy with, and I'd be more comfortable using something which other people are also using. ------------------------------------------------------------ Kevin Ashley K.Ashley@ulcc.ac.uk Digital Preservation http://www.ulcc.ac.uk/Staff/Kevin+Ashley ULCC This is not a signature ###### From: aek@spies.com (Al Kossow) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 10 Dec 2000 18:24:51 -0800 Organization: Spies In The Wire Lines: 17 Message-ID: <911dtj$7ha$1@spies.com> References: <3A34390D.BACC980D@ulcc.ac.uk> NNTP-Posting-Host: spies.com X-Trace: 10 Dec 2000 18:29:35 -0800, spies.com Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!news.nextra.ch!news1.sunrise.ch!news.imp.ch!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!news.kjsl.com!news.spies.com!localhost!not-for-mail Xref: chonsp.franklin.ch alt.sys.pdp10:1795 alt.folklore.computers:70615 From article <3A34390D.BACC980D@ulcc.ac.uk>, by Kevin Ashley : > > I missed the beginning of this thread but would be interested to > see what you come up with. My group does a fair amount of > this stuff as part of our digital preservation work. We have > a hastily-designed in-house format which I'm not happy with, > and I'd be more comfortable using something which other people > are also using. > Do you know about the mailing list that I created for people interested in mini and mainframe software preservation? bitsavers@spies.com You may be aware of Tim Shoppa's (trailing-edge.com) work as well. ###### From: jmaynard@thebrain.conmicro.cx (Jay Maynard) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 20 Dec 2000 21:46:19 GMT Organization: Neosoft (using Airnews.net!) Lines: 9 Message-ID: <81638778B12FE05A.630746B8B0E36C7A.7F492EA256DD3A6B@lp.airnews.net> X-Orig-Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> Reply-To: jmaynard@conmicro.cx Abuse-Reports-To: abuse at airmail.net to report improper postings NNTP-Proxy-Relay: library2.airnews.net NNTP-Posting-Time: Wed Dec 20 15:46:19 2000 NNTP-Posting-Host: !Y&Wb1k-Y wrote: >Are there already existing formats for 'tapes in disc files' >(although it would be useful for disc containers with bad > blocks too) that save detected bad block information? One format that you might want to look at is the AWSTAPE format used by the IBM P/390 small mainframes and the Hercules emulator. It doesn't have a bad block flag (though there's room for it), I wrote up a description of the format in afc a while back, and can find it agian if needed. ###### From: aek@spies.com (Al Kossow) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Wed, 20 Dec 2000 14:27:40 -0800 Organization: Apple Computer, Inc. Lines: 74 Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> <81638778B12FE05A.630746B8B0E36C7A.7F492EA256DD3A6B@lp.airnews.net> NNTP-Posting-Host: haxrus.apple.com X-Trace: news.apple.com 977351260 24248 17.205.21.66 (20 Dec 2000 22:27:40 GMT) X-Complaints-To: usenet@news.apple.com NNTP-Posting-Date: 20 Dec 2000 22:27:40 GMT Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!newsfeed-zh.ip-plus.net!news.ip-plus.net!news.tesion.net!news.belwue.de!news-stu1.dfn.de!news-koe1.dfn.de!news-was.dfn.de!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.stanford.edu!paloalto-snf1.gtei.net!news.gtei.net!forum.apple.com!news.apple.com!haxrus.apple.com!user Xref: chonsp.franklin.ch alt.sys.pdp10:2088 alt.folklore.computers:71232 In article <81638778B12FE05A.630746B8B0E36C7A.7F492EA256DD3A6B@lp.airnews.net>, jmaynard@conmicro.cx wrote: > On Sat, 09 Dec 2000 13:52:13 -0800, Al Kossow wrote: > >Are there already existing formats for 'tapes in disc files' > >(although it would be useful for disc containers with bad > > blocks too) that save detected bad block information? > > One format that you might want to look at is the AWSTAPE format used by the > IBM P/390 small mainframes and the Hercules emulator. It doesn't have a bad > block flag (though there's room for it), I wrote up a description of the > format in afc a while back, and can find it agian if needed. Subject: Re: Archives of OS/360-ish public domain software? Author: Jay Maynard On Thu, 29 Jun 2000 10:52:26 -0400, Tim Shoppa wrote: >Well, the raw format of a tape (possibly variable-length tape blocks >separated by tape marks) is pretty simple to begin with, and I firmly >believe that any archive format shouldn't complicate things >unnecessarily :-). Is there a description of the AWSTAPE format >online somewhere? I'd gladly volunteer to write a conversion >utility to/from DECUS "TPC" format and the closely-related John >Wilson's "E11" format used by his emulator and Bob Supnik's simulators. I don't know if it's documented anywhere, but it's very simple: An AWSTAPE file consists of blocks of data, preceded by a block header of 6 bytes. Each block corresponds to a block on the tape. The block header is defined as: /*-------------------------------------------------------------------*/ /* Structure definition for AWSTAPE block header */ /*-------------------------------------------------------------------*/ typedef struct _AWSTAPE_BLKHDR { HWORD curblkl; /* Length of this block */ HWORD prvblkl; /* Length of previous block */ BYTE flags1; /* Flags byte 1 */ BYTE flags2; /* Flags byte 2 */ } AWSTAPE_BLKHDR; /* Definitions for AWSTAPE_BLKHDR flags byte 1 */ #define AWSTAPE_FLAG1_NEWREC 0x80 /* Start of new record */ #define AWSTAPE_FLAG1_TAPEMARK 0x40 /* Tape mark */ #define AWSTAPE_FLAG1_ENDREC 0x20 /* End of record */ The two HWORD values are 16-bit little-endian integers. curblkl is the length of the block described by this block header; prvblkl is the length of the immediately preceding block, and is set to zero for the first block on the tape. Data records are a maximum of 65535 bytes long; longer records on tape are segmented, with the NEWREC and ENDREC flags denoting the first and last segments. A record of less than 65535 bytes has both the NEWREC and ENDREC flags set. A tapemark is denoted by a current length of zero and the TAPEMARK flag set (but not the NEWREC or ENDREC flags). Does that cover the bases? I'm guessing at the function of the NEWREC and ENDREC flags; since an IBM tape can only have 65535 bytes in a record, I've never seen the case where segmentation is needed (and the Hercules tapecopy program, used to create AWSTAPE files from real tapes, is hardcoded to set both flags on data records because of this). For further details, look at the tapecopy, tapemap, and tapesplit utilities in the Hercules package. I whipped up tapemap and tapesplit in a couple of hours after looking at tapecopy for a bit, so it can't be that hard. -- The eBay Curse: "May you find everything you're looking for.." ###### From: Bruce Crabill Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: Thu, 21 Dec 2000 19:09:35 -0500 Organization: Univ. of Maryland Lines: 114 Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> <81638778B12FE05A.630746B8B0E36C7A.7F492EA256DD3A6B@lp.airnews.net> NNTP-Posting-Host: bruce-mac.umd.edu X-Trace: dailyplanet.wam.umd.edu 977443841 10729 129.2.8.90 (22 Dec 2000 00:10:41 GMT) X-Complaints-To: abuse@wam.umd.edu NNTP-Posting-Date: 22 Dec 2000 00:10:41 GMT User-Agent: MT-NewsWatcher/3.0 (PPC) Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!newsfeed-zh.ip-plus.net!news.ip-plus.net!news.tesion.net!news.belwue.de!news-stu1.dfn.de!news-mue1.dfn.de!news-nue1.dfn.de!uni-erlangen.de!newsfeeds.belnet.be!news.belnet.be!xfer13.netnews.com!netnews.com!howland.erols.net!bloom-beacon.mit.edu!news.umd.edu!news.wam.umd.edu!bruce Xref: chonsp.franklin.ch alt.sys.pdp10:2105 alt.folklore.computers:71250 In article , aek@spies.com (Al Kossow) wrote: > Does that cover the bases? I'm guessing at the function of the NEWREC > and ENDREC flags; since an IBM tape can only have 65535 bytes in a > record, I've never seen the case where segmentation is needed (and the > Hercules tapecopy program, used to create AWSTAPE files from real > tapes, is hardcoded to set both flags on data records because of this). Actually, I believe you could write tape records longer than 64K by using the Chain Data flag in the CCWs. 3420s probably didn't care how much data they were asked to write, but there might have been problems on the buffered Control Units like the 3480. Back in 1994, I wrote a program that would create an AWSTAPE file under VM. The following is a portion of the code that delt with writing tape records. It is written in IBM's Pascal/VS compiler. Const Min_Tape_RSize = 4005; Max_Tape_RSize = 65535; Def_Tape_RSize = 4005; Min_Tape_DSize = 1; Max_Tape_DSize = 4096; Def_Tape_DSize = 4096; Min_Tape_BSize = 1; Max_Tape_BSize = 65535; Def_Tape_BSize = 4096; VMFPLC2_Tag_Size = 5; VMFPLC2_Header_Tag = '02'XC||'PLCH'; VMFPLC2_Data_Tag = '02'XC||'PLCD'; VMFPLC2_Blk_Size = 800; VMFPLC2_Max_Blks = 5; FST_Ext_Size = '18'X; Old_FST_Size = L_FST_Rec - FST_Ext_Size; Type Tape_Header_Rec = Record THR_Size: Half_Word; THR_PSize: Half_Word; THR_Flags: Half_Word; End; Const { ====================== THR_Flags constants. ====================== } THRF_NewRec = '8000'X; THRF_EOF = '4000'X; THRF_EndRec = '2000'X; Type P_VMFPLC2_Tag = -> VMFPLC2_Tag; VMFPLC2_Tag = Packed array [1..VMFPLC2_Tag_Size] of Char; P_VMFPLC2_Data = -> VMFPLC2_Data; VMFPLC2_Data = Packed array [0..(VMFPLC2_Blk_Size-1)] of Byte; P_VMFPLC2_Header_Rec = -> VMFPLC2_Header_Rec; VMFPLC2_Header_Rec = Packed record VHR_Tag: VMFPLC2_Tag; VHR_Old_FST: Packed array [0..(Old_FST_Size-1)] of Byte; VHR_LastBlks: Word_Int; VHR_FullRecs: Word_Int; VHR_FST_Ext: Packed array [0..(FST_Ext_Size-1)] of Byte; End; P_VMFPLC2_Data_Rec = -> VMFPLC2_Data_Rec; VMFPLC2_Data_Rec = Packed record VDR_Tag: VMFPLC2_Tag; VDR_Data: Array [1..VMFPLC2_Max_Blks] of VMFPLC2_Data; End; Procedure Tape_Write( BAddr: Word; BLen: Integer); Var Header: Tape_Header_Rec; Cur: Integer; First_Blk: Boolean; Last_Blk: Boolean; Begin If (BLen < 1) or (BLen > Tape_RSize) then Message(MSG250,ModId,MT_Term,Format( 'Internal error: BLen=%d, Tape_RSize=%d.', Args2( BLen, Tape_RSize))); First_Blk := True; Last_Blk := False; While BLen > 0 do Begin Cur := Min(BLen,Tape_DSize); If Cur = BLen then Last_Blk := True; Header.THR_Size := Byte_Swap(Cur); Header.THR_PSize := Byte_Swap(Tape_PSize); Header.THR_Flags := 0; If First_Blk then Set_Bits(Header.THR_Flags,THRF_NewRec); If Last_Blk then Set_Bits(Header.THR_Flags,THRF_EndRec); Tape_Put(Addr(Header),Sizeof(Header)); Tape_Put(BAddr,Cur); Tape_PSize := Cur; Inc_By(BAddr,Cur); Dec_By(BLen,Cur); First_Blk := False; End; End; ###### From: jmaynard@thebrain.conmicro.cx (Jay Maynard) Newsgroups: alt.sys.pdp10,alt.folklore.computers Subject: Re: New Tape Container format Date: 22 Dec 2000 00:58:36 GMT Organization: Neosoft (using Airnews.net!) Lines: 20 Message-ID: <473F6ABF55EFA7A0.DD2627AF7A323FB6.614375FE146EB49E@lp.airnews.net> X-Orig-Message-ID: References: <4ub43tskou7ev3slch65fvp65mju167q2c@4ax.com> <90u806$9iu$1@spies.com> <81638778B12FE05A.630746B8B0E36C7A.7F492EA256DD3A6B@lp.airnews.net> Reply-To: jmaynard@conmicro.cx Abuse-Reports-To: abuse at airmail.net to report improper postings NNTP-Proxy-Relay: library2.airnews.net NNTP-Posting-Time: Thu Dec 21 18:58:36 2000 NNTP-Posting-Host: !^Eh51k-W1>8$.D (Encoded at Airnews!) User-Agent: slrn/0.9.5.4 (UNIX) Path: chonsp.franklin.ch!pfaff.ethz.ch!news-zh.switch.ch!newsfeed-zh.ip-plus.net!news.ip-plus.net!News.Amsterdam.UnisourceCS!skynet.be!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!howland.erols.net!news-out.nntp.airnews.net.MISMATCH!cabal10.airnews.net!news.airnews.net!cabal14.airnews.net!news.airnews.net!cabal1.airnews.net!news-f.iadfw.net!jmaynard Xref: chonsp.franklin.ch alt.sys.pdp10:2061 alt.folklore.computers:71189 On Thu, 21 Dec 2000 19:09:35 -0500, Bruce Crabill wrote: >In article , aek@spies.com (Al >Kossow) wrote: [actually quoting a previous post of mine] >> Does that cover the bases? I'm guessing at the function of the NEWREC >> and ENDREC flags; since an IBM tape can only have 65535 bytes in a >> record, I've never seen the case where segmentation is needed (and the >> Hercules tapecopy program, used to create AWSTAPE files from real >> tapes, is hardcoded to set both flags on data records because of this). >Actually, I believe you could write tape records longer than 64K by using >the Chain Data flag in the CCWs. 3420s probably didn't care how much >data they were asked to write, but there might have been problems on the >buffered Control Units like the 3480. Good question. I got called on that one not long after posting that; it seems that I'd been thinking in terms of BSAM and QSAM, both of which have a 32K limit, and totally forgot things like geophysical tapes which can have records in the hundreds of megabytes. At some point, I might update tapecopy to handle segmenting records, but it'd be nice to see an example of a file with long records before doing that.