OPENTEXTSTREAM leaves underlying file stream open after breaks cause premature return (and no textstream object)

[fghalasz]

This issue was discovered in the failures we were seeing in HCFILES. The TEDIT files (from HCFILES) that exhibit the issue are listed below. For each of these files that I've tried. OPENTEXTSTREAM breaks under \TEDIT.OPENTEXTSTREAM.PIECES, usually with some font issue. When you ^ out of the break, the underlying file stream is left open with no easy handle to that file stream since no TEXTSTREAM is returned.

Example:

5<- (SETQ S T)
T
6 <- (NLSETQ (SETQ S (OPENTEXTSTREAM "docs/medley-irm/01-INTRO.TEDIT")))
NIL
7<-  S
T
8<- (OPENP)
(...  #<Input Stream on {DSK}<home>frank>il>medley>docs>medley-irm>01-INTRO.TEDIT;1/167,136000> ...)

One way to fix this is to wrap some sort of ERRORSET around \TEDIT.OPENTEXTSTREAM.PIECES inside OPENTEXTSTREAM to close the underlying file stream before its local binding disappears. But TEdit just too complicated to know if this will have negative side effect elsewhere, so I implemented a work around in HCFILES that uses the (OPENP) at the HCFILES level to close the errant file streams.

This issue asks for a proper fix for OPENTEXTSTREAM that closes the underlying open file stream (if any) when OPENTEXTSTREAM errors out.

Problematic TEDIT files from HCFILES: (Most testing done with files in docs/medley-irm)

docs/dinfo/CHAPBREAKING.TEDIT
notecards/docs/user-guide-v1.2/from_envos/000-OUTLINE.TEDIT
notecards/docs/user-guide-v1.2/from_envos/001-TITLE-PAGE.TEDIT
notecards/docs/user-guide-v1.2/from_envos/002-PREFACE.TEDIT
notecards/docs/user-guide-v1.2/from_envos/003-TOC.TEDIT
notecards/docs/user-guide-v1.2/from_envos/01-INTRODUCTION.TEDIT
notecards/docs/user-guide-v1.2/from_envos/03-SOFTWARE-INSTALLATION.TEDIT
notecards/docs/user-guide-v1.2/from_envos/04-SYSTEM-USE-ISSUES.TEDIT
notecards/docs/user-guide-v1.2/from_envos/05-NOTECARDS-BASICS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/06-BUILDING-NC-STRUCTURES.TEDIT
notecards/docs/user-guide-v1.2/from_envos/07-THE-USER-INTERFACE.TEDIT
docs/dinfo/CHAPCLISP.TEDIT
notecards/docs/user-guide-v1.2/from_envos/08-LINKS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/09-CARDS-&-BANNERS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/10-USER-CARDS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/11-SYSTEM-CARDS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/12-MENUBOX-ICON.TEDIT
notecards/docs/user-guide-v1.2/from_envos/13-PARAMETERS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/14-FILEBROWSER.TEDIT
notecards/docs/user-guide-v1.2/from_envos/15-OTHER-TOOLS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/16-PRINTING.TEDIT
notecards/docs/user-guide-v1.2/from_envos/17-PROBLEMS-&-ERRORS.TEDIT
docs/dinfo/CHAPCOMPILER.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-A-NOTEFILE-CONCEPTS.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-B-NOTEFILE-INSPECTOR.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-C-INITIALIZATION.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-D-CHECKSUM.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-G-GLOSSARY.TEDIT
notecards/docs/user-guide-v1.2/from_envos/APP-Z-PROG-INTERFACE.TEDIT
docs/dinfo/CHAPCONTROLFNS.TEDIT
docs/dinfo/CHAPDWIM.TEDIT
docs/dinfo/CHAPEDITOR.TEDIT
docs/dinfo/CHAPERRORS.TEDIT
docs/dinfo/CHAPEXEC.TEDIT
docs/dinfo/CHAPFILES.TEDIT
docs/dinfo/CHAPFNDEF.TEDIT
docs/dinfo/CHAPFP.TEDIT
docs/dinfo/CHAPGRAPHICS.TEDIT
docs/dinfo/CHAPHARDCOPY.TEDIT
docs/dinfo/CHAPHASHARRAYS.TEDIT
docs/dinfo/CHAPIO.TEDIT
docs/dinfo/CHAPLISTS.TEDIT
docs/dinfo/CHAPLITATOMS.TEDIT
docs/dinfo/CHAPMASTERSCOPE.TEDIT
docs/dinfo/CHAPMISC.TEDIT
docs/dinfo/CHAPNUMBERS.TEDIT
docs/dinfo/CHAPPERFORMANCE.TEDIT
docs/dinfo/CHAPPROCESSES.TEDIT
docs/dinfo/CHAPRECORDPACKAGE.TEDIT
docs/dinfo/CHAPSTACK.TEDIT
docs/dinfo/CHAPSTRINGS.TEDIT
docs/dinfo/CHAPTERMINAL.TEDIT
docs/dinfo/CHAPUSERIO.TEDIT
docs/dinfo/CHAPWINDOWS.TEDIT
docs/internal/DIGguide.TEdit
docs/medley-irm/01-INTRO.TEDIT
docs/medley-irm/02-LITATOM.TEDIT
docs/medley-irm/04-STRINGS.TEDIT
docs/medley-irm/05-ARRAY.TEDIT
docs/medley-irm/06-HASHARRAYS.TEDIT
docs/medley-irm/07-NUMBERS.TEDIT
docs/medley-irm/08-RECORDPACKAGE.TEDIT
docs/medley-irm/09-conditionals.TEDIT
docs/medley-irm/10-FUNC-DEF.TEDIT
docs/medley-irm/11-VAR-BINDINGS.TEDIT
docs/medley-irm/12-MISC.TEDIT
docs/medley-irm/13-EXECUTIVE.TEDIT
docs/medley-irm/14-ERRORS.TEDIT
docs/medley-irm/15-BREAKING.TEDIT
docs/medley-irm/16-SEDIT.TEDIT
docs/medley-irm/17-FILEPACKAGE.TEDIT
docs/medley-irm/18-COMPILER.TEDIT
docs/medley-irm/19-DWIM.TEDIT
docs/medley-irm/20-CLISP.TEDIT
docs/medley-irm/21-PERFORMANCE.TEDIT
docs/medley-irm/22-PROCESSES.TEDIT
docs/medley-irm/23-SSTREAMS.TEDIT
docs/medley-irm/23-STREAMS.TEDIT
docs/medley-irm/24-IO.TEDIT
docs/medley-irm/25-USERIO-PACKAGES.TEDIT
docs/medley-irm/26-GRAPHICS.TEDIT
docs/medley-irm/27-WINDOWS.TEDIT
docs/medley-irm/28-HARDCOPY.TEDIT
docs/medley-irm/29-TERMINAL.TEDIT
docs/ReleaseNote/APPENDIXD-FREEMENU.TEDIT
internal/envos/LISPDIAGNOSTICS.TEDIT
docs/dinfo/CHAPACK.TEDIT
library/lafite/LAFITE-MAILSCAVENGE.TEDIT
lispusers/tmax/TMAX.TEDIT
notecards/library/bitmapeditor.ted
notecards/library/nccasecluster.ted
notecards/library/ncclustercard.ted
notecards/library/nccollaboration.ted
notecards/library/nchistorycard.ted
notecards/library/ncplotcard.ted
notecards/library/nctabletopcard.ted
notecards/library/nctoulmincard.ted
obsolete/lispusers/COLOROBJ.TEDIT
loops/doc/manual/CUSTOMIZEMASTERSCOPE.TEDIT
notecards/docs/misc/notefileinspector.ted
notecards/docs/misc/progintface.ted
notecards/docs/misc/releaseannouncement.ted
notecards/docs/misc/releasenotes.ted
notecards/docs/user-guide-v1.2/03-SOFTWARE-INSTALLATION.TEDIT
notecards/docs/user-guide-v1.2/04-SYSTEM-USE-ISSUES.TEDIT
notecards/docs/user-guide-v1.2/05-NOTECARDS-BASICS.TEDIT
notecards/docs/user-guide-v1.2/11-SYSTEM-CARDS.TEDIT
docs/dinfo/CHAPARRAYS.TEDIT
notecards/docs/user-guide-v1.2/APP-Z-PROG-INTERFACE.TEDIT
library/lafite/docs/release-notes/Lafite-85.TEdit
library/lafite/docs/release-notes/LAFITEDELTA.TED
library/lafite/docs/users-guide/LAFITEMANUAL-INDEXINTERNAL.TEDIT
loops/obsolete/from1.1/old/LOOPSIMAGEOBJECTS.TEDIT
notecards/docs/misc/from_envos/notefileinspector.ted
notecards/docs/misc/from_envos/progintface.ted
notecards/docs/misc/from_envos/releaseannouncement.ted
notecards/docs/misc/from_envos/releasenotes.ted
notecards/docs/misc/from_envos/typesmech.ted

[rmkaplan]

There was an internal bug (probably introduced in a recent round) in reading old-format Tedit files (version 2 from around 1985 I think). That may have accounted for some of the failures in the notecards files. But some of the files have internal failures even in the Venue sysout.

I wrapped a RESETSAVE around the OPENSTREAM in \TEDIT.OPENTEXTSTREAM.PIECES, and it appears that that does the trick when the files really do generate an error under the NLSETQ. I saw one font-not-found error for Modern 13, that looks suspiciously like the file has a CR/LF corruption.

There are lots of warning messages that show up in the promptwindow, usually having to do with image object problems. Those don't cause errors, they would just open with bad image object boxes, but it is usually a signal that something is corrupted. Eventually I'll try those in the Venue sysout to rule out more recent bugs.

I will try to put out a PR that has just this Tedit fix disentangled from the MCCS stuff, hopefully that will solve this issue at least wrt getting HCFILES to go all the way through.

[pamoroso]

Addressed by #2284.