npm/build.sh now builds each WASM variant out-of-source in
build-wasm-<variant> under the repo root (fixed names build-wasm-tcl /
build-wasm-notcl). They are disposable (recreated each run) and must not
be tracked. Confirmed the relocated build dirs receive all output while
the source module dirs stay clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
npm/build.sh built in-tree: it cd'd into the source root, ran
`./configure` there, appended the WASM defs.mak to the source-tree
defs.mak, and copied magic.js/.wasm out of the source magic/ dir.
Run each variant in a dedicated out-of-source build dir instead
(build-wasm-<variant>, overridable via WASM_BUILD_DIR):
* configure is invoked as "$REPO_ROOT/configure" from $build_dir, so
config.status and all generated files land under $build_dir; the
source tree is only read (VPATH).
* the WASM defs.mak is appended to $build_dir/defs.mak.
* artifacts are copied from $build_dir/magic/.
* a fresh `rm -rf $build_dir` per run replaces the old in-tree
`make distclean` (which also removes the need to touch the source
tree, and sidesteps the tracked-rules.mak distclean hazard entirely).
CRLF stripping now targets the source-tree configure inputs explicitly.
Note: build-wasm-<variant> dirs are not gitignored (left per task scope);
they are disposable and recreated each run.
(Static edit; not runtime-verified — emscripten is not available here.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Emscripten link embedded data files via magic-relative paths
(../scmos, ../windows/...), which only resolve when the build dir is the
source dir. Split them by which tree the data lives in:
* generated scmos tech files -> ${MAGICDIR}/scmos (the build tree, where
`make techs` writes them);
* verbatim window glyphs -> ${MAGICSRC}/windows/... (the source tree).
Out-of-tree, ../windows/windows7.glyphs pointed at build/windows/ (which
holds only objects) and the link would fail to embed it. In-tree the two
trees coincide, so the resulting .wasm is unchanged.
(Static edit; not runtime-verified — emscripten is not available here.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
magic/Makefile ran the WASM post-processor via ${MAGICDIR} (the build
top). Out-of-tree that resolves to build/toolchains/... which does not
exist — the script is a source file. Use ${MAGICSRC}. In-tree the two
are identical, so behaviour is unchanged there.
(Static edit; not runtime-verified — emscripten is not available here.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the out-of-tree build to the outliers deferred earlier, and
audit install sourcing. All are read-source / write-build now.
Tcl auto-load shared libraries (built via each module's `tcl-main`;
not part of the default TCL build, but now correct when invoked):
* lef/plot/router/ext2spice/ext2sim: the hand-written object rules
(tcllef.o, tclplot.o, tclroute.o, spicewrap.o, spicehier.o, simwrap.o,
ext2sim_main.o) compiled a bare-name source from the cwd; switch to
`-c $< -o $@` so the source is found via VPATH and the object lands in
the build tree. ext2sim_main.o also swaps the hard-coded -I${MAGICDIR}
for ${CPPFLAGS} (build + source include paths). ext2spice spice2sim
install copies $< (source script). Sibling ${MAGICDIR}/<mod>/lib*.o
refs already resolve into the build tree (MAGICDIR = build top).
oa (disabled by default; OpenAccess):
* CXX_INCLUDES -I. -> -I$(srcdir) -I${MAGICDIR} -I${MAGICSRC} so the C++
sources, generated database/database.h, and cross-module headers all
resolve out-of-tree; .deps and objects were already build-local.
doc/latexfiles PostScript regeneration:
* Split PS_SRCDIR (pre-built PS shipped in the source tree, read-only)
from PS_GENDIR (= psfiles in the build tree). `all` / the .tex.ps
rule now regenerate into PS_GENDIR (mkdir -p; latex reads $(srcdir)
via TEXINPUTS; dvips writes the build tree) instead of writing back
into the source psfiles dir. install takes each .ps from PS_GENDIR if
regenerated there, else the pre-built PS_SRCDIR copy.
Verified out-of-tree (make modules; then per-module tcl-main):
* magiclef.so / tclplot.so / tclroute.so / exttospice.so / exttosim.so
all build into the build tree (rc=0); oa.o builds into the build tree;
source tree stays clean.
* install-tcl copies each .so from the build tree.
* Install-location audit on a full `make install DESTDIR=...`: generated
data (scmos.tech, .magicrc, magic.tcl, tclmagic.so) matches the build
tree; verbatim data (glyphs, fonts, magicps.pro, man pages, doc PS)
matches the source tree.
* DESTDIR: two distinct prefixes install 376 files each, independent and
fully contained (nothing written outside DESTDIR).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The out-of-tree conversion turned the top-level Makefile into a
configure-generated file (from Makefile.in), and config.status now runs
in the build directory (the source top for an in-tree build). Ignore
both at the top level only (anchored) so the tracked per-module
Makefiles are unaffected. config.log is already ignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fix the remaining non-standard Makefiles so a full out-of-tree
`../configure && make && make install` builds and installs a complete
magic TCL tree, sources read from the source tree, all outputs in the
build tree.
Shared linker/preprocessor paths (scripts/configure.in, regenerated
scripts/configure):
* --version-script=${MAGICDIR}/magic/symbol.map -> ${MAGICSRC}/... so
every shared-library link (tclmagic.so, ...) finds the source map.
* MCPP preproc.py path ${MAGICDIR}/scripts -> ${MAGICSRC}/scripts.
Programs (built via tcllibrary, PROGRAMS = magic tcltk):
* magic/Makefile: proto.magicrc, magicWasm.o and the bitmaps/.initrc/
magicps.pro install copies use $< / $(srcdir) instead of bare names.
* tcltk/Makefile: magicexec/magicdnull compile $<; the sed'd launcher
scripts (magic.tcl/.sh, ext2spice.sh, ext2sim.sh) read $<; VERSION dep
-> ${MAGICSRC}; install-tcl copies each TCL file from the build dir if
present else $(srcdir) (magic.tcl is generated, the rest are source).
Techs (scmos):
* scmos/Makefile: SC_PP template include -> $(srcdir)/extract_template;
tech-file inputs (scmos.tech.in, *.tech.in) via $(srcdir)/$< ; recurse
into cif_template with mkdir + -f ${MAGICSRC}/scmos/cif_template/Makefile;
install copies from build-or-$(srcdir).
* scmos/cif_template/Makefile: MKDIR -> ${MAGICSRC}/scripts/mkdirs;
SC_CPP gains -I$(srcdir); cpp reads $(srcdir)/cif{in,out}.c so the .gen
includes resolve; objs/ generated in the build tree.
Data-file installs (source files copied from $(srcdir)):
* graphics/Makefile: glyphs, outline fonts, and the X11 helper build.
* windows/Makefile: glyphs and vector fonts.
* doc/Makefile: recurse into man/tutcells/html/latexfiles build-aware;
doc/man, doc/tutcells install with $< ; doc/html tars $(srcdir);
doc/latexfiles reads pre-built PostScript from ${MAGICSRC}/doc/psfiles.
Verified out-of-tree (mkdir _build; cd _build; ../configure): `make`
returns rc=0 (33 module libs, tclmagic.so, magicexec/magicdnull, scmos
tech files + cif_template objs, proto.magicrc), and
`make install DESTDIR=...` returns rc=0 installing 376 files
(bin/magic, tcl/tclmagic.so, sys/ glyphs+fonts+techs+dstyles), with the
source tree left completely clean.
Not exercised by the default TCL build/install and therefore not
converted here: the standalone .so/program variants of
lef/plot/router/ext2spice/ext2sim (their module libs do build), the
disabled oa module (-I. -> -I$(srcdir) still needed if enabled), the
WASM path, and doc/latexfiles PostScript *regeneration* (which still
writes into the source psfiles dir).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a plain `make` in a build directory build every module into the
build tree. The top Makefile becomes a configure-generated Makefile.in,
and each recursion drives a build subdirectory (created on demand) using
the module Makefile from the source tree.
Makefile -> Makefile.in (config.status generates Makefile in the build
top via AC_CONFIG_FILES([Makefile:Makefile.in])):
* Add MAGICSRC = @abs_top_srcdir@; read VERSION and re-run configure from
${MAGICSRC}.
* Every descent `(cd $$dir && ${MAKE} <goal>)` and `${MAKE} -C $(dir $@)
<goal>` becomes:
mkdir -p <builddir> && ${MAKE} -C <builddir> \
-f ${MAGICSRC}/<dir>/Makefile <goal>
so the build subdir need not pre-exist and the source Makefile is used.
Applied to modules/libs/depend pattern rules and the
tcllibrary/mains/techs/install/clean loops.
* database/database.h: generate from ${MAGICSRC}/.../database.h.in into
the build tree (mkdir -p database), via the generated build-tree
${MAGICDIR}/scripts/makedbh.
* install-dirs/install-tcl-dirs use ${SCRIPTS}/mkdirs (now source-tree);
clean skips absent build subdirs; distclean/dist/tags updated for the
split source/build layout.
scripts/defs.mak.in:
* SCRIPTS now points at the source tree (${MAGICSRC}/scripts) for the
static helpers (mkdirs, printmans, ...); the one generated helper,
makedbh, is referenced directly at ${MAGICDIR}/scripts (build tree).
* Guard VPATH so it is set only inside module subdirectories, never at
the build top -- a top-level VPATH into the source tree could let a
stale source database/database.h shadow the build-tree copy.
scripts/configure.in: AC_CONFIG_FILES([Makefile:Makefile.in]); regenerated
scripts/configure.
Verified out-of-tree: `../configure && make modules && make libs` builds
all 33 module libraries (lib*.o and lib*.a) plus the generated
database/database.h into the build tree, with no errors and the source
tree left clean. A full `make` (link the magic program, build techs)
still needs the later outlier fixes (magic/, scmos/, ...).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the autoconf layer relocatable, teach the shared make
files to read sources from the source tree and write objects into the
build tree. The ~26 "standard" and "near-standard" module Makefiles now
build out-of-tree with no per-module edits.
scripts/defs.mak.in:
* Add MAGICSRC = @abs_top_srcdir@ (absolute source top) and derive each
module's srcdir/VPATH from CURDIR relative to the build top:
BUILD_TOP := $(abspath $(MAGICDIR))
MODULE_SUBPATH := $(subst $(BUILD_TOP),,$(CURDIR))
srcdir := $(MAGICSRC)$(MODULE_SUBPATH)
VPATH := $(srcdir)
(MAGICDIR keeps meaning the *build* top; in-tree the trees coincide.)
* CPPFLAGS: add -I${MAGICSRC} (source top, for "<module>/<hdr>.h") and
-I${srcdir} (same-dir headers), keeping -I${MAGICDIR} for the generated
database/database.h in the build top.
* MAGIC_VERSION/REVISION read ${MAGICSRC}/VERSION; MAGIC_COMMIT uses
`git -C ${MAGICSRC}` so it works from a build dir outside the work-tree.
rules.mak:
* Compile rule: `-c $*.c` -> `-c $< -o $@` so the source is found via
VPATH in srcdir and the object is written to the build cwd.
* Depend and tags rules: prefix sources with $(srcdir)/ (they are passed
to the compiler/ctags directly, which do not honor VPATH).
scripts/configure.in: AC_CONFIG_FILES([rules.mak:rules.mak]) copies the
VPATH-aware rules.mak into the build top, so a single ${MAGICDIR}
resolves both `include`s in every subdir Makefile. Regenerated configure.
Known limitation (v1): the Depend post-processing sed strips
absolute-path headers, so with an absolute srcdir most header deps are
dropped; out-of-tree incremental header-change rebuilds are weaker.
Depend is already "optimistic/optional"; relativizing is a follow-up.
Verified out-of-tree: `tiles` and `drc` (the latter #includes the
generated database/database.h) build to completion; all objects land in
the build tree; the source tree stays clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enable `mkdir build; cd build; ../configure` to run entirely in the build
directory, with generated config landing there instead of in the source
tree. This is the autoconf-layer groundwork for a full VPATH build; the
make layer (defs.mak/rules.mak VPATH support, per-dir orchestration) is
still to come, so an out-of-tree `make` is not yet expected to work.
Changes:
* configure (wrapper): run the generated config.status in the *current*
directory with --srcdir pointing back at the source tree (instead of
`cd scripts`), so outputs land in the build dir. Uses an absolute
basedir so any build-dir location works, not just a subdir of srcdir.
* scripts/configure.in:
- AC_CONFIG_AUX_DIR(.) -> AC_CONFIG_AUX_DIR(scripts). Once srcdir
resolves to the true top (scripts' parent, where rules.mak lives),
autoconf searches for install-sh/config.sub/... relative to srcdir;
"." pointed at the wrong place and out-of-tree configure died with
"cannot find install-sh".
- Name the .in templates explicitly
(defs.mak:scripts/defs.mak.in, scripts/makedbh:scripts/makedbh.in)
so config.status finds them from a separate build dir. defs.mak is
written to the build top; makedbh under build/scripts/ to match
${SCRIPTS}/makedbh.
- Drop the trailing `cp defs.mak ..`; defs.mak is now generated in the
build top directly, and from a build dir the copy would clobber the
source tree's defs.mak.
* scripts/configure: regenerated with autoconf 2.69.
Verified: out-of-tree `../configure` completes rc=0, writes defs.mak /
scripts/makedbh / config.status into the build dir, records
srcdir=<source top>, and leaves the source tree's tracked files and
generated defs.mak/makedbh untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The top-level ./configure wrapper did `cd scripts`, which is resolved
against the caller's current directory. Running it from anywhere other
than the source top (e.g. `mkdir build; cd build; ../configure`) failed
immediately with:
../configure: line 12: cd: scripts: No such file or directory
Derive the script's own location via `dirname "$0"` and cd into
"${basedir}/scripts" so the wrapper works regardless of CWD.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
magic replaces Tcl's stdin channel with its own (TerminalInputProc) during
startup (_magic_startup). The replacement builds a brand-new Tcl channel over
the same fd; the new channel starts with an empty buffer. If anything had
already read() stdin bytes out of the kernel into the *old* channel's buffer
before the swap -- e.g. a script/commands piped in fast enough that Tcl buffered
them during startup -- those bytes were silently dropped, and every command
after that point vanished. This was timing-dependent, so it surfaced only
intermittently ("piped input discarded sometimes").
Fix: before creating the replacement channel, drain any bytes still buffered in
the old channel (Tcl_InputBuffered / Tcl_ReadChars) and hand them to
TerminalInputProc via TxBuffer, which it already serves ahead of any fresh
read(). The swap is now lossless by construction rather than by luck. On the
normal path Tcl_InputBuffered() is 0, so the new branch is inert.
How to reproduce / observe the original loss:
Temporarily force the old channel to buffer input just before the swap, right
after `fsOrig = Tcl_GetChannelInstanceData(oldchannel);` in _magic_startup():
{ Tcl_Obj *l = Tcl_NewObj(); Tcl_IncrRefCount(l);
Tcl_GetsObj(oldchannel, l); /* consume 1 line, buffering the rest */
fprintf(stderr, "buffered=%d\n", (int)Tcl_InputBuffered(oldchannel));
Tcl_DecrRefCount(l); }
then pipe a burst of commands (more than one line):
printf 'puts a\nputs b\n...\nquit -noprompt\n' | ./run_magicnull.sh
Comment the drain out and every command after the first disappears; restore it
and they all survive. Measured with 218 bytes buffered: 0/20 commands survive
without the drain, 19/20 with it (the one line the probe consumed aside).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
For tarball source: show tarball filename, download URL, and actual
SHA256 values with "(SHA matches expected value)" suffix, or
"expected SHA not known, please update workflow .yml" when no SHA is
known. The Tcl ref is omitted since it is derived and not user-supplied.
For github source: show clone URL, ref, and resolved commit-id/date.
Tarball and SHA rows are omitted as they are not relevant.
The TCL/TK BUILD INFO section in Create RELEASE-NOTES.txt is updated
with the same conditional logic using the exported env vars.
The TCL_SHA_STATUS env var is removed (replaced by per-SHA display
logic and the new TCL_TARBALL_FILENAME / TK_TARBALL_FILENAME vars).
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Also fixed up CC/CXX/CXXFLAGS in the same way.
Main notable concern the tclmagic.so now has reordered position of
$LDFLAGS (was start AND end of list, now start of list only),
$LDDL_FLAGS would have already provided it start of list, so it
would be duplicated set of options.
@LDDL_FLAGS@ no longer prepend $LDFLAGS which is now explicit in the
Makefile to allow specific target control.
The purpose of all this is the toplevel ./configure now has LDFLAGS
control passed down to linker command lines as configure --help
indicates.
CoPilot analysis
Bonus: fetch-tags workaround may be removable
The appimage workflows have this comment throughout:
#fetch-tags: true # this should work see actions/checkout~issue#1471
...and then manually run git fetch --tags as a workaround. The fetch-tags input bug referenced
in issue #1471 was fixed in a v4.x patch release. After upgrading to v7, it would be worth
testing fetch-tags: true directly and removing the manual git fetch --tags workaround in the
appimage "Create RELEASE-NOTES.txt" steps — though this is independent of the version upgrade
itself.
Previously it was an archive of an archive (of the 'npm pack' output)
published under the same artifict filename (which becomes the default
download filenmame) which is a headache when you start to download more
than one (like when testing between versions).
Now the 'npm pack' output is attached as-is using its original filename,
but also the extracted contents are attached as standard GitHub
artifact ZIP, using a unique filename (based on the unique build version).
Filter out unwanted tag version as candiates to build with:
refs/tags/core-9-0-b3 UNWANTED
refs/tags/core-9-0-b2 UNWANTED
refs/tags/core-9-0-b1-rc4 UNWANTED
refs/tags/core-9-0-b1 UNWANTED
refs/tags/core-9-0-a3 UNWANTED
refs/tags/core-9-0-a1 UNWANTED
refs/tags/core-9-0-4
refs/tags/core-9-0-3
refs/tags/core-9-0-2
refs/tags/core-9-0-1
refs/tags/core-9-0-0
Previously the lexical order would result in alpha/beta/rc
matching in preference to the stable Tcl version IDs. So our WASM
build would not be against the upstream stable release which is
the intention.
previously node20 was the default, with override to force node22 but
node24 has been out since May 2025 and is current LTS, with node26
out since May 2026 and current stable (but non-LTS).
Darryl L. Miles