With the bundled fallback gone (Patch 1) the readline/ special-casing is dead
machinery. Remove it:
* Makefile.in: $(submake) drops the `= readline` `-f`/mkdir branch and is now
plain `$(MAKE) -C <dir> <goal>`; the clean loop drops its matching branch;
`readline` is removed from BUNDLED_MODULES.
* scripts/defs.mak.in: remove the `ifeq (@MAKE_READLINE@,1) -I${MAGICSRC}/
readline` block (system readline headers are on the standard search path) and
the MAKE_READLINE variable; tidy the srcdir-fallback comment (only the
top-level Makefile uses it now).
* scripts/configure.in: drop AC_SUBST(MAKE_READLINE); regenerate configure.
The system readline link flags (rl_libs = -lreadline via READLINE_LIBS) are
unchanged. readline/ itself is deleted in the next patch.
Verified: no unsubstituted @MAKE_READLINE@; Tcl -j12 builds+installs 375 files
with readline/ not built; --without-tcl --disable-readline -j12 builds (x2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main@ depends on both `modules` and `libs`, so `module@X` (`make -C X module`)
and `lib@X` (`make -C X lib`) could run in the same directory concurrently under
-j and race on X's shared objects -- each %.o recipe does `rm $*.o; cc -c`, so
one make would delete/rebuild an object while the other was linking it, e.g.
`ld: cannot find touchtypes.o` while linking libutils.o. It was rare (timing
dependent) but real; it surfaced on a --without-tcl --disable-readline -j12 build.
Make `lib@%` depend on its own `module@%`: module@X builds the objects, lib@X
then only archives them, so the two never touch X's *.o at the same time.
Cross-directory parallelism is unchanged (lib@X waits only on module@X, not the
whole modules phase).
Verified: 4/4 clean --without-tcl -j12 builds pass (previously intermittently
failed); Tcl -j12 still builds 375 files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From an independent review of the branch:
* Makefile.in: `tcltk` is in both INSTALL_CAD_DIRS and PROGRAMS (Tcl build),
so install@tcltk / install-tcl@tcltk were defined twice -> "given more than
once" warnings on every command. Wrap both install lists in $(sort ...) to
dedup (as CLEAN_TARGETS already does). Also fix a stale comment (module@ now
waits for ${DATABASE_H}, not a `depend` phase).
* toolchains/emscripten/defs.mak: the scmos display-file embed globbed
`mos.*.dstyle`, which misses mos.7bit.mraster_dstyle (an underscore, not a
dot) -- so the mono-raster style was not embedded in WASM, and the comment
wrongly called that file non-existent. Glob `mos.*dstyle` to cover both.
* npm/build.sh: sed_strip_cr wrote the file back unconditionally, bumping the
mtimes of configure and every scripts/ file on each WASM build (contradicting
the "source never mutated" guarantee and able to force a coexisting native
rebuild). Only write back when a CR was actually stripped.
* graphics/Makefile.in: remove the dead DEPSRCS block left behind by retiring
the Depend machinery (it also referenced an undefined ${THREE_D_SRCS}).
* scripts/defs.mak.in: clarify that only the SOURCE_DATE_EPOCH branch is UTC.
Verified: 0 duplicate-target warnings, wildcard now lists mos.7bit.mraster_dstyle,
full Tcl build+install rc=0 (375 files), graphics lib builds, build.sh syntax OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With per-file dependencies now generated during compilation (previous commit),
the serial `depend` phase and its barrier are obsolete. Drop SUBDIRS_DEPEND and
the `<dir>/Depend: ${DATABASE_H}` rule, and replace the `module@/lib@/tech@ %:
depend` prerequisite with `: ${DATABASE_H}` -- the only real ordering constraint
is the generated header, not a whole dependency-scanning pass. Modules therefore
start compiling in parallel immediately instead of waiting for every subdir's
deps to be scanned first.
`make depend` is kept as a top-level no-op (the WASM build script still calls
`emmake make depend`).
Verified out-of-tree serial and -j12 and in-tree -j12: rc=0, 375 files, serial
and -j installs byte-identical, zero Depend files, source tree clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes make a parallel `make -jN` deterministic:
1. lib@database: module@tiles module@utils. libdatabase.a physically
archives tiles/libtiles.o and utils/libutils.o (database/Makefile.in
LIB_OBJS +=), so lib@database must wait for those module objects. A
serial build got this from prerequisite order; under -j it was a latent
race (it happened to win most of the time because tiles/utils are quick).
Audited all Makefile.in: no other default-built module archives another
module's objects (magic's EXTRA_LIBS is covered by main@: modules libs;
the ext2*/router tools are not in the default build).
2. docs runs the doc/latexfiles sub-make with -j1. Those rules share
per-run .aux/.log files in one directory and are not parallel-safe;
under an inherited jobserver they raced and dropped documents (5-9 of 28
failed under -j vs 1 serial). Doc regeneration is best-effort (pre-built
PostScript is the install fallback), so forcing it serial costs nothing.
Verified: 5x clean out-of-tree `make -j12` all rc=0 with zero core-build
errors and a stable 27/28 docs; in-tree -j12 installs 375 files;
--without-tcl -j12 builds readline + magic. Serial behaviour unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the shell `for dir` loops and the FORCE pattern-rule hack with a
uniform per-directory PHONY target scheme: each (phase, dir) is a target
"<phase>@<dir>" that recurses into <dir> with that phase's goal via
$(submake), and each phase aggregate (depend, modules, libs, techs,
install, install-tcl, clean, mains, tcllibrary) is just the list of its
per-dir targets as prerequisites.
Inter-phase ordering moves from the aggregate onto the per-dir targets
(module@/lib@/tech@ each require `depend`; main@ requires modules+libs),
so the dependency graph is correct under `make -jN`, not only in a
left-to-right serial build. '@' is a safe namespace char (never in a real
target name). readline/ still takes the -f path inside $(submake).
The `depend` rule is left as real file targets (<dir>/Depend) so Depend is
still regenerated incrementally, not on every build. clean-mains loses its
one-iteration shell loop too.
The one cross-module edge (libdatabase.a needs tiles/utils modules first)
is preserved by prerequisite order for serial builds here; its explicit
-j edge is fixed in the following commit.
Verified serial rc=0, 375 files, identical installs in-tree and
out-of-tree; --without-tcl builds readline + magic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rules.mak has no @...@ substitutions, so there is no reason to copy it into
the build top. Point all 38 subdir Makefile.in includes at the source copy
directly -- ${MAGICSRC}/rules.mak instead of ${MAGICDIR}/rules.mak -- and
remove AC_CONFIG_FILES([rules.mak:rules.mak]) from configure. defs.mak (the
one make fragment with substitutions) remains the only generated file in the
build top. readline/ and the top Makefile never included rules.mak, so they
are unaffected; AC_CONFIG_SRCDIR(rules.mak) (the srcdir sentinel) is kept.
Now editing rules.mak in the source tree takes effect without reconfiguring.
Verified rc=0, 375 files, both in-tree and out-of-tree; out-of-tree build
top has no rules.mak; distclean rc=0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every module is now a configure-generated Makefile, so the
generic "use -f if there is no build Makefile" fallback in $(submake) and
in the clean loop is only ever reached by readline/ (the one module that
is intentionally never converted).
Replace the generic `test -f $dir/Makefile` probe with an explicit
`$dir = readline` check: recurse natively everywhere, take the -f path
(against the source Makefile, creating the build subdir on demand) only
for readline. A converted module whose build Makefile is somehow missing
now fails loudly instead of silently falling back to -f.
Verified out-of-tree: default (Tcl) make+install+clean rc=0 (375 files);
--without-tcl make+clean rc=0 with readline/libreadline.a + magic/magic
built through the readline -f path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the last nested children to Makefile.in and switch their parents
from the -f fallback to native recursion (they now have generated build
Makefiles). Rename and rewire are one unit: a pure rename would break the
parent's -f path to the (now-.in) child.
Converted (5): scmos/cif_template, doc/man, doc/tutcells, doc/html,
doc/latexfiles. Pure renames -- still build via the srcdir derivation;
@srcdir@ substitution follows.
Rewired to native `make -C <child>` (config.status now creates the child
build dirs + Makefiles):
* scmos/Makefile.in: cif_template build + clean.
* doc/Makefile.in: install/install-tcl over man/tutcells/html/latexfiles.
* Makefile.in: the docs: target -> `make -k -C doc/latexfiles all`.
The `mkdir -p` and `-f ${MAGICSRC}/.../Makefile` are dropped in each.
* configure.in: AC_CONFIG_FILES for the 5 children; regenerated configure.
* .gitignore: ignore each generated child Makefile, same commit.
Verified out-of-tree: full `make && make install` rc=0, 376 files --
scmos recurses natively into cif_template (8 objs), docs regenerate via
native doc/latexfiles (27/28), man pages install via native doc/man.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabler for the Makefile -> Makefile.in migration. No module is
converted yet; behaviour is identical.
* scripts/defs.mak.in: dual-mode. A module converted to Makefile.in will
set `srcdir` (and VPATH) from @srcdir@ before including defs.mak; only
derive them (the $(abspath)/$(patsubst) srcdir computation) when it
did not -- `ifeq ($(origin srcdir),undefined)`. BUILD_TOP (needed by
GENINC) is computed either way.
* Makefile.in: route every subdir recursion through a single helper,
$(call submake,<dir>,<goal>)
which uses the generated build Makefile if config.status made one there
(a converted Makefile.in) and otherwise falls back to the source
Makefile with -f. Today nothing is converted, so out-of-tree always
takes the -f path and in-tree uses the (identical) static Makefile
natively. clean/ keeps its own no-mkdir variant. readline/ is
intentionally never converted and always takes -f.
* .gitignore: document the migration policy -- each converted module adds
its generated "<dir>/Makefile" here in the same commit so an in-tree
build can never commit a generated Makefile. (None converted yet.)
Verified in-tree and out-of-tree: `configure && make && make install`
each produce 376 files with the header in the right place (database/
database.h in-tree, include/database/database.h relocated), 27/28 docs,
source tree clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The generated header's path now depends on in-tree vs relocated builds
(database/database.h vs include/database/database.h), so instructions
that named it directly ("make database/database.h") are brittle.
Add a stable, path-independent top-level target:
.PHONY: prepare
prepare: ${DATABASE_H}
Being .PHONY the name is always valid, but its prerequisite is the real
file target ${DATABASE_H}, so the header is still regenerated only when
out of date -- same result, no exposed path.
Convert every "make database/database.h" in the docs, appimage
Dockerfiles, and CI workflows to "make prepare".
Verified: relocated `make prepare` generates include/database/database.h;
re-running is a no-op; touching database.h.in triggers regeneration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously the generated database/database.h was written to the build
top and exposed with -I${MAGICDIR}. Out-of-tree that placed the raw
build directory on the compiler's include search path.
Introduce ${GENINC}, the single include dir for generated headers:
* in-tree (build top == source top): ${MAGICDIR}, so the header stays
at $(builddir)/database/database.h and -I${GENINC} is exactly the old
-I${MAGICDIR} -- behaviour is unchanged.
* relocated (out-of-tree): ${MAGICDIR}/include, so the header is written
to $(builddir)/include/database/database.h and only that clean
include/ dir is on the search path; the raw build top never is.
* scripts/defs.mak.in: derive GENINC from BUILD_TOP vs MAGICSRC; add
DATABASE_H = ${GENINC}/database/database.h; CPPFLAGS uses -I${GENINC}.
* Makefile.in: the makedbh rule targets ${DATABASE_H} (mkdir its dir);
every prereq and the distclean use ${DATABASE_H}.
* database/Makefile: the fallback delegate targets ${DATABASE_H}.
Verified: relocated `make modules` builds all 33 module libraries with
the header at build/include/database/database.h and no raw build-top on
any -I line; in-tree the header stays at database/database.h and the
-I set is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When latex and dvips are available, the default no-arg "make" now
regenerates the PostScript documentation from its .tex sources into the
build tree; otherwise the pre-built PostScript shipped in the source tree
is used as-is at install time. Documentation is always available either
way -- installing latex simply refreshes it from source.
configure (scripts/configure.in, regenerated scripts/configure):
* AC_PATH_PROG for latex + dvips; DOCS_TARGET = "docs" only when both are
found; AC_SUBST(LATEX/DVIPS/DOCS_TARGET).
* Configuration summary prints "LaTeX docs: yes/no".
scripts/defs.mak.in: export LATEX/DVIPS.
Makefile.in:
* all: ... $(DOCS_TARGET) -- pulls in "docs" only when latex was found.
* docs: best-effort -- `make -k` + a warning on failure so a document
that will not typeset (or a missing texlive package) never fails the
build; install falls back to the pre-built PostScript for those.
doc/latexfiles/Makefile:
* use ${LATEX}/${DVIPS} from defs.mak.
* the .tex include figures as ../psfigures/*.ps; point the build-tree
../psfigures at ${MAGICSRC}/doc/psfigures (no-op in-tree) so latex and
dvips find them while outputs stay in the build tree.
CI: main.yml and main-aarch64.yml install
texlive-latex-base/-recommended + texlive-fonts-recommended so the Linux
builds exercise doc regeneration (best-effort, never fatal).
README.Tcl: document latex/dvips as an optional prerequisite for docs.
.gitignore: ignore in-tree doc build artifacts (doc/latexfiles/psfiles/,
*.dvi/*.aux/*.log); the pre-built doc/psfiles/ stays tracked.
Verified out-of-tree with TeX Live 2024: configure reports "LaTeX docs:
yes"; `make` regenerates 27/28 documents into the build tree (maint1.tex
has a genuine LaTeX error and falls back), stays exit 0; `make install`
installs the 27 regenerated PostScript from the build tree and maint1
from the source pre-built copy; the source tree is not modified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a plain `make` in a build directory build every module into the
build tree. The top Makefile becomes a configure-generated Makefile.in,
and each recursion drives a build subdirectory (created on demand) using
the module Makefile from the source tree.
Makefile -> Makefile.in (config.status generates Makefile in the build
top via AC_CONFIG_FILES([Makefile:Makefile.in])):
* Add MAGICSRC = @abs_top_srcdir@; read VERSION and re-run configure from
${MAGICSRC}.
* Every descent `(cd $$dir && ${MAKE} <goal>)` and `${MAKE} -C $(dir $@)
<goal>` becomes:
mkdir -p <builddir> && ${MAKE} -C <builddir> \
-f ${MAGICSRC}/<dir>/Makefile <goal>
so the build subdir need not pre-exist and the source Makefile is used.
Applied to modules/libs/depend pattern rules and the
tcllibrary/mains/techs/install/clean loops.
* database/database.h: generate from ${MAGICSRC}/.../database.h.in into
the build tree (mkdir -p database), via the generated build-tree
${MAGICDIR}/scripts/makedbh.
* install-dirs/install-tcl-dirs use ${SCRIPTS}/mkdirs (now source-tree);
clean skips absent build subdirs; distclean/dist/tags updated for the
split source/build layout.
scripts/defs.mak.in:
* SCRIPTS now points at the source tree (${MAGICSRC}/scripts) for the
static helpers (mkdirs, printmans, ...); the one generated helper,
makedbh, is referenced directly at ${MAGICDIR}/scripts (build tree).
* Guard VPATH so it is set only inside module subdirectories, never at
the build top -- a top-level VPATH into the source tree could let a
stale source database/database.h shadow the build-tree copy.
scripts/configure.in: AC_CONFIG_FILES([Makefile:Makefile.in]); regenerated
scripts/configure.
Verified out-of-tree: `../configure && make modules && make libs` builds
all 33 module libraries (lib*.o and lib*.a) plus the generated
database/database.h into the build tree, with no errors and the source
tree left clean. A full `make` (link the magic program, build techs)
still needs the later outlier fixes (magic/, scmos/, ...).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Darryl L. Miles