Previously the generated database/database.h was written to the build
top and exposed with -I${MAGICDIR}. Out-of-tree that placed the raw
build directory on the compiler's include search path.
Introduce ${GENINC}, the single include dir for generated headers:
* in-tree (build top == source top): ${MAGICDIR}, so the header stays
at $(builddir)/database/database.h and -I${GENINC} is exactly the old
-I${MAGICDIR} -- behaviour is unchanged.
* relocated (out-of-tree): ${MAGICDIR}/include, so the header is written
to $(builddir)/include/database/database.h and only that clean
include/ dir is on the search path; the raw build top never is.
* scripts/defs.mak.in: derive GENINC from BUILD_TOP vs MAGICSRC; add
DATABASE_H = ${GENINC}/database/database.h; CPPFLAGS uses -I${GENINC}.
* Makefile.in: the makedbh rule targets ${DATABASE_H} (mkdir its dir);
every prereq and the distclean use ${DATABASE_H}.
* database/Makefile: the fallback delegate targets ${DATABASE_H}.
Verified: relocated `make modules` builds all 33 module libraries with
the header at build/include/database/database.h and no raw build-top on
any -I line; in-tree the header stays at database/database.h and the
-I set is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The non-Tcl ("standard") configuration links the bundled GNU readline,
and that path was broken two ways:
* headers: readline's own headers use <readline/*.h> includes, so the
readline source's parent must be on the include path. Nothing added
it, so every module including readline (commands, textio, ...) failed
with "readline/rlstdc.h: No such file". Add -I${MAGICSRC}/readline to
CPPFLAGS for the bundled build only (@MAKE_READLINE@ = 1).
* library: readline/Makefile ran "../readline/configure" from the
build-readline dir -- a source-relative path that, out-of-tree, points
into the (empty) build tree. Use ${MAGICSRC}/readline/readline/
configure so the out-of-source readline build finds its sources.
Verified: `mkdir b; cd b; ../configure --without-tcl && make` now
completes (rc=0) and produces the native magic binary; previously it
died at commands/CmdFI.o (headers) then at the readline library build.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When latex and dvips are available, the default no-arg "make" now
regenerates the PostScript documentation from its .tex sources into the
build tree; otherwise the pre-built PostScript shipped in the source tree
is used as-is at install time. Documentation is always available either
way -- installing latex simply refreshes it from source.
configure (scripts/configure.in, regenerated scripts/configure):
* AC_PATH_PROG for latex + dvips; DOCS_TARGET = "docs" only when both are
found; AC_SUBST(LATEX/DVIPS/DOCS_TARGET).
* Configuration summary prints "LaTeX docs: yes/no".
scripts/defs.mak.in: export LATEX/DVIPS.
Makefile.in:
* all: ... $(DOCS_TARGET) -- pulls in "docs" only when latex was found.
* docs: best-effort -- `make -k` + a warning on failure so a document
that will not typeset (or a missing texlive package) never fails the
build; install falls back to the pre-built PostScript for those.
doc/latexfiles/Makefile:
* use ${LATEX}/${DVIPS} from defs.mak.
* the .tex include figures as ../psfigures/*.ps; point the build-tree
../psfigures at ${MAGICSRC}/doc/psfigures (no-op in-tree) so latex and
dvips find them while outputs stay in the build tree.
CI: main.yml and main-aarch64.yml install
texlive-latex-base/-recommended + texlive-fonts-recommended so the Linux
builds exercise doc regeneration (best-effort, never fatal).
README.Tcl: document latex/dvips as an optional prerequisite for docs.
.gitignore: ignore in-tree doc build artifacts (doc/latexfiles/psfiles/,
*.dvi/*.aux/*.log); the pre-built doc/psfiles/ stays tracked.
Verified out-of-tree with TeX Live 2024: configure reports "LaTeX docs:
yes"; `make` regenerates 27/28 documents into the build tree (maint1.tex
has a genuine LaTeX error and falls back), stays exit 0; `make install`
installs the 27 regenerated PostScript from the build tree and maint1
from the source pre-built copy; the source tree is not modified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fix the remaining non-standard Makefiles so a full out-of-tree
`../configure && make && make install` builds and installs a complete
magic TCL tree, sources read from the source tree, all outputs in the
build tree.
Shared linker/preprocessor paths (scripts/configure.in, regenerated
scripts/configure):
* --version-script=${MAGICDIR}/magic/symbol.map -> ${MAGICSRC}/... so
every shared-library link (tclmagic.so, ...) finds the source map.
* MCPP preproc.py path ${MAGICDIR}/scripts -> ${MAGICSRC}/scripts.
Programs (built via tcllibrary, PROGRAMS = magic tcltk):
* magic/Makefile: proto.magicrc, magicWasm.o and the bitmaps/.initrc/
magicps.pro install copies use $< / $(srcdir) instead of bare names.
* tcltk/Makefile: magicexec/magicdnull compile $<; the sed'd launcher
scripts (magic.tcl/.sh, ext2spice.sh, ext2sim.sh) read $<; VERSION dep
-> ${MAGICSRC}; install-tcl copies each TCL file from the build dir if
present else $(srcdir) (magic.tcl is generated, the rest are source).
Techs (scmos):
* scmos/Makefile: SC_PP template include -> $(srcdir)/extract_template;
tech-file inputs (scmos.tech.in, *.tech.in) via $(srcdir)/$< ; recurse
into cif_template with mkdir + -f ${MAGICSRC}/scmos/cif_template/Makefile;
install copies from build-or-$(srcdir).
* scmos/cif_template/Makefile: MKDIR -> ${MAGICSRC}/scripts/mkdirs;
SC_CPP gains -I$(srcdir); cpp reads $(srcdir)/cif{in,out}.c so the .gen
includes resolve; objs/ generated in the build tree.
Data-file installs (source files copied from $(srcdir)):
* graphics/Makefile: glyphs, outline fonts, and the X11 helper build.
* windows/Makefile: glyphs and vector fonts.
* doc/Makefile: recurse into man/tutcells/html/latexfiles build-aware;
doc/man, doc/tutcells install with $< ; doc/html tars $(srcdir);
doc/latexfiles reads pre-built PostScript from ${MAGICSRC}/doc/psfiles.
Verified out-of-tree (mkdir _build; cd _build; ../configure): `make`
returns rc=0 (33 module libs, tclmagic.so, magicexec/magicdnull, scmos
tech files + cif_template objs, proto.magicrc), and
`make install DESTDIR=...` returns rc=0 installing 376 files
(bin/magic, tcl/tclmagic.so, sys/ glyphs+fonts+techs+dstyles), with the
source tree left completely clean.
Not exercised by the default TCL build/install and therefore not
converted here: the standalone .so/program variants of
lef/plot/router/ext2spice/ext2sim (their module libs do build), the
disabled oa module (-I. -> -I$(srcdir) still needed if enabled), the
WASM path, and doc/latexfiles PostScript *regeneration* (which still
writes into the source psfiles dir).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a plain `make` in a build directory build every module into the
build tree. The top Makefile becomes a configure-generated Makefile.in,
and each recursion drives a build subdirectory (created on demand) using
the module Makefile from the source tree.
Makefile -> Makefile.in (config.status generates Makefile in the build
top via AC_CONFIG_FILES([Makefile:Makefile.in])):
* Add MAGICSRC = @abs_top_srcdir@; read VERSION and re-run configure from
${MAGICSRC}.
* Every descent `(cd $$dir && ${MAKE} <goal>)` and `${MAKE} -C $(dir $@)
<goal>` becomes:
mkdir -p <builddir> && ${MAKE} -C <builddir> \
-f ${MAGICSRC}/<dir>/Makefile <goal>
so the build subdir need not pre-exist and the source Makefile is used.
Applied to modules/libs/depend pattern rules and the
tcllibrary/mains/techs/install/clean loops.
* database/database.h: generate from ${MAGICSRC}/.../database.h.in into
the build tree (mkdir -p database), via the generated build-tree
${MAGICDIR}/scripts/makedbh.
* install-dirs/install-tcl-dirs use ${SCRIPTS}/mkdirs (now source-tree);
clean skips absent build subdirs; distclean/dist/tags updated for the
split source/build layout.
scripts/defs.mak.in:
* SCRIPTS now points at the source tree (${MAGICSRC}/scripts) for the
static helpers (mkdirs, printmans, ...); the one generated helper,
makedbh, is referenced directly at ${MAGICDIR}/scripts (build tree).
* Guard VPATH so it is set only inside module subdirectories, never at
the build top -- a top-level VPATH into the source tree could let a
stale source database/database.h shadow the build-tree copy.
scripts/configure.in: AC_CONFIG_FILES([Makefile:Makefile.in]); regenerated
scripts/configure.
Verified out-of-tree: `../configure && make modules && make libs` builds
all 33 module libraries (lib*.o and lib*.a) plus the generated
database/database.h into the build tree, with no errors and the source
tree left clean. A full `make` (link the magic program, build techs)
still needs the later outlier fixes (magic/, scmos/, ...).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the autoconf layer relocatable, teach the shared make
files to read sources from the source tree and write objects into the
build tree. The ~26 "standard" and "near-standard" module Makefiles now
build out-of-tree with no per-module edits.
scripts/defs.mak.in:
* Add MAGICSRC = @abs_top_srcdir@ (absolute source top) and derive each
module's srcdir/VPATH from CURDIR relative to the build top:
BUILD_TOP := $(abspath $(MAGICDIR))
MODULE_SUBPATH := $(subst $(BUILD_TOP),,$(CURDIR))
srcdir := $(MAGICSRC)$(MODULE_SUBPATH)
VPATH := $(srcdir)
(MAGICDIR keeps meaning the *build* top; in-tree the trees coincide.)
* CPPFLAGS: add -I${MAGICSRC} (source top, for "<module>/<hdr>.h") and
-I${srcdir} (same-dir headers), keeping -I${MAGICDIR} for the generated
database/database.h in the build top.
* MAGIC_VERSION/REVISION read ${MAGICSRC}/VERSION; MAGIC_COMMIT uses
`git -C ${MAGICSRC}` so it works from a build dir outside the work-tree.
rules.mak:
* Compile rule: `-c $*.c` -> `-c $< -o $@` so the source is found via
VPATH in srcdir and the object is written to the build cwd.
* Depend and tags rules: prefix sources with $(srcdir)/ (they are passed
to the compiler/ctags directly, which do not honor VPATH).
scripts/configure.in: AC_CONFIG_FILES([rules.mak:rules.mak]) copies the
VPATH-aware rules.mak into the build top, so a single ${MAGICDIR}
resolves both `include`s in every subdir Makefile. Regenerated configure.
Known limitation (v1): the Depend post-processing sed strips
absolute-path headers, so with an absolute srcdir most header deps are
dropped; out-of-tree incremental header-change rebuilds are weaker.
Depend is already "optimistic/optional"; relativizing is a follow-up.
Verified out-of-tree: `tiles` and `drc` (the latter #includes the
generated database/database.h) build to completion; all objects land in
the build tree; the source tree stays clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enable `mkdir build; cd build; ../configure` to run entirely in the build
directory, with generated config landing there instead of in the source
tree. This is the autoconf-layer groundwork for a full VPATH build; the
make layer (defs.mak/rules.mak VPATH support, per-dir orchestration) is
still to come, so an out-of-tree `make` is not yet expected to work.
Changes:
* configure (wrapper): run the generated config.status in the *current*
directory with --srcdir pointing back at the source tree (instead of
`cd scripts`), so outputs land in the build dir. Uses an absolute
basedir so any build-dir location works, not just a subdir of srcdir.
* scripts/configure.in:
- AC_CONFIG_AUX_DIR(.) -> AC_CONFIG_AUX_DIR(scripts). Once srcdir
resolves to the true top (scripts' parent, where rules.mak lives),
autoconf searches for install-sh/config.sub/... relative to srcdir;
"." pointed at the wrong place and out-of-tree configure died with
"cannot find install-sh".
- Name the .in templates explicitly
(defs.mak:scripts/defs.mak.in, scripts/makedbh:scripts/makedbh.in)
so config.status finds them from a separate build dir. defs.mak is
written to the build top; makedbh under build/scripts/ to match
${SCRIPTS}/makedbh.
- Drop the trailing `cp defs.mak ..`; defs.mak is now generated in the
build top directly, and from a build dir the copy would clobber the
source tree's defs.mak.
* scripts/configure: regenerated with autoconf 2.69.
Verified: out-of-tree `../configure` completes rc=0, writes defs.mak /
scripts/makedbh / config.status into the build dir, records
srcdir=<source top>, and leaves the source tree's tracked files and
generated defs.mak/makedbh untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Also fixed up CC/CXX/CXXFLAGS in the same way.
Main notable concern the tclmagic.so now has reordered position of
$LDFLAGS (was start AND end of list, now start of list only),
$LDDL_FLAGS would have already provided it start of list, so it
would be duplicated set of options.
@LDDL_FLAGS@ no longer prepend $LDFLAGS which is now explicit in the
Makefile to allow specific target control.
The purpose of all this is the toplevel ./configure now has LDFLAGS
control passed down to linker command lines as configure --help
indicates.
Bump VERSION to 8.3.645.
magic.wasm can now be built as two variants packaged in the same npm
release: notcl/ (legacy, magic's own parser) and tcl/ (intubun/tcl 9.x
statically linked, commands evaluated by Tcl_EvalEx). The TCL fork is
pinned via npm/tcl.ref and cloned/built by magic itself — the tcl/
checkout is treated as read-only and built out-of-source into
magic/build-tcl-wasm/.
Configure layer:
- New usingTk variable decoupled from usingTcl in scripts/configure.in
+ scripts/configure, so --with-tcl --without-tk is finally a valid
combination. Native Linux Tcl+Tk builds keep their previous behaviour
(both flags default to enabled).
- When usingTk is empty, configure passes -DMAGIC_NO_TK so the small
number of remaining Tk callsites in tcltk/tclmagic.{h,c} compile out,
and TKCOMMON_SRCS / USE_TK_STUBS are omitted from the link.
WASM build orchestration:
- toolchains/emscripten/build-tcl-wasm.sh builds libtcl9.x.a + libtclstub.a
+ tclConfig.sh out-of-source from a pristine intubun/tcl checkout.
- npm/build.sh grew a --variant=<tcl|notcl|both> flag and writes its
outputs into npm/tcl/ and npm/notcl/. It also clones intubun/tcl with
autocrlf=false at the SHA pinned by npm/tcl.ref.
- magic/Makefile (WASM block only): magicWasm.o is now compiled with
DFLAGS_NOSTUB so Tcl_CreateInterp resolves to libtcl9.x directly
before tclStubsPtr is set. magic.js link pulls in LIB_SPECS_NOSTUB
and -ltclstub. After rules.mak include, magic: is a phony alias for
magic.js so the generic ${MODULE} recipe doesn't fight it.
- toolchains/emscripten/defs.mak: add -sUSE_ZLIB=1 (libtcl9 references
zlib), replace -sSTACK_SIZE=N with -Wl,-z,stack-size=N (emcc >=5
rejects the setting form).
- magic/magicWasm.c bootstraps the embedded interp under MAGIC_WRAPPER
(Tcl_CreateInterp -> Tcl_Init -> Tclmagic_Init) and routes
run_command through Tcl_EvalEx.
- magic/magicTop.c: gate MagicVersion/Revision/CompileTime on
!MAGIC_WRAPPER so they don't collide with the copies in
tcltk/tclmagic.c when both objects land in the same wasm binary.
npm package:
- Subpath exports: ".", "./tcl", "./notcl". Default import keeps the
pre-existing non-TCL behaviour for backward compatibility.
- examples/smoke-tcl.mjs exercises the TCL variant.
CI:
- main-wasm.yml clones intubun/tcl at the pinned ref, builds both
variants via npm/build.sh --variant=both, runs the existing notcl
test suite and the new TCL smoke test, and publishes only on a
v<x.y.z>... git tag. Tag name (minus the leading v) becomes the
npm version.
./configure
# If you are brave, enable with your favourite editor after ./configure
sed -e 's/^#FEATURE_FLAGS /FEATURE_FLAGS /' -i defs.mak
make
make install
This is a standard autoconf variable, it was probably disabled due to
how the graphics/Makefile works and the more complex detection taking
place there.
Needed to use it to add -lpthread to some platforms (freebsd/openbsd)
to make it build out the box.
This split has the effect (with modified .o.c Makefile rule of
correctly using the $CC for .c files and not try to build .cpp
with C compiler, which fails in oa/** for example.
This can only cause trouble being a global default setting.
All project include files are qualified with a path prefix to help
disambiguate duplication of header file names to effectively provide
a namespace.
When removing it nothing seemed break. Maybe more configurations
should be tested. They were all ok.
Previously the variable resolution was passed to compiler command line
when ideally this only needs to be resolved once per build and the
entrie build uses the same value.
Compiler command line is computed at least once per file (plus Depends
pass), so this is forking at least 5 extra times during the build.
This improves things somewhat so resolution is performed less.
This may have the effect of making the build appear faster when
ccache is working. As the command line options maybe seen as a
constant.
Darryl L. Miles
Enno Schnackenberg