build: top Makefile.in + build-dir orchestration (out-of-tree)

Make a plain `make` in a build directory build every module into the
build tree.  The top Makefile becomes a configure-generated Makefile.in,
and each recursion drives a build subdirectory (created on demand) using
the module Makefile from the source tree.

Makefile -> Makefile.in (config.status generates Makefile in the build
top via AC_CONFIG_FILES([Makefile:Makefile.in])):
* Add MAGICSRC = @abs_top_srcdir@; read VERSION and re-run configure from
  ${MAGICSRC}.
* Every descent `(cd $$dir && ${MAKE} <goal>)` and `${MAKE} -C $(dir $@)
  <goal>` becomes:
      mkdir -p <builddir> && ${MAKE} -C <builddir> \
              -f ${MAGICSRC}/<dir>/Makefile <goal>
  so the build subdir need not pre-exist and the source Makefile is used.
  Applied to modules/libs/depend pattern rules and the
  tcllibrary/mains/techs/install/clean loops.
* database/database.h: generate from ${MAGICSRC}/.../database.h.in into
  the build tree (mkdir -p database), via the generated build-tree
  ${MAGICDIR}/scripts/makedbh.
* install-dirs/install-tcl-dirs use ${SCRIPTS}/mkdirs (now source-tree);
  clean skips absent build subdirs; distclean/dist/tags updated for the
  split source/build layout.

scripts/defs.mak.in:
* SCRIPTS now points at the source tree (${MAGICSRC}/scripts) for the
  static helpers (mkdirs, printmans, ...); the one generated helper,
  makedbh, is referenced directly at ${MAGICDIR}/scripts (build tree).
* Guard VPATH so it is set only inside module subdirectories, never at
  the build top -- a top-level VPATH into the source tree could let a
  stale source database/database.h shadow the build-tree copy.

scripts/configure.in: AC_CONFIG_FILES([Makefile:Makefile.in]); regenerated
scripts/configure.

Verified out-of-tree: `../configure && make modules && make libs` builds
all 33 module libraries (lib*.o and lib*.a) plus the generated
database/database.h into the build tree, with no errors and the source
tree left clean.  A full `make` (link the magic program, build techs)
still needs the later outlier fixes (magic/, scmos/, ...).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

scripts/configure

@@ -9296,6 +9296,8 @@ MAKE_CAIRO=$usingCairo
 MAKE_READLINE=$use_bundled_readline
 
 
+ac_config_files="$ac_config_files Makefile:Makefile.in"
+
 ac_config_files="$ac_config_files defs.mak:scripts/defs.mak.in"
 
 ac_config_files="$ac_config_files scripts/makedbh:scripts/makedbh.in"
@@ -10009,6 +10011,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
 for ac_config_target in $ac_config_targets
 do
   case $ac_config_target in
+    "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile:Makefile.in" ;;
     "defs.mak") CONFIG_FILES="$CONFIG_FILES defs.mak:scripts/defs.mak.in" ;;
     "scripts/makedbh") CONFIG_FILES="$CONFIG_FILES scripts/makedbh:scripts/makedbh.in" ;;
     "rules.mak") CONFIG_FILES="$CONFIG_FILES rules.mak:rules.mak" ;;

scripts/configure.in

@@ -1980,6 +1980,7 @@ dnl Name the .in templates explicitly (relative to srcdir) so config.status can
 dnl find them when run from a separate build directory.  Outputs land in the
 dnl build directory (config.status' CWD): defs.mak at the build top, makedbh
 dnl under build/scripts/ (matching ${SCRIPTS}/makedbh = ${MAGICDIR}/scripts).
+AC_CONFIG_FILES([Makefile:Makefile.in])
 AC_CONFIG_FILES([defs.mak:scripts/defs.mak.in])
 AC_CONFIG_FILES([scripts/makedbh:scripts/makedbh.in], [chmod +x scripts/makedbh])
 dnl Copy rules.mak into the build top (it has no @...@ substitutions; this just

scripts/defs.mak.in

@@ -26,7 +26,12 @@ BUILD_TOP              := $(abspath $(MAGICDIR))
 # this cannot mis-fire if BUILD_TOP happens to recur later in the path.
 MODULE_SUBPATH         := $(patsubst $(BUILD_TOP)%,%,$(CURDIR))
 srcdir                 := $(MAGICSRC)$(MODULE_SUBPATH)
+# Only set VPATH inside a module (sub)directory, never at the build top: a
+# top-level VPATH into the source tree could let a stale source-tree
+# database/database.h shadow the copy we must (re)generate in the build tree.
+ifneq ($(MODULE_SUBPATH),)
 VPATH                  := $(srcdir)
+endif
 # --------------------------------------------------------------------------
 
 prefix = @prefix@
@@ -35,7 +40,9 @@ bindir = @bindir@
 libdir = @libdir@
 mandir = @mandir@
 
-SCRIPTS		       = ${MAGICDIR}/scripts
+# Source-tree helper scripts (mkdirs, printmans, preproc.py, ...).  The one
+# generated script, makedbh, lives in the *build* tree at ${MAGICDIR}/scripts.
+SCRIPTS		       = ${MAGICSRC}/scripts
 
 INSTALL = @INSTALL@
 INSTALL_PROGRAM = @INSTALL_PROGRAM@