Remove the need for resetLocalAttrs. Fixes #994, #952.

The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.

util/appmacro/src/main/scala/sbt/appmacro/ContextUtil.scala

@@ -35,10 +35,15 @@ object ContextUtil {
 /** Utility methods for macros.  Several methods assume that the context's universe is a full compiler (`scala.tools.nsc.Global`).
 * This is not thread safe due to the underlying Context and related data structures not being thread safe.
 * Use `ContextUtil[c.type](c)` to construct. */
-final class ContextUtil[C <: Context](val ctx: C) 
+final class ContextUtil[C <: Context](val ctx: C)
 {
 		import ctx.universe.{Apply=>ApplyTree,_}
 
+	val powerContext = ctx.asInstanceOf[reflect.macros.runtime.Context]
+	val global: powerContext.universe.type = powerContext.universe
+	def callsiteTyper: global.analyzer.Typer = powerContext.callsiteTyper
+	val initialOwner: Symbol = callsiteTyper.context.owner.asInstanceOf[ctx.universe.Symbol]
+
 	lazy val alistType = ctx.typeOf[AList[KList]]
 	lazy val alist: Symbol = alistType.typeSymbol.companionSymbol
 	lazy val alistTC: Type = alistType.typeConstructor
@@ -52,12 +57,15 @@ final class ContextUtil[C <: Context](val ctx: C)
 	* (The current implementation uses Context.fresh, which increments*/
 	def freshTermName(prefix: String) = newTermName(ctx.fresh("$" + prefix))
 
-	/** Constructs a new, local ValDef with the given Type, a unique name, 
+	/** Constructs a new, synthetic, local ValDef Type `tpe`, a unique name,
-	* the same position as `sym`, and an empty implementation (no rhs). */
+	* Position `pos`, an empty implementation (no rhs), and owned by `owner`. */
-	def freshValDef(tpe: Type, sym: Symbol): ValDef =
+	def freshValDef(tpe: Type, pos: Position, owner: Symbol): ValDef =
 	{
-		val vd = localValDef(TypeTree(tpe), EmptyTree)
+		val SYNTHETIC = (1 << 21).toLong.asInstanceOf[FlagSet]
-		vd setPos getPos(sym)
+		val sym = owner.newTermSymbol(freshTermName("q"), pos, SYNTHETIC)
+		setInfo(sym, tpe)
+		val vd = ValDef(sym, EmptyTree)
+		vd.setPos(pos)
 		vd
 	}
 
@@ -65,7 +73,7 @@ final class ContextUtil[C <: Context](val ctx: C)
 
 	/** Collects all definitions in the tree for use in checkReferences.
 	* This excludes definitions in wrapped expressions because checkReferences won't allow nested dereferencing anyway. */
-	def collectDefs(tree: Tree, isWrapper: (String, Type, Tree) => Boolean): collection.Set[Symbol] = 
+	def collectDefs(tree: Tree, isWrapper: (String, Type, Tree) => Boolean): collection.Set[Symbol] =
 	{
 		val defs = new collection.mutable.HashSet[Symbol]
 		// adds the symbols for all non-Ident subtrees to `defs`.
@@ -106,17 +114,17 @@ final class ContextUtil[C <: Context](val ctx: C)
 
 	/** Constructs a tuple value of the right TupleN type from the provided inputs.*/
 	def mkTuple(args: List[Tree]): Tree =
-	{
-		val global: Global = ctx.universe.asInstanceOf[Global]
 		global.gen.mkTuple(args.asInstanceOf[List[global.Tree]]).asInstanceOf[ctx.universe.Tree]
-	}
+
+	def setSymbol[Tree](t: Tree, sym: Symbol): Unit =
+		t.asInstanceOf[global.Tree].setSymbol(sym.asInstanceOf[global.Symbol])
+	def setInfo[Tree](sym: Symbol, tpe: Type): Unit =
+		sym.asInstanceOf[global.Symbol].setInfo(tpe.asInstanceOf[global.Type])
 
 	/** Creates a new, synthetic type variable with the specified `owner`. */
 	def newTypeVariable(owner: Symbol, prefix: String = "T0"): TypeSymbol =
-	{
-		val global: Global = ctx.universe.asInstanceOf[Global]
 		owner.asInstanceOf[global.Symbol].newSyntheticTypeParam(prefix, 0L).asInstanceOf[ctx.universe.TypeSymbol]
-	}
+
 	/** The type representing the type constructor `[X] X` */
 	lazy val idTC: Type =
 	{
@@ -136,21 +144,42 @@ final class ContextUtil[C <: Context](val ctx: C)
 	/** >: Nothing <: Any */
 	def emptyTypeBounds: TypeBounds = TypeBounds(definitions.NothingClass.toType, definitions.AnyClass.toType)
 
+	/** Creates a new anonymous function symbol with Position `pos`. */
+	def functionSymbol(pos: Position): Symbol =
+		callsiteTyper.context.owner.newAnonymousFunctionValue(pos.asInstanceOf[global.Position]).asInstanceOf[ctx.universe.Symbol]
+
 	def functionType(args: List[Type], result: Type): Type =
 	{
-		val global: Global = ctx.universe.asInstanceOf[Global]
 		val tpe = global.definitions.functionType(args.asInstanceOf[List[global.Type]], result.asInstanceOf[global.Type])
 		tpe.asInstanceOf[Type]
 	}
 
-	/** Create a Tree that references the `val` represented by `vd`. */
+	/** Create a Tree that references the `val` represented by `vd`, copying attributes from `replaced`. */
-	def refVal(vd: ValDef, pos: Position): Tree =
+	def refVal(replaced: Tree, vd: ValDef): Tree =
+		treeCopy.Ident(replaced, vd.name).setSymbol(vd.symbol)
+
+	/** Creates a Function tree using `functionSym` as the Symbol and changing `initialOwner` to `functionSym` in `body`.*/
+	def createFunction(params: List[ValDef], body: Tree, functionSym: Symbol): Tree =
 	{
-		val t = Ident(vd.name)
+		changeOwner(body, initialOwner, functionSym)
-		assert(vd.tpt.tpe != null, "val type is null: " + vd + ", tpt: " + vd.tpt.tpe)
+		val f = Function(params, body)
-		t.setType(vd.tpt.tpe)
+		setSymbol(f, functionSym)
-		t.setPos(pos)
+		f
-		t
+	}
+
+	def changeOwner(tree: Tree, prev: Symbol, next: Symbol): Unit =
+		new ChangeOwnerAndModuleClassTraverser(prev.asInstanceOf[global.Symbol], next.asInstanceOf[global.Symbol]).traverse(tree.asInstanceOf[global.Tree])
+
+	// Workaround copied from scala/async:can be removed once https://github.com/scala/scala/pull/3179 is merged.
+	private[this] class ChangeOwnerAndModuleClassTraverser(oldowner: global.Symbol, newowner: global.Symbol) extends global.ChangeOwnerTraverser(oldowner, newowner)
+	{
+		override def traverse(tree: global.Tree) {
+			tree match {
+				case _: global.DefTree => change(tree.symbol.moduleClass)
+				case _ =>
+			}
+			super.traverse(tree)
+		}
 	}
 
 	/** Returns the Symbol that references the statically accessible singleton `i`. */
@@ -164,7 +193,6 @@ final class ContextUtil[C <: Context](val ctx: C)
 
 	/** Returns the symbol for the non-private method named `name` for the class/module `obj`. */
 	def method(obj: Symbol, name: String): Symbol = {
-		val global: Global = ctx.universe.asInstanceOf[Global]
 		val ts: Type = obj.typeSignature
 		val m: global.Symbol = ts.asInstanceOf[global.Type].nonPrivateMember(global.newTermName(name))
 		m.asInstanceOf[Symbol]
@@ -176,7 +204,6 @@ final class ContextUtil[C <: Context](val ctx: C)
 	**/
 	def extractTC(tcp: AnyRef with Singleton, name: String)(implicit it: ctx.TypeTag[tcp.type]): ctx.Type =
 	{
-		val global: Global = ctx.universe.asInstanceOf[Global]
 		val itTpe = it.tpe.asInstanceOf[global.Type]
 		val m = itTpe.nonPrivateMember(global.newTypeName(name))
 		val tc = itTpe.memberInfo(m).asInstanceOf[ctx.universe.Type]
@@ -187,8 +214,8 @@ final class ContextUtil[C <: Context](val ctx: C)
 	/** Substitutes wrappers in tree `t` with the result of `subWrapper`.
 	* A wrapper is a Tree of the form `f[T](v)` for which isWrapper(<Tree of f>, <Underlying Type>, <qual>.target) returns true.
 	* Typically, `f` is a `Select` or `Ident`.
-	* The wrapper is replaced with the result of `subWrapper(<Type of T>, <Tree of v>)` */
+	* The wrapper is replaced with the result of `subWrapper(<Type of T>, <Tree of v>, <wrapper Tree>)` */
-	def transformWrappers(t: Tree, subWrapper: (String, Type, Tree) => Converted[ctx.type]): Tree =
+	def transformWrappers(t: Tree, subWrapper: (String, Type, Tree, Tree) => Converted[ctx.type]): Tree =
 	{
 		// the main tree transformer that replaces calls to InputWrapper.wrap(x) with
 		//  plain Idents that reference the actual input value
@@ -197,7 +224,7 @@ final class ContextUtil[C <: Context](val ctx: C)
 			override def transform(tree: Tree): Tree =
 				tree match
 				{
-					case ApplyTree(TypeApply(Select(_, nme), targ :: Nil), qual :: Nil) => subWrapper(nme.decoded, targ.tpe, qual) match {
+					case ApplyTree(TypeApply(Select(_, nme), targ :: Nil), qual :: Nil) => subWrapper(nme.decoded, targ.tpe, qual, tree) match {
 						case Converted.Success(t, finalTx) => finalTx(t)
 						case Converted.Failure(p,m) => ctx.abort(p, m)
 						case _: Converted.NotApplicable[_] => super.transform(tree)

util/appmacro/src/main/scala/sbt/appmacro/Instance.scala

@@ -61,7 +61,7 @@ object Instance
 	* These converted inputs are passed to `builder` as well as the list of these synthetic `ValDef`s.
 	* The `TupleBuilder` instance constructs a tuple (Tree) from the inputs and defines the right hand side of the vals
 	* that unpacks the tuple containing the results of the inputs.
-	* 
+	*
 	* The constructed tuple of inputs and the code that unpacks the results of the inputs are then passed to the `i`,
 	* which is an implementation of `Instance` that is statically accessible.
 	* An Instance defines a applicative functor associated with a specific type constructor and, if it implements MonadInstance as well, a monad.
@@ -70,18 +70,18 @@ object Instance
 	*  while the full check for static accessibility is done at macro expansion time.
 	* Note: Ideally, the types would verify that `i: MonadInstance` when `t.isRight`.
 	* With the various dependent types involved, this is not worth it.
-	* 
+	*
 	* The `t` argument is the argument of the macro that will be transformed as described above.
 	* If the macro that calls this method is for a multi-input map (app followed by map),
 	* `t` should be the argument wrapped in Left.
-	* If this is for multi-input flatMap (app followed by flatMap), 
+	* If this is for multi-input flatMap (app followed by flatMap),
 	*  this should be the argument wrapped in Right.
 	*/
 	def contImpl[T,N[_]](c: Context, i: Instance with Singleton, convert: Convert, builder: TupleBuilder)(t: Either[c.Expr[T], c.Expr[i.M[T]]], inner: Transform[c.type,N])(
 		implicit tt: c.WeakTypeTag[T], nt: c.WeakTypeTag[N[T]], it: c.TypeTag[i.type]): c.Expr[i.M[N[T]]] =
 	{
 			import c.universe.{Apply=>ApplyTree,_}
-		
+
 		val util = ContextUtil[c.type](c)
 		val mTC: Type = util.extractTC(i, InstanceTCName)
 		val mttpe: Type = appliedType(mTC, nt.tpe :: Nil).normalize
@@ -91,19 +91,24 @@ object Instance
 			case Left(l) => (l.tree, nt.tpe.normalize)
 			case Right(r) => (r.tree, mttpe)
 		}
+		// the Symbol for the anonymous function passed to the appropriate Instance.map/flatMap/pure method
+		// this Symbol needs to be known up front so that it can be used as the owner of synthetic vals
+		val functionSym = util.functionSymbol(tree.pos)
 
 		val instanceSym = util.singleton(i)
-		// A Tree that references the statically accessible Instance that provides the actual implementations of map, flatMap, ... 
+		// A Tree that references the statically accessible Instance that provides the actual implementations of map, flatMap, ...
 		val instance = Ident(instanceSym)
 
 		val isWrapper: (String, Type, Tree) => Boolean = convert.asPredicate(c)
 
+		// Local definitions `defs` in the macro.  This is used to ensure references are to M instances defined outside of the macro call.
+		// Also `refCount` is the number of references, which is used to create the private, synthetic method containing the body
+		val defs = util.collectDefs(tree, isWrapper)
+		val checkQual: Tree => Unit = util.checkReferences(defs, isWrapper)
+
 		type In = Input[c.universe.type]
 		var inputs = List[In]()
 
-		// Local definitions in the macro.  This is used to ensure references are to M instances defined outside of the macro call.
-		val defs = util.collectDefs(tree, isWrapper)
-		val checkQual: Tree => Unit = util.checkReferences(defs, isWrapper)
 
 		// transforms the original tree into calls to the Instance functions pure, map, ...,
 		//  resulting in a value of type M[T]
@@ -118,7 +123,8 @@ object Instance
 		def pure(body: Tree): Tree =
 		{
 			val typeApplied = TypeApply(util.select(instance, PureName), TypeTree(treeType) :: Nil)
-			val p = ApplyTree(typeApplied, Function(Nil, body) :: Nil)
+			val f = util.createFunction(Nil, body, functionSym)
+			val p = ApplyTree(typeApplied, f :: Nil)
 			if(t.isLeft) p else flatten(p)
 		}
 		// m should have type M[M[T]]
@@ -133,9 +139,10 @@ object Instance
 		def single(body: Tree, input: In): Tree =
 		{
 			val variable = input.local
-			val param = ValDef(util.parameterModifiers, variable.name, variable.tpt, EmptyTree)
+			val param = treeCopy.ValDef(variable, util.parameterModifiers, variable.name, variable.tpt, EmptyTree)
 			val typeApplied = TypeApply(util.select(instance, MapName), variable.tpt :: TypeTree(treeType) :: Nil)
-			val mapped = ApplyTree(typeApplied, input.expr :: Function(param :: Nil, body) :: Nil)
+			val f = util.createFunction(param :: Nil, body, functionSym)
+			val mapped = ApplyTree(typeApplied, input.expr :: f :: Nil)
 			if(t.isLeft) mapped else flatten(mapped)
 		}
 
@@ -145,37 +152,37 @@ object Instance
 			val result = builder.make(c)(mTC, inputs)
 			val param = util.freshMethodParameter( appliedType(result.representationC, util.idTC :: Nil) )
 			val bindings = result.extract(param)
-			val f = Function(param :: Nil, Block(bindings, body))
+			val f = util.createFunction(param :: Nil, Block(bindings, body), functionSym)
 			val ttt = TypeTree(treeType)
 			val typedApp = TypeApply(util.select(instance, ApplyName), TypeTree(result.representationC) :: ttt :: Nil)
 			val app = ApplyTree(ApplyTree(typedApp, result.input :: f :: Nil), result.alistInstance :: Nil)
 			if(t.isLeft) app else flatten(app)
 		}
 
-		// called when transforming the tree to add an input
+		// Called when transforming the tree to add an input.
-		//  for `qual` of type M[A], and a selection qual.value,
+		//  For `qual` of type M[A], and a `selection` qual.value,
 		//  the call is addType(Type A, Tree qual)
-		// the result is a Tree representing a reference to
+		// The result is a Tree representing a reference to
-		//  the bound value of the input
+		//  the bound value of the input.
-		def addType(tpe: Type, qual: Tree): Tree =
+		def addType(tpe: Type, qual: Tree, selection: Tree): Tree =
 		{
 			qual.foreach(checkQual)
-			val vd = util.freshValDef(tpe, qual.symbol)
+			val vd = util.freshValDef(tpe, qual.symbol.pos, functionSym)
 			inputs ::= new Input(tpe, qual, vd)
-			util.refVal(vd, qual.pos)
+			util.refVal(selection, vd)
 		}
-		def sub(name: String, tpe: Type, qual: Tree): Converted[c.type] =
+		def sub(name: String, tpe: Type, qual: Tree, replace: Tree): Converted[c.type] =
 		{
 			val tag = c.WeakTypeTag[T](tpe)
 			convert[T](c)(name, qual)(tag) transform { tree =>
-				addType(tpe, tree)
+				addType(tpe, tree, replace)
 			}
 		}
 
 		// applies the transformation
-		val tx = util.transformWrappers(tree, (n,tpe,t) => sub(n,tpe,t))
+		val tx = util.transformWrappers(tree, (n,tpe,t,replace) => sub(n,tpe,t,replace))
 		// resetting attributes must be: a) local b) done here and not wider or else there are obscure errors
-		val tr = makeApp( c.resetLocalAttrs( inner(tx) ) ) 
+		val tr = makeApp( inner(tx) )
 		c.Expr[i.M[N[T]]](tr)
 	}
 

util/appmacro/src/main/scala/sbt/appmacro/KListBuilder.scala

@@ -33,8 +33,10 @@ object KListBuilder extends TupleBuilder
 		def bindKList(prev: ValDef, revBindings: List[ValDef], params: List[ValDef]): List[ValDef] =
 			params match
 			{
-				case ValDef(mods, name, tpt, _) :: xs =>
+				case (x @ ValDef(mods, name, tpt, _)) :: xs =>
-					val head = ValDef(mods, name, tpt, select(Ident(prev.name), "head"))
+					val rhs = select(Ident(prev.name), "head")
+					val head = treeCopy.ValDef(x, mods, name, tpt, rhs)
+					util.setSymbol(head, x.symbol)
 					val tail = localValDef(TypeTree(), select(Ident(prev.name), "tail"))
 					val base = head :: revBindings
 					bindKList(tail, if(xs.isEmpty) base else tail :: base, xs)
@@ -53,7 +55,7 @@ object KListBuilder extends TupleBuilder
 		val klist = makeKList(inputs.reverse, knil, knilType)
 
 		/** The input types combined in a KList type.  The main concern is tracking the heterogeneous types.
-		* The type constructor is tcVariable, so that it can be applied to [X] X or M later.  
+		* The type constructor is tcVariable, so that it can be applied to [X] X or M later.
 		* When applied to `M`, this type gives the type of the `input` KList. */
 		val klistType: Type = (inputs :\ knilType)( (in, klist) => kconsType(in.tpe, klist) )
 

util/appmacro/src/main/scala/sbt/appmacro/TupleNBuilder.scala

@@ -42,9 +42,11 @@ object TupleNBuilder extends TupleBuilder
 		def bindTuple(param: ValDef, revBindings: List[ValDef], params: List[ValDef], i: Int): List[ValDef] =
 			params match
 			{
-				case ValDef(mods, name, tpt, _) :: xs =>
+				case (x @ ValDef(mods, name, tpt, _)) :: xs =>
-					val x = ValDef(mods, name, tpt, select(Ident(param.name), "_" + i.toString))
+					val rhs = select(Ident(param.name), "_" + i.toString)
-					bindTuple(param, x :: revBindings, xs, i+1)
+					val newVal = treeCopy.ValDef(x, mods, name, tpt, rhs)
+					util.setSymbol(newVal, x.symbol)
+					bindTuple(param, newVal :: revBindings, xs, i+1)
 				case Nil => revBindings.reverse
 			}
 	}