luke
/
sbt
mirror of https://github.com/sbt/sbt.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

initial creation

Havoc Pennington 2013-10-09 07:38:01 -07:00
parent ee669879b3
commit b13bf28a28
1 changed files with 49 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
Client-server-discovery-lifecycle.md Normal file

@ -0,0 +1,49 @@
How should the [server](https://github.com/sbt/sbt/wiki/Client-server-split) be discovered, started, and shut down?
## Some Objectives
* Clients should auto-start the server as needed
* If several clients start at once (not unlikely e.g. on login), we want to end up with only one server, at least "almost always in practice," even if we aren't theoretically race-free
* If a server dies, we want to auto-start a new one
* We don't want people to need to manually kill orphaned (clientless) servers
* Server is scoped "per-build" (single instance for each root project directory)
* Ctrl+C or OS reboot or hotspot crash should not result in people having to do manual lockfile cleanup
* Potentially controversial, but based on experience: operating system lockfile functionality tends to be platform-specific and flaky (e.g. not working on NFS/AFS, not working between threads, stuff like that), so avoid if possible. This is likely a main question (whether to use FileLock).
* There are more robust platform-specific solutions available, such as D-Bus on Linux, COM or window-based mechanisms on Windows, and launchd on Mac. These are likely a total pain in the ass to use from the JVM, though. Is there any sane-from-JVM way to do a reliable named lock?
## Proposal 1: active.properties
This proposal is intended to "self-heal" and work in practice without asking users to kill processes or delete lockfiles - but nobody will call it elegant. The general idea is that a server suicides if the file pointing to that server is overwritten or deleted.
* create a directory myproject/.build-server (let's call it SERVERDIR)
* SERVERDIR/active.properties contains properties `url`, `pid`, and `version`
* the `url` is expected to be `protocol://127.0.0.1:PPPP` always for now (don't listen on 0.0.0.0 since that will include external interfaces). PPPP should be an OS-assigned random port. Protocol is likely http. This url would be a base URL and there may be conventional paths within it.
* the `pid` for now is just for use in debugging or manual cleanup by a human
* the `version` is build server version in case we need to do something different based on this someday
* when a client needs a server, it first loads active.properties and tries to ping the `url` for aliveness
* if the `url` is alive, client uses it
* otherwise, client forks a new build server and starts checking for active.properties to appear at short intervals
* client gives up after some timeout (long enough for build server to reliably start even on lame computers)
* whenever a client loses its server connection, it starts over discovering/starting a new server.
* the "ping" for server liveness should verify that the pinged server goes with the project directory we care about (because the server could be for a different project that just happened to recycle the same url). So the ping could for example contain the canonical path to the active.properties file and fail if the server pinged is for a different path.
* when the build server starts up, it needs to proceed in this order:
1. ping any existing active.properties url and exit if the other server is alive
2. listen on a port (so now we know our url and we are pingable)
4. take over or fail to take over active.properties
5. actually start up the sbt engine if it takes over
6. exit without doing anything if it fails to take over
* to take over active.properties, the build server does the following:
1. we should already be pingable
2. write out active.properties atomically (by creating a temp file and renaming over on unix, and doing whatever one can do on Windows)
3. exit if active.properties ever changes so it no longer refers to us. (use file watches or just check at intervals.)
* another server starting up may have pinged us and failed, then we both started listening, then we both wrote active.properties.
* user may have moved the project directory.
* user may have deleted active.properties by hand.
* the build server should wait some reasonable timeframe for the first client to connect, and exit if no client ever connects. It should also exit a (possibly very short) time after the last client disconnects.
* on clean exit, don't delete active.properties since we're probably just as likely to delete one that doesn't refer to us and cause trouble. Just rely on future pings failing - stale active.properties should be automatically handled.
* "herd of server starts" problem: when a server exits and has 4 clients, they might all immediately try to restart a new server, which should work out but would launch a lot of pointless JVMs in the meantime. To avoid this, they could each wait a short random interval before restarting, OR they could each be assigned a "restart delay" by the server, OR something else.
* note that we are not expecting pings to fail due to network problems, since we are on a local machine.
## Proposal 2: Some other idea here
* is there some way we can get an actual, reliable lock on a project directory, cross-platform?