diff --git a/README.md b/README.md new file mode 100644 index 0000000..c98d081 --- /dev/null +++ b/README.md @@ -0,0 +1,75 @@ + +## Introduction +This project bundles a set of scripts to simplify the deployment of docker containers. + +The basic idea is adapted from the excellent [Ubuntu base image](https://github.com/phusion/baseimage-docker): + +- install software using bash scripts +- use a tailored init script to ensure correct startup and shutdown behavior +- use [runit](http://smarden.org/runit/) to supervise services +- provide a preconfigured SSH server +- provide a consistent way of managing environment variables + +## Installation +Make sure Docker is [installed](https://docs.docker.com/installation/) correctly, in most cases, the following command should suffice: +``` +curl http://get.docker.io | sudo sh +``` + +Then change /etc/default/docker to read: +``` +DOCKER_OPTS="-e lxc -r=false" +``` + +And make sure to restart the docker daemon. This enables LXC related tricks such as +```docker attach``` and to allows to modify networking configuration using ```--lxc-conf``` in ```docker run```. It also disables the automatic restart of previously running containers. + +## Utility scripts + +### In $DOCKER_HOME/bin + +#### app +#### attach +#### build +#### clean +#### killall +#### make +#### purge +#### run +#### ssh +#### stopall + +### In $DOCKER_HOME/images/<image>/ +#### app + +#### run + +#### ssh + + + +Images are build using Makefiles in order to provide reusable pieces of functionality. + +## Build files + +### Boot +These scripts are executed by the init script before runit starts any services. Numeric prefixes are used to enforce a specific order of execution. These scripts are used to prepare the container with live parameters from the host. It is more convenient to make service specific modifications in the respective runit scripts than to create separate boot scripts. + +### Script +These scripts are executed during the build and install the software. Numeric prefixes are used to enforce a specific order of execution. The following convention is recommended: + +- 00_ for essential modifications to the base image without which other software might fail to install. +- 01_ for essential modifications regarding the functioning of the base image. +- 02_ for system wide modifications or the installation of very general system components. +- 03_ for the installation of image specific software. +- 99_ to clean the image from unnecessary junk. + +### Runit +These scripts are executed by runit to start services. Runit requires services to run in the foreground and correct parameters must be passed to prevent the service from daemonising. These scripts are also used to initialise files and directories on host volumes. + +### Make +The Makefiles link (symbolic link) the required files (installation script, boot script and runit script) for each separate component into the build directory. These files are quite general since the aforementioned scripts for each separate component are typically named after the build target. Dependencies on other component are defined here. + +## Current limitations +- This approach does not benefit from the caching mechanisms used by Docker. The Dockerfile is generated at build time and installation scripts are added dynamically. +- There is no mechanism to keep track of software installed during the build. It would be desirable to have a *developer mode* that keeps useful software (e.g., vim, curl, wget) after the build or a *production mode* that removes this software to obtain as small an image as possible. \ No newline at end of file diff --git a/bin/app b/bin/app new file mode 100644 index 0000000..43bb5a9 --- /dev/null +++ b/bin/app @@ -0,0 +1,4 @@ +DIR=`pwd` +cd /opt/docker/images/$1/ +bin/app "$2" +cd $DIR \ No newline at end of file diff --git a/bin/attach b/bin/attach new file mode 100644 index 0000000..ee4900b --- /dev/null +++ b/bin/attach @@ -0,0 +1 @@ +lxc-attach -n `docker ps --no-trunc | grep $1: | cut -d' ' -f1` diff --git a/bin/build b/bin/build new file mode 100644 index 0000000..8c21750 --- /dev/null +++ b/bin/build @@ -0,0 +1,4 @@ +DIR=`pwd` +cd /opt/docker/images/$1/ +make build +cd $DIR diff --git a/bin/clean b/bin/clean new file mode 100644 index 0000000..5cc8391 --- /dev/null +++ b/bin/clean @@ -0,0 +1,4 @@ +IDS=`comm -3 <(docker ps -a -q | sort) <(docker ps -q | sort)` +if [[ ! -z "$IDS" ]]; then + docker rm $IDS +fi \ No newline at end of file diff --git a/bin/killall b/bin/killall new file mode 100644 index 0000000..ba2299a --- /dev/null +++ b/bin/killall @@ -0,0 +1 @@ +docker kill `docker ps -q` diff --git a/bin/make b/bin/make new file mode 100644 index 0000000..5d1587b --- /dev/null +++ b/bin/make @@ -0,0 +1,4 @@ +DIR=`pwd` +cd /opt/docker/images/$1/ +make $2 +cd $DIR diff --git a/bin/purge b/bin/purge new file mode 100644 index 0000000..d976bdf --- /dev/null +++ b/bin/purge @@ -0,0 +1 @@ +docker images | grep '' | awk '{print $3}' | xargs docker rmi \ No newline at end of file diff --git a/bin/run b/bin/run new file mode 100644 index 0000000..0f042d7 --- /dev/null +++ b/bin/run @@ -0,0 +1,31 @@ +#!/bin/bash +# Arguments: +# $1 = redirected ? -t : +# $2 = redirected ? : +# $3 = redirected ? : +# When redirected: +# $4 = +# $5 = +if [ -n "$1" ] && [ $1 == "-t" ]; then + HOST_ADDR=`ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{print $1}'` + PARAMS="--rm -e HOST_ADDR=$HOST_ADDR $3 --name $2" + if [ -n "$4" ] && [ $4 == "-i" ]; then + # Run without services, provide shell + docker run $PARAMS -i -t $2 bash + elif [ -n "$4" ] && [ $4 == "-x" ]; then + # Run with services, provide shell + docker run $PARAMS -i -t $2 /opt/init -- bash -l + elif [ -n "$4" ] && [ $4 == "-c" ] && [ -n "$5" ]; then + # Run with services, provide shell + docker run $PARAMS -i -t $2 /opt/init -- $5 + elif [ -n "$4" ] && [ $4 == "-w" ]; then + # Run without services + docker run $PARAMS $2 + else + # Run with services + docker run $PARAMS $2 /opt/init + fi +else + /opt/docker/bin/clean # > /dev/null 2>&1 + /opt/docker/images/$1/bin/run $2 $3 +fi diff --git a/bin/ssh b/bin/ssh new file mode 100644 index 0000000..e3552cf --- /dev/null +++ b/bin/ssh @@ -0,0 +1,4 @@ +DIR=`pwd` +cd /opt/docker/images/$1/ +bin/ssh "$2" "$3" +cd $DIR \ No newline at end of file diff --git a/bin/stopall b/bin/stopall new file mode 100644 index 0000000..6be390b --- /dev/null +++ b/bin/stopall @@ -0,0 +1 @@ +docker stop `docker ps -q` diff --git a/build/Dockerfile b/build/Dockerfile new file mode 100644 index 0000000..7cbfb1b --- /dev/null +++ b/build/Dockerfile @@ -0,0 +1,5 @@ +FROM ubuntu:precise + +ADD build /build + +RUN for s in /build/scripts/*.sh; do [ -x $s ] && $s || : ; done \ No newline at end of file diff --git a/build/Makefile b/build/Makefile new file mode 100644 index 0000000..e506485 --- /dev/null +++ b/build/Makefile @@ -0,0 +1,49 @@ +include $(wildcard /opt/docker/build/make/*.mk) +BASE = /opt/docker/build + +all: build bin service clean + +build: + cp $(BASE)/Dockerfile . + docker build -t $(NAME):$(VERSION) --rm . + +directory: + @mkdir -p build/boot build/runit build/scripts + +service: + @if test "${DEPENDS}" != ""; then \ + echo start on started docker-$(subst $(eval) $(eval), and started docker-,$(DEPENDS)) >> init.conf ;\ + echo stop on stopped docker-$(subst $(eval) $(eval), or stopped docker-,$(DEPENDS)) >> init.conf ;\ + else \ + echo start on started docker >> init.conf ;\ + echo stop on stopped docker >> init.conf ;\ + fi + @echo exec /opt/docker/bin/run $(NAME) >> init.conf + @echo respawn >> init.conf + @mv init.conf /etc/init/docker-$(NAME).conf + +bin: + @mkdir -p bin + @ln -sf $(BASE)/bin/run bin + @ln -sf $(BASE)/bin/ssh bin + @ln -sf $(BASE)/bin/app bin + @sed -i "1iARGS=\"$(RUN)\"" bin/run + @sed -i "1iARGS=\"$(SSH)\"" bin/ssh + +clean: + @rm -f Dockerfile + @rm -rf build + @rm -rf id_rsa + +tag_latest: + @docker tag $(NAME):$(VERSION) $(NAME):latest + +ssh: + @ID=$$(docker ps | grep "$(NAME):$(VERSION)" | awk '{ print $$1 }') && \ + if test "$$ID" = ""; then echo "Container is not running."; exit 1; fi && \ + if ! test -s id_rsa; then \ + docker cp $$ID:/opt/id_rsa . ;\ + fi && \ + IP=$$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $$ID) && \ + echo "SSHing into $$IP" && \ + ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i id_rsa root@$$IP ${CMD} \ No newline at end of file diff --git a/build/bin/app b/build/bin/app new file mode 100644 index 0000000..d88d4fd --- /dev/null +++ b/build/bin/app @@ -0,0 +1,18 @@ +DIR=`dirname $0` +DIR=`readlink -e $DIR` +BASE=`basename $DIR` +IFS='/' read -ra ADDR <<< "$DIR" +CONTAINER=${ADDR[-2]} +while : ; do + ID=`docker ps --no-trunc | grep $CONTAINER: | cut -d' ' -f1` + if [ -n "$ID" ]; then + break + fi + if [ ! -n "$STARTED" ]; then + echo "Starting container." + /opt/docker/bin/run $CONTAINER & + STARTED=yes + sleep 10 + fi +done +/opt/docker/bin/ssh $CONTAINER $1 "source .profile; /opt/$CONTAINER" diff --git a/build/bin/run b/build/bin/run new file mode 100644 index 0000000..18a8ad4 --- /dev/null +++ b/build/bin/run @@ -0,0 +1,6 @@ +DIR=`dirname $0` +DIR=`readlink -e $DIR` +BASE=`basename $DIR` +IFS='/' read -ra ADDR <<< "$DIR" +CONTAINER=${ADDR[-2]} +/opt/docker/bin/run -t $CONTAINER "$ARGS" $1 "$2" diff --git a/build/bin/ssh b/build/bin/ssh new file mode 100644 index 0000000..71845a8 --- /dev/null +++ b/build/bin/ssh @@ -0,0 +1,23 @@ +DIR=`dirname $0` +DIR=`readlink -e $DIR` +BASE=`basename $DIR` +IFS='/' read -ra ADDR <<< "$DIR" +CONTAINER=${ADDR[-2]} +ID=`docker ps --no-trunc | grep $CONTAINER: | cut -d' ' -f1` +if [ -n "$ID" ]; then + IP=`docker inspect --format '{{ .NetworkSettings.IPAddress }}' $ID` + COMMAND="ssh -X -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i id_rsa $ARGS root@$IP" + if [ ! -e id_rsa ]; then + docker cp $ID:/opt/id_rsa . + fi + if [ ! -z "$1" ] && [ "$1" == "-l" ]; then + DISPLAY=:0 + chmod 0666 id_rsa + su user -c "$COMMAND \"$2\"" + else + chmod 0600 id_rsa + eval "$COMMAND \"$1\"" + fi +else + echo "Container is not running." +fi \ No newline at end of file diff --git a/build/boot/00_hosts.sh b/build/boot/00_hosts.sh new file mode 100644 index 0000000..cf95dc4 --- /dev/null +++ b/build/boot/00_hosts.sh @@ -0,0 +1,4 @@ +#!/bin/bash +set -e +echo "127.0.0.1 localhost" >> /tmp/hosts +echo "$HOST_ADDR host" >> /tmp/hosts diff --git a/build/boot/01_hamachi.sh b/build/boot/01_hamachi.sh new file mode 100644 index 0000000..41bf6be --- /dev/null +++ b/build/boot/01_hamachi.sh @@ -0,0 +1,5 @@ +#!/bin/bash +set -e +cd /opt +dpkg -i hamachi.deb +kill -9 `pgrep hamachid` \ No newline at end of file diff --git a/build/config b/build/config new file mode 100644 index 0000000..e386f1b --- /dev/null +++ b/build/config @@ -0,0 +1,3 @@ +nnexport LC_ALL=C +export DEBIAN_FRONTEND=noninteractive +minimal_apt_get_install='apt-get install -y --no-install-recommends' diff --git a/build/init b/build/init new file mode 100644 index 0000000..57e1698 --- /dev/null +++ b/build/init @@ -0,0 +1,349 @@ +#!/usr/bin/python2 -u +# Copyright (c) 2014 Rik Veenboer +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# This file incorporates work covered by the following copyright and permission notice: +# Copyright (c) 2013-2014 Phusion +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. +import os, os.path, sys, stat, signal, errno, argparse, time, json, re, posixfile + +KILL_PROCESS_TIMEOUT = 5 +KILL_ALL_PROCESSES_TIMEOUT = 5 + +LOG_LEVEL_ERROR = 1 +LOG_LEVEL_WARN = 1 +LOG_LEVEL_INFO = 2 +LOG_LEVEL_DEBUG = 3 + +log_level = None + +class AlarmException(Exception): + pass + +def error(message): + if log_level >= LOG_LEVEL_ERROR: + sys.stderr.write("*** %s\n" % message) + +def warn(message): + if log_level >= LOG_LEVEL_WARN: + print("*** %s" % message) + +def info(message): + if log_level >= LOG_LEVEL_INFO: + print("*** %s" % message) + +def debug(message): + if log_level >= LOG_LEVEL_DEBUG: + print("*** %s" % message) + +def ignore_signals_and_raise_keyboard_interrupt(signame): + signal.signal(signal.SIGTERM, signal.SIG_IGN) + signal.signal(signal.SIGINT, signal.SIG_IGN) + raise KeyboardInterrupt(signame) + +def raise_alarm_exception(): + raise AlarmException('Alarm') + +def listdir(path): + try: + result = os.stat(path) + except OSError: + return [] + if stat.S_ISDIR(result.st_mode): + return sorted(os.listdir(path)) + else: + return [] + +def is_exe(path): + try: + return os.path.isfile(path) and os.access(path, os.X_OK) + except OSError: + return False + +def import_envvars(clear_existing_environment = True): + new_env = {} + for envfile in listdir("/etc/container_environment"): + name = os.path.basename(envfile) + with open("/etc/container_environment/" + envfile, "r") as f: + value = f.read() + new_env[name] = value + if clear_existing_environment: + os.environ.clear() + for name, value in new_env.items(): + os.environ[name] = value + +def export_envvars(to_dir = True): + shell_dump = "" + for name, value in os.environ.items(): + if to_dir: + with open("/etc/container_environment/" + name, "w") as f: + f.write(value) + shell_dump += "export " + shquote(name) + "=" + shquote(value) + "\n" + with open("/etc/container_environment.sh", "w") as f: + f.write(shell_dump) + with open("/etc/container_environment.json", "w") as f: + f.write(json.dumps(dict(os.environ))) + +_find_unsafe = re.compile(r'[^\w@%+=:,./-]').search + +def shquote(s): + """Return a shell-escaped version of the string *s*.""" + if not s: + return "''" + if _find_unsafe(s) is None: + return s + + # use single quotes, and put single quotes into double quotes + # the string $'b is then quoted as '$'"'"'b' + return "'" + s.replace("'", "'\"'\"'") + "'" + +def waitpid_reap_other_children(pid): + done = False + status = None + try: + this_pid, status = os.waitpid(pid, os.WNOHANG) + except OSError as e: + if e.errno == errno.ECHILD or e.errno == errno.ESRCH: + return None + else: + raise + while not done: + this_pid, status = os.waitpid(-1, 0) + done = this_pid == pid + return status + +def stop_child_process(name, pid, signo = signal.SIGTERM, time_limit = KILL_PROCESS_TIMEOUT): + info("Shutting down %s (PID %d)..." % (name, pid)) + try: + os.kill(pid, signo) + except OSError: + pass + signal.alarm(time_limit) + try: + try: + waitpid_reap_other_children(pid) + except OSError: + pass + except AlarmException: + warn("%s (PID %d) did not shut down in time. Forcing it to exit." % (name, pid)) + try: + os.kill(pid, signal.SIGKILL) + except OSError: + pass + try: + waitpid_reap_other_children(pid) + except OSError: + pass + finally: + signal.alarm(0) + +def run_command_killable(*argv): + filename = argv[0] + status = None + pid = os.spawnvp(os.P_NOWAIT, filename, argv) + try: + status = waitpid_reap_other_children(pid) + except BaseException as s: + warn("An error occurred. Aborting.") + stop_child_process(filename, pid) + raise + if status != 0: + if status is None: + error("%s exited with unknown status\n" % filename) + else: + error("%s failed with status %d\n" % (filename, os.WEXITSTATUS(status))) + sys.exit(1) + +def run_command_killable_and_import_envvars(*argv): + run_command_killable(*argv) + import_envvars() + export_envvars(False) + +def kill_all_processes(time_limit): + info("Killing all processes...") + try: + os.kill(-1, signal.SIGTERM) + except OSError: + pass + signal.alarm(time_limit) + try: + # Wait until no more child processes exist. + done = False + while not done: + try: + os.waitpid(-1, 0) + except OSError as e: + if e.errno == errno.ECHILD: + done = True + else: + raise + except AlarmException: + warn("Not all processes have exited in time. Forcing them to exit.") + try: + os.kill(-1, signal.SIGKILL) + except OSError: + pass + finally: + signal.alarm(0) + +def run_startup_files(): + # Run /opt/init.d/* + for name in listdir("/opt/init.d"): + filename = "/opt/init.d/" + name + if is_exe(filename): + info("Running %s..." % filename) + run_command_killable_and_import_envvars(filename) + + # Run /etc/rc.local. + if is_exe("/etc/rc.local"): + info("Running /etc/rc.local...") + run_command_killable_and_import_envvars("/etc/rc.local") + +def start_runit(): + info("Booting runit daemon...") + pid = os.spawnl(os.P_NOWAIT, "/usr/bin/runsvdir", "/usr/bin/runsvdir", + "-P", "/etc/service", "log: %s" % ('.' * 395)) + info("Runit started as PID %d" % pid) + return pid + +def wait_for_runit_or_interrupt(pid): + try: + status = waitpid_reap_other_children(pid) + return (True, status) + except KeyboardInterrupt: + return (False, None) + +def shutdown_runit_services(): + debug("Begin shutting down runit services...") + os.system("/usr/bin/sv down /etc/service/*") + +def wait_for_runit_services(): + debug("Waiting for runit services to exit...") + done = False + while not done: + done = os.system("/usr/bin/sv status /etc/service/* | grep -q '^run:'") != 0 + if not done: + time.sleep(0.1) + +def install_insecure_key(): + info("Installing insecure SSH key for user root") + run_command_killable("/usr/sbin/enable_insecure_key") + +def main(args): + import_envvars(False) + export_envvars() + + if args.enable_insecure_key: + install_insecure_key() + + if not args.skip_startup_files: + run_startup_files() + + runit_exited = False + exit_code = None + + if not args.skip_runit: + runit_pid = start_runit() + try: + exit_status = None + if len(args.main_command) == 0: + runit_exited, exit_code = wait_for_runit_or_interrupt(runit_pid) + if runit_exited: + if exit_code is None: + info("Runit exited with unknown status") + exit_status = 1 + else: + exit_status = os.WEXITSTATUS(exit_code) + info("Runit exited with status %d" % exit_status) + else: + info("Running %s..." % " ".join(args.main_command)) + pid = os.spawnvp(os.P_NOWAIT, args.main_command[0], args.main_command) + try: + exit_code = waitpid_reap_other_children(pid) + if exit_code is None: + info("%s exited with unknown status." % args.main_command[0]) + exit_status = 1 + else: + exit_status = os.WEXITSTATUS(exit_code) + info("%s exited with status %d." % (args.main_command[0], exit_status)) + except KeyboardInterrupt: + stop_child_process(args.main_command[0], pid) + except BaseException as s: + warn("An error occurred. Aborting.") + stop_child_process(args.main_command[0], pid) + raise + sys.exit(exit_status) + finally: + if not args.skip_runit: + shutdown_runit_services() + if not runit_exited: + stop_child_process("runit daemon", runit_pid) + wait_for_runit_services() + +# Parse options. +parser = argparse.ArgumentParser(description = 'Initialize the system.') +parser.add_argument('main_command', metavar = 'MAIN_COMMAND', type = str, nargs = '*', + help = 'The main command to run. (default: runit)') +parser.add_argument('--enable-insecure-key', dest = 'enable_insecure_key', + action = 'store_const', const = True, default = False, + help = 'Install the insecure SSH key') +parser.add_argument('--skip-startup-files', dest = 'skip_startup_files', + action = 'store_const', const = True, default = False, + help = 'Skip running /opt/init.d/* and /etc/rc.local') +parser.add_argument('--skip-runit', dest = 'skip_runit', + action = 'store_const', const = True, default = False, + help = 'Do not run runit services') +parser.add_argument('--no-kill-all-on-exit', dest = 'kill_all_on_exit', + action = 'store_const', const = False, default = True, + help = 'Don\'t kill all processes on the system upon exiting') +parser.add_argument('--quiet', dest = 'log_level', + action = 'store_const', const = LOG_LEVEL_WARN, default = LOG_LEVEL_INFO, + help = 'Only print warnings and errors') +args = parser.parse_args() +log_level = args.log_level + +if args.skip_runit and len(args.main_command) == 0: + error("When --skip-runit is given, you must also pass a main command.") + sys.exit(1) + +# Run main function. +signal.signal(signal.SIGTERM, lambda signum, frame: ignore_signals_and_raise_keyboard_interrupt('SIGTERM')) +signal.signal(signal.SIGINT, lambda signum, frame: ignore_signals_and_raise_keyboard_interrupt('SIGINT')) +signal.signal(signal.SIGALRM, lambda signum, frame: raise_alarm_exception()) +try: + main(args) +except KeyboardInterrupt: + warn("Init system aborted.") + exit(2) +finally: + if args.kill_all_on_exit: + kill_all_processes(KILL_ALL_PROCESSES_TIMEOUT) diff --git a/build/make/base.mk b/build/make/base.mk new file mode 100644 index 0000000..607880d --- /dev/null +++ b/build/make/base.mk @@ -0,0 +1,10 @@ +base: directory cron syslog sshd hosts + ln -f $(BASE)/config build + ln -f $(BASE)/init build + ln -f $(BASE)/scripts/*_prepare.sh build/scripts + ln -f $(BASE)/scripts/*_boot.sh build/scripts + ln -f $(BASE)/scripts/*_init.sh build/scripts + ln -f $(BASE)/scripts/*_runit.sh build/scripts + ln -f $(BASE)/scripts/*_logrotate.sh build/scripts + ln -f $(BASE)/scripts/*_utilities.sh build/scripts + ln -f $(BASE)/scripts/*_cleanup.sh build/scripts \ No newline at end of file diff --git a/build/make/btsync.mk b/build/make/btsync.mk new file mode 100644 index 0000000..6f9468f --- /dev/null +++ b/build/make/btsync.mk @@ -0,0 +1,3 @@ +btsync: + cp $(BASE)/scripts/*_$@.sh build/scripts + cp $(BASE)/runit/$@ build/runit \ No newline at end of file diff --git a/build/make/chrome.mk b/build/make/chrome.mk new file mode 100644 index 0000000..0615c90 --- /dev/null +++ b/build/make/chrome.mk @@ -0,0 +1,2 @@ +chrome: + ln -f $(BASE)/scripts/*_$@.sh build/scripts \ No newline at end of file diff --git a/build/make/cron.mk b/build/make/cron.mk new file mode 100644 index 0000000..66c367b --- /dev/null +++ b/build/make/cron.mk @@ -0,0 +1,3 @@ +cron: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/runit/$@ build/runit \ No newline at end of file diff --git a/build/make/firefox.mk b/build/make/firefox.mk new file mode 100644 index 0000000..4d61e45 --- /dev/null +++ b/build/make/firefox.mk @@ -0,0 +1,2 @@ +firefox: user + ln -f $(BASE)/scripts/*_$@.sh build/scripts \ No newline at end of file diff --git a/build/make/hamachi.mk b/build/make/hamachi.mk new file mode 100644 index 0000000..182334c --- /dev/null +++ b/build/make/hamachi.mk @@ -0,0 +1,4 @@ +hamachi: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/runit/$@ build/runit + ln -f $(BASE)/boot/*_$@.sh build/boot \ No newline at end of file diff --git a/build/make/hosts.mk b/build/make/hosts.mk new file mode 100644 index 0000000..7d1ac1c --- /dev/null +++ b/build/make/hosts.mk @@ -0,0 +1,3 @@ +hosts: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/boot/*_$@.sh build/boot \ No newline at end of file diff --git a/build/make/pulseaudio.mk b/build/make/pulseaudio.mk new file mode 100644 index 0000000..5535553 --- /dev/null +++ b/build/make/pulseaudio.mk @@ -0,0 +1,2 @@ +pulseaudio: + ln -f $(BASE)/scripts/*_$@.sh build/scripts \ No newline at end of file diff --git a/build/make/redis.mk b/build/make/redis.mk new file mode 100644 index 0000000..3330d0a --- /dev/null +++ b/build/make/redis.mk @@ -0,0 +1,3 @@ +redis: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/runit/$@ build/runit \ No newline at end of file diff --git a/build/make/sshd.mk b/build/make/sshd.mk new file mode 100644 index 0000000..3478c37 --- /dev/null +++ b/build/make/sshd.mk @@ -0,0 +1,3 @@ +sshd: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/runit/$@ build/runit \ No newline at end of file diff --git a/build/make/syslog.mk b/build/make/syslog.mk new file mode 100644 index 0000000..9d52c48 --- /dev/null +++ b/build/make/syslog.mk @@ -0,0 +1,3 @@ +syslog: + ln -f $(BASE)/scripts/*_$@.sh build/scripts + ln -f $(BASE)/runit/$@ build/runit \ No newline at end of file diff --git a/build/make/user.mk b/build/make/user.mk new file mode 100644 index 0000000..d98b7a2 --- /dev/null +++ b/build/make/user.mk @@ -0,0 +1,2 @@ +user: + ln -f $(BASE)/scripts/*_$@.sh build/scripts \ No newline at end of file diff --git a/build/runit/btsync b/build/runit/btsync new file mode 100644 index 0000000..72f50fa --- /dev/null +++ b/build/runit/btsync @@ -0,0 +1,8 @@ +#!/bin/sh +mkdir -p $BTSYNC_DATA +if [ ! -e $BTSYNC_CONFIG ]; then + mkdir -p `dirname $BTSYNC_CONFIG` + /opt/btsync --dump-sample-config > $BTSYNC_CONFIG + sed -i "s,\(\"storage_path\"\s*:\).*,\1 \"$BTSYNC_DATA\"\,," $BTSYNC_CONFIG +fi +/opt/btsync --nodaemon --config $BTSYNC_CONFIG diff --git a/build/runit/cron b/build/runit/cron new file mode 100644 index 0000000..3638fb8 --- /dev/null +++ b/build/runit/cron @@ -0,0 +1,2 @@ +#!/bin/sh +/usr/sbin/cron -f diff --git a/build/runit/hamachi b/build/runit/hamachi new file mode 100644 index 0000000..5f05d6e --- /dev/null +++ b/build/runit/hamachi @@ -0,0 +1,4 @@ +#!/bin/sh +/opt/logmein-hamachi/bin/hamachid -c $HAMACHI_DATA +strace -qqe '' -p `pgrep hamachid` +/host/bin/proxy \ No newline at end of file diff --git a/build/runit/redis b/build/runit/redis new file mode 100644 index 0000000..b51f46e --- /dev/null +++ b/build/runit/redis @@ -0,0 +1,4 @@ +#!/bin/sh +mkdir -p `dirname $REDIS_LOG` $REDIS_DATA +sysctl vm.overcommit_memory=1 +/usr/bin/redis-server $REDIS_CONFIG diff --git a/build/runit/sshd b/build/runit/sshd new file mode 100644 index 0000000..ee85db5 --- /dev/null +++ b/build/runit/sshd @@ -0,0 +1,2 @@ +#!/bin/sh +/usr/sbin/sshd -D diff --git a/build/runit/syslog b/build/runit/syslog new file mode 100644 index 0000000..c850a34 --- /dev/null +++ b/build/runit/syslog @@ -0,0 +1,24 @@ +#!/bin/sh +set -e + +SYSLOGNG_OPTS="" + +[ -r /etc/default/syslog-ng ] && . /etc/default/syslog-ng + +case "x$CONSOLE_LOG_LEVEL" in + x[1-8]) + dmesg -n $CONSOLE_LOG_LEVEL + ;; + x) + ;; + *) + echo "CONSOLE_LOG_LEVEL is of unaccepted value." + ;; +esac + +if [ ! -e /dev/xconsole ] +then + mknod -m 640 /dev/xconsole p +fi + +syslog-ng -F -p /var/run/syslog-ng.pid $SYSLOGNG_OPTS diff --git a/build/scripts/00_prepare.sh b/build/scripts/00_prepare.sh new file mode 100644 index 0000000..206cf00 --- /dev/null +++ b/build/scripts/00_prepare.sh @@ -0,0 +1,39 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Temporarily disable dpkg fsync to make building faster. +echo force-unsafe-io > /etc/dpkg/dpkg.cfg.d/02apt-speedup + +## Prevent initramfs updates from trying to run grub and lilo. +## https://journal.paul.querna.org/articles/2013/10/15/docker-ubuntu-on-rackspace/ +## http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=594189 +export INITRD=no +mkdir -p /etc/container_environment +echo -n no > /etc/container_environment/INITRD + +## Update package list +apt-get update + +## Fix some issues with APT packages +## See https://github.com/dotcloud/docker/issues/1024 +dpkg-divert --local --rename --add /sbin/initctl +ln -sf /bin/true /sbin/initctl + +## Replace the 'ischroot' tool to make it always return true +## Prevent initscripts updates from breaking /dev/shm +## https://journal.paul.querna.org/articles/2013/10/15/docker-ubuntu-on-rackspace/ +## https://bugs.launchpad.net/launchpad/+bug/974584 +dpkg-divert --local --rename --add /usr/bin/ischroot +ln -sf /bin/true /usr/bin/ischroot + +## Install HTTPS support for APT +$minimal_apt_get_install apt-transport-https + +## Upgrade all packages +apt-get dist-upgrade -y --no-install-recommends + +## Fix locale +$minimal_apt_get_install language-pack-en +locale-gen en_US \ No newline at end of file diff --git a/build/scripts/01_boot.sh b/build/scripts/01_boot.sh new file mode 100644 index 0000000..0ad29df --- /dev/null +++ b/build/scripts/01_boot.sh @@ -0,0 +1,8 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Boot entries +mkdir /opt/init.d/ +mv /build/boot/*.sh /opt/init.d/ diff --git a/build/scripts/01_cron.sh b/build/scripts/01_cron.sh new file mode 100644 index 0000000..0efba45 --- /dev/null +++ b/build/scripts/01_cron.sh @@ -0,0 +1,13 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install cron daemon +$minimal_apt_get_install cron +mkdir -p /etc/service/cron +mv /build/runit/cron /etc/service/cron/run + +## Remove useless cron entries +# Checks for lost+found and scans for mtab +rm -f /etc/cron.daily/standard diff --git a/build/scripts/01_init.sh b/build/scripts/01_init.sh new file mode 100644 index 0000000..278c35a --- /dev/null +++ b/build/scripts/01_init.sh @@ -0,0 +1,12 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install init process +mv /build/init /opt/ +mkdir -p /etc/container_environment +touch /etc/container_environment.sh +touch /etc/container_environment.json +chmod 700 /etc/container_environment +chmod 600 /etc/container_environment.sh /etc/container_environment.json diff --git a/build/scripts/01_logrotate.sh b/build/scripts/01_logrotate.sh new file mode 100644 index 0000000..173ffd6 --- /dev/null +++ b/build/scripts/01_logrotate.sh @@ -0,0 +1,7 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install logrotate +$minimal_apt_get_install logrotate \ No newline at end of file diff --git a/build/scripts/01_runit.sh b/build/scripts/01_runit.sh new file mode 100644 index 0000000..8919aa2 --- /dev/null +++ b/build/scripts/01_runit.sh @@ -0,0 +1,7 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install runit +$minimal_apt_get_install runit diff --git a/build/scripts/01_sshd.sh b/build/scripts/01_sshd.sh new file mode 100644 index 0000000..0e8edba --- /dev/null +++ b/build/scripts/01_sshd.sh @@ -0,0 +1,28 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install the SSH server +$minimal_apt_get_install openssh-server +mkdir /var/run/sshd +mkdir -p /etc/service/sshd +cp /build/runit/sshd /etc/service/sshd/run + +## Install root key +cd /opt +AUTHORIZED_KEYS=/root/.ssh/authorized_keys +DIR=`dirname "$AUTHORIZED_KEYS"` +mkdir -p "$DIR" +chmod 700 "$DIR" +chown root:root "$DIR" +pwd +ssh-keygen -t rsa -N "" -f id_rsa +cat /opt/id_rsa.pub >> "$AUTHORIZED_KEYS" + +## X11 forwarding +$minimal_apt_get_install xauth + +## Setup environment +sed -i "1iexport HOME=/root" /root/.profile +sed -i "1isource /etc/container_environment.sh" /root/.profile diff --git a/build/scripts/01_syslog.sh b/build/scripts/01_syslog.sh new file mode 100644 index 0000000..9908a7f --- /dev/null +++ b/build/scripts/01_syslog.sh @@ -0,0 +1,11 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Install a syslog daemon +$minimal_apt_get_install syslog-ng-core +mkdir /etc/service/syslog-ng +mv /build/runit/syslog-ng /etc/service/syslog-ng/run +mkdir -p /var/lib/syslog-ng +sed -i "s/^\(#SYSLOGNG_OPTS=\).*$/\1\"--no-caps --default-modules=affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat\"/" /etc/default/syslog-ng diff --git a/build/scripts/02_hosts.sh b/build/scripts/02_hosts.sh new file mode 100644 index 0000000..2a23e9b --- /dev/null +++ b/build/scripts/02_hosts.sh @@ -0,0 +1,13 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Hosts file hack +LD_LIBRARY_PATH=/root/lib +mkdir -p $LD_LIBRARY_PATH +cp /lib/x86_64-linux-gnu/libnss_files.so.2 $LD_LIBRARY_PATH +sed -i 's,/etc/hosts,/tmp/hosts,' $LD_LIBRARY_PATH/libnss_files.so.2 + +## Environment variable +echo -n $LD_LIBRARY_PATH > /etc/container_environment/LD_LIBRARY_PATH \ No newline at end of file diff --git a/build/scripts/02_user.sh b/build/scripts/02_user.sh new file mode 100644 index 0000000..bb62e14 --- /dev/null +++ b/build/scripts/02_user.sh @@ -0,0 +1,7 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## User +adduser --disabled-password --gecos "" user diff --git a/build/scripts/02_utilities.sh b/build/scripts/02_utilities.sh new file mode 100644 index 0000000..2b840e9 --- /dev/null +++ b/build/scripts/02_utilities.sh @@ -0,0 +1,10 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Often used tools +$minimal_apt_get_install wget curl python-pip inetutils-ping telnet sox + +## Often used python modules +pip install argparse diff --git a/build/scripts/03_btsync.sh b/build/scripts/03_btsync.sh new file mode 100644 index 0000000..c7d98ca --- /dev/null +++ b/build/scripts/03_btsync.sh @@ -0,0 +1,21 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Redis +cd opt +wget -O btsync.tar.gz http://download-lb.utorrent.com/endpoint/btsync/os/linux-x64/track/stable +tar xzf btsync.tar.gz +rm btsync.tar.gz + +export BTSYNC_CONFIG=/host/etc/btsync.conf +export BTSYNC_DATA=/host/var/btsync + +## Runit script +mkdir /etc/service/btsync +mv /build/runit/btsync /etc/service/btsync/run + +## Environment variables +echo -n $BTSYNC_CONFIG > /etc/container_environment/BTSYNC_CONFIG +echo -n $BTSYNC_DATA > /etc/container_environment/BTSYNC_DATA diff --git a/build/scripts/03_chrome.sh b/build/scripts/03_chrome.sh new file mode 100644 index 0000000..ac2a6b5 --- /dev/null +++ b/build/scripts/03_chrome.sh @@ -0,0 +1,14 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Chrome dependencies +$minimal_apt_get_install gconf-service libasound2 libatk1.0-0 libcairo2 libcap2 libcups2 libcurl3 libfontconfig1 libgdk-pixbuf2.0-0 libgtk2.0-0 libnspr4 libnss3 libpango1.0-0 librtmp0 libxss1 libxtst6 xdg-utils + +## Chrome +mkdir -p /usr/share/icons/hicolor +cd /opt +wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb +dpkg -i google-chrome-stable_current_amd64.deb +rm google-chrome-stable_current_amd64.deb diff --git a/build/scripts/03_firefox.sh b/build/scripts/03_firefox.sh new file mode 100644 index 0000000..a6326c6 --- /dev/null +++ b/build/scripts/03_firefox.sh @@ -0,0 +1,11 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Chrome dependencies +$minimal_apt_get_install firefox + +## Pulseaudio script +echo "PULSE_SERVER=host firefox" > /opt/firefox +chmod +x /opt/firefox diff --git a/build/scripts/03_hamachi.sh b/build/scripts/03_hamachi.sh new file mode 100644 index 0000000..b28f3a3 --- /dev/null +++ b/build/scripts/03_hamachi.sh @@ -0,0 +1,18 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Hamachi dependency +$minimal_apt_get_install lsb-core net-tools socat strace + +## Hamachi +cd /opt +export HAMACHI_DATA=/host/var/lib/logmein-hamachi +export HAMACHI_VERSION=2.1.0.119 +wget -O hamachi.deb https://secure.logmein.com/labs/logmein-hamachi_$HAMACHI_VERSION-1_amd64.deb +mkdir -p /etc/service/hamachi +cp /build/runit/hamachi /etc/service/hamachi/run + +## Environment variables +echo -n $HAMACHI_DATA > /etc/container_environment/HAMACHI_DATA \ No newline at end of file diff --git a/build/scripts/03_pulseaudio.sh b/build/scripts/03_pulseaudio.sh new file mode 100644 index 0000000..c8ad7d5 --- /dev/null +++ b/build/scripts/03_pulseaudio.sh @@ -0,0 +1,10 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Pulseaudio +$minimal_apt_get_install pulseaudio + +## Setup environment +sed -i "1iexport PULSE_SERVER=host" /root/.profile \ No newline at end of file diff --git a/build/scripts/03_redis.sh b/build/scripts/03_redis.sh new file mode 100644 index 0000000..2c23089 --- /dev/null +++ b/build/scripts/03_redis.sh @@ -0,0 +1,24 @@ +#!/bin/bash +set -e +source /build/config +set -x + +## Redis +$minimal_apt_get_install redis-server + +export REDIS_CONFIG=/etc/redis/redis.conf +export REDIS_LOG=/host/var/log/redis/redis.log +export REDIS_DATA=/host/var/lib/redis + +sed -i "s,^\(daemonize\s*\).*$,\1no," $REDIS_CONFIG +sed -i "s,^\(logfile\s*\).*$,\1$REDIS_LOG," $REDIS_CONFIG +sed -i "s,^\(dir\s*\).*$,\1$REDIS_DATA," $REDIS_CONFIG + +## Runit script +mkdir /etc/service/redis +mv /build/runit/redis /etc/service/redis/run + +## Environment variables +echo -n $REDIS_CONFIG > /etc/container_environment/REDIS_CONFIG +echo -n $REDIS_LOG > /etc/container_environment/REDIS_LOG +echo -n $REDIS_DATA > /etc/container_environment/REDIS_DATA diff --git a/build/scripts/99_cleanup.sh b/build/scripts/99_cleanup.sh new file mode 100644 index 0000000..70eaf47 --- /dev/null +++ b/build/scripts/99_cleanup.sh @@ -0,0 +1,10 @@ +#!/bin/bash +set -e +source /build/config +set -x + +apt-get clean +rm -rf /build +rm -rf /tmp/* /var/tmp/* +rm -rf /var/lib/apt/lists/* +rm -f /etc/dpkg/dpkg.cfg.d/02apt-speedup diff --git a/images/base/Makefile b/images/base/Makefile new file mode 100644 index 0000000..95c8518 --- /dev/null +++ b/images/base/Makefile @@ -0,0 +1,6 @@ +include /opt/docker/build/Makefile + +NAME = base +VERSION = latest + +build: base \ No newline at end of file diff --git a/images/btsync/Makefile b/images/btsync/Makefile new file mode 100644 index 0000000..a45f834 --- /dev/null +++ b/images/btsync/Makefile @@ -0,0 +1,7 @@ +include /opt/docker/build/Makefile + +NAME = btsync +VERSION = latest +RUN = -p 8888:8888 -v /opt/docker/images/btsync/fs:/host + +build: base btsync \ No newline at end of file diff --git a/images/chrome/Makefile b/images/chrome/Makefile new file mode 100644 index 0000000..63ff628 --- /dev/null +++ b/images/chrome/Makefile @@ -0,0 +1,6 @@ +include /opt/docker/build/Makefile + +NAME = chrome +VERSION = latest + +build: base pulseaudio chrome \ No newline at end of file diff --git a/images/firefox/Makefile b/images/firefox/Makefile new file mode 100644 index 0000000..4457f5a --- /dev/null +++ b/images/firefox/Makefile @@ -0,0 +1,6 @@ +include /opt/docker/build/Makefile + +NAME = firefox +VERSION = latest + +build: base pulseaudio firefox \ No newline at end of file diff --git a/images/hamachi/Makefile b/images/hamachi/Makefile new file mode 100644 index 0000000..37f92dd --- /dev/null +++ b/images/hamachi/Makefile @@ -0,0 +1,7 @@ +include /opt/docker/build/Makefile + +NAME = hamachi +VERSION = latest +RUN = --privileged -v /opt/docker/images/hamachi/fs:/host + +build: base hamachi \ No newline at end of file diff --git a/images/redis/Makefile b/images/redis/Makefile new file mode 100644 index 0000000..1704c83 --- /dev/null +++ b/images/redis/Makefile @@ -0,0 +1,7 @@ +include /opt/docker/build/Makefile + +NAME = redis +VERSION = latest +RUN = -p 6379:6379 -v /opt/docker/images/$CONTAINER/fs:/host + +build: base redis \ No newline at end of file diff --git a/setup b/setup index d37118b..1c0b710 100644 --- a/setup +++ b/setup @@ -1,2 +1,28 @@ #!/bin/sh set -e +# +# This script is meant for quick & easy install via: +# 'curl -sL https://raw.githubusercontent.com/Boukefalos/docker-deployment/master/setup | sh' +# or: +# 'wget -qO- https://raw.githubusercontent.com/Boukefalos/docker-deployment/master/setup | sh' + +home=/etc/environment +url='https://get.docker.io/' + +command_exists() { + command -v "$@" > /dev/null 2>&1 +} + +if !command_exists docker; then + echo Docker should be installed! + exit 0 +fi + +curl='' +if command_exists curl; then + curl='curl -sL' +elif command_exists wget; then + curl='wget -qO-' +elif command_exists busybox && busybox --list-modules | grep -q wget; then + curl='busybox wget -qO-' +fi \ No newline at end of file