Minimal Docker Image of eXist-db NoSQL Database Client/Server with FO support
This module holds the source files for building a minimal docker image of the exist-db xml database, images are automatically updated as part of the build-test life-cycle. The images are based on Google Cloud Platform's "Distroless" Docker Images.
- Docker:
18-stable
Pre-build images are available on DockerHub. There are two continuously updated channels:
release
for the stable releases based on themaster
branchlatest
for the latest commit to thedevelop
branch.
Multi-arch Images are currently produced for linux/amd64
and linux/arm64
.
To download tanhe image run:
docker pull duncdrum/existdb:latest
Once the download is complete, you can run the image
docker run -dit -p 8080:8080 -p 8443:8443 --name exist duncdrum/existdb:latest
-it
allocates a TTY and keeps STDIN open. This allows you to interact with the running Docker container via your console.-d
detaches the container from the terminal that started it. So your container won't stop when you close the terminal.-p
maps the Containers internal and external port assignments (we recommend sticking with matching pairs). This allows you to connect to the eXist-db Web Server running in the Docker container.--name
lets you provide a name (instead of using a randomly generated one)
The only required parts are docker run duncdrum/existdb
.
For a full list of available options see the official Docker documentation
After running the pull
and run
commands, you can access eXist-db via localhost:8080 in your browser.
To stop the container issue:
docker stop exist
or if you omitted the -d
flag earlier press CTRL-C
inside the terminal showing the exist logs.
You can interact with a running container as if it were a regular Linux host (without a shell in our case). You can issue shell-like commands to the Java admin client, as we do throughout this readme, but you can't open the shell in interactive mode.
The name of the container in this readme is exist
, adjust the name in the commands to suit your needs:
# Using java syntax on a running eXist-db instances
docker exec exist java org.exist.start.Main client --no-gui --xpath "system:get-version()"
# Interacting with the JVM
docker exec exist java -version
Containers build from this image run periodical healthchecks to ensure that eXist-db is operating normally.
If docker ps
reports unhealthy
you can get a more detailed report with this command:
docker inspect --format='{{json .State.Health}}' exist
There is a slight modification to eXist's logger to ease access to the logs via:
docker logs exist
This works best when providing the -t
flag when running an image.
A common usage of these images is as a base image for your own applications.
We'll take a quick look at three scenarios of increasing complexity, to demonstrate how to achieve common tasks from inside Dockerfile
.
The simplest and straightforward case assumes that you have a .xar
app inside a build
folder on the same level as the Dockerfile
.
To get an image of an eXist-db instance with your app installed and running, simply adopt the docker cp ...
command to the appropriate Dockerfile
syntax.
FROM duncdrum/existdb:6.0.1
COPY build/*.xar /exist/autodeploy
You should see something like this:
Sending build context to Docker daemon 4.337MB
Step 1/2 : FROM duncdrum/existdb:6.0.1
---> 3f4dbbce9afa
Step 2/2 : COPY build/*.xar /exist/autodeploy
---> ace38b0809de
The result is a new image of your app installed into eXist-db.
Since you didn't provide further instructions it will simply reuse the EXPOSE
, CMD
, HEALTHCHECK
, etc instructions defined by the base image.
You can now publish this image to a docker registry and share it with others.
The following example will install your app, but also modify the underlying eXist-db instance in which your app is running.
Instead of a local build directory, we'll download the .xar
from the web, and copy a modified conf.xml
from a src/
directory along side your Dockerfile
.
To execute any of the docker exec …
style commands from this readme, we need to use RUN
.
FROM duncdrum/existdb
# NOTE: this is for syntax demo purposes only
RUN [ "java", "org.exist.start.Main", "client", "--no-gui", "-l", "-u", "admin", "-P", "", "-x", "sm:passwd('admin','123')" ]
# use a modified conf.xml
COPY src/conf.xml /exist/etc
ADD https://github.com/eXist-db/documentation/releases/download/4.0.4/exist-documentation-4.0.4.xar /exist/autodeploy
The above is intended to demonstrate the kind of operations available to you in a single stage build.
For security reasons more elaborate techniques for not sharing your password in the clear are highly recommended,
such as the use of secure variables inside your CI environment.
However, the above shows you how to execute the Java Admin Client from inside a Dockerfile
,
which in turn allows you to run any XQuery code you want when modifying the eXist-db instance that will ship with your images. You can also chain multiple RUN
commands.
As for the sequence of the commands, those with the most frequent changes should come last to avoid cache busting.
Chances are, you wouldn't change the admin password very often, but the .xar
might change more frequently.
Lastly, you can eliminate external dependencies even further by using a multi-stage build. To ensure compatibility between different Java engines we recommend sticking with debian based images for the builder stage.
The following 2-stage build will download and install ant
and nodeJS
into a builder stage which then downloads frontend dependencies before building the .xar
file.
The second stage (each FROM
begins a stage) is just the simple example from above.
Such a setup ensures that non of your collaborators has to have java
or nodeJS
installed, and is great for fully automated builds and deployment.
# START STAGE 1
FROM openjdk:8-jdk-slim as builder
USER root
ENV ANT_VERSION 1.10.5
ENV ANT_HOME /etc/ant-${ANT_VERSION}
WORKDIR /tmp
RUN wget http://www-us.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz \
&& mkdir ant-${ANT_VERSION} \
&& tar -zxvf apache-ant-${ANT_VERSION}-bin.tar.gz \
&& mv apache-ant-${ANT_VERSION} ${ANT_HOME} \
&& rm apache-ant-${ANT_VERSION}-bin.tar.gz \
&& rm -rf ant-${ANT_VERSION} \
&& rm -rf ${ANT_HOME}/manual \
&& unset ANT_VERSION
ENV PATH ${PATH}:${ANT_HOME}/bin
WORKDIR /home/my-app
COPY . .
RUN apk add --no-cache --virtual .build-deps \
nodejs \
nodejs-npm \
git \
&& npm i npm@latest -g \
&& ant
# START STAGE 2
FROM duncdrum/existdb:release
COPY --from=builder /home/my-app/build/*.xar /exist/autodeploy
EXPOSE 8080 8443
CMD [ "java", "org.exist.start.Main", "jetty" ]
The basic idea of the multi-staging is that everything you need for building your software should be managed by docker,
so that all collaborators can rely on one stable environment. In the end, and after how ever many stages you need,
only the files necessary to run your app should go into the final stage. The possibilities are virtually endless,
but with this example and the Dockerfile
in this repo you should get a pretty good idea of how you might apply this idea to your own projects.
We highly recommend use of a docker-compose.yml
for use with docker-compose.
docker-compose for local development or integration into multi-container environments.
For options on how to configure your own compose file, follow the link at the beginning of this paragraph.
To start exist using a compose file, type:
# starting eXist-db
docker-compose up -d
# stop eXist-db
docker-compose down
Volumes let you ensure data persistence between reboots, in particular:
exist/data
so that any database changes persist through reboots and updates.exist/etc
so you can configure eXist startup options.
can be declared as mount volumes.
You can configure additional volumes e.g. for backups,
or additional services such as an nginx reverse proxy via a docker-compose.yml
, to suite your needs.
To update the exist-docker image from a newer version
docker-compose pull
As with normal installations, the password for the default dba user admin
is empty.
Change it via the usermanager or from CLI (s.a.).
There are unit tests for our images that run on CI using the bats framework. The test are located in exist-docker/src/test/bats
.
To execute them run:
bats exist-docker/src/test/bats/*.bats
The tests use fixtures and are creating a modified image call ex-mod
. By default they expect a name container exist-ci
to be up and running. When running test locally you must ensure that no previous image ex-mod
exists, and that exist-ci
is running before starting the testsuite.
eXist-db's cache size and maximum brokers can be configured at build time using the following syntax.
mvn -DskipTests clean package docker:build --build-arg MAX_CACHE=312 MAX_BROKER=15 .
NOTE: Due to the fact that the final images does not provide a shell, setting ENV variables via docker does not work.
# !This has no effect!
docker run -dit -p8080:8080 -e MAX_BROKER=10 ae4d6d653d30
If you wish to permanently adopt a customized cache or broker configuration,
you can either make a local copy of the Dockerfile
and edit the default values there.
ARG MAX_BROKER=10
Or modify eXist-db's configuration files via xslt scripts located at exist-docker/src/main/xslt/
.
For multi-stage builds e.g. xmlstarlet let's you modify the default config files from within the builder stage,
e.g.:
# Config files are modified here
RUN echo 'modifying conf files'\
&& cd $EXIST_HOME/etc \
&& xmlstarlet ed -L -s '/Configuration/Loggers/Root' -t elem -n 'AppenderRefTMP' -v '' \
-i //AppenderRefTMP -t attr -n 'ref' -v 'STDOUT'\
-r //AppenderRefTMP -v AppenderRef \
log4j2.xml
This image uses an advanced JVM configuration, via the JAVA_TOOL_OPTIONS
env variable inside the Dockerfile.
You should avoid the traditional way of setting the heap size via -Xmx
arguments,
this can lead to frequent crashes since Java8 and Docker are (literally) not on the same page concerning available memory.
Instead, use the -XX:MaxRAMPercentage
argument to modify the memory available to the JVM inside the container.
The default of 75%
should be save for production use.
To allocate e.g. 600mb to the container around the JVM use, however, you can tweak this at buildtime via the JVM_MAX_RAM_PERCENTAGE
argument. Note that percentages should include a decimal point 80.0
= 80%
.
To allocate e.g. 600mb to the container around the JVM use:
docker run -m 600m …
Lastly, this image uses a new garbage collection mechanism
garbage first (G1) -XX:+UseG1GC
and string deduplication -XX:+UseStringDeduplication
to improve performance.
Customizing individual arguments is not possible at run time. To adjust individual parameters the the whole JAVA_TOOL_OPTIONS
environment variable must be passed to docker run …