initial commit for effective docker proposal

[coolbreeze413]

Added some content for using docker effectively like a local installation, so that we have best of both worlds.
It is not yet completely ready as in for end users, but is usable to check if this seems to be a direction we can go in.
Hence raised a draft PR to get some feedback if this seems to be good to pursue further.
All the new files are in a separate directory: https://github.com/QuickLogic-Corp/quicklogic-fpga-toolchain/tree/proposal-for-effective-docker/more_effective_docker

[Thirsty2]

Thanks for the suggested improvements - something like this is certainly a great idea. As you point out, running as root in the container makes using the resulting files from your own bind mounted user directory difficult as they don't have the correct UID or GID. I am learning as I go here, and welcome any suggestions. One thing I learned as I was trying to get a Dockerfile accepted in a different project (iverilog) is that it is difficult to make changes inside the dockerfile once you create a new user and start running as that user in the dockerfile. As a result, it may be best to add and run as the user late in the Dockerfile, after all updates etc. have been made. Alternatively, using a dockerfile that does everything as root and has no entrypoint as a base image, then building a new dockerfile starting FROM that dockerfile image and adding a user and entrypoint may actually be cleaner. One reason for this becomes obvious if you take a look at my pull request to the iverilog project here: steveicarus/iverilog#373
Notice that this dockerfile starts by creating a base image that is common to both the build and final target images, and that there are test images that start from those points. Further, notice that there are two points in this dockerfile where the user is added. This made me feel icky, and prevented me from adding further ARG elements for user id and group id, and caused me to re-think things. I haven't changed the iverilog pull request yet, but I think simplifying it a bit and removing the entrypoint is a good idea there. You can always build a new dockerfile that starts FROM any particular dockerfile, but doing so is probably easier if you haven't already chosen entrypoints and added a user.

[coolbreeze413]

Yes I agree thay adding a user late in the dockerfile is good, however in this case, I find that once we have the toolchain installed as root, we cannot run that as any other user, all the binaries etc have their permission levels at root.
The only way to get the toolchain to execute is as root.
Hence I have added the user first, and then invoke the toolchain installer.
This is geared more toward the end developer than the CI, so have kept this separate from the main docker approach that can continue to be used for CI and installer verification.
I have seen the same problem when using an Android toolchain (aosp) via docker as we need various different dependencies for Android 5, 6, 7 and 8+.
If again there, we don't run the toolchain installation as user, and keep it as root, we can no longer invoke the toolchain binaries while running as user later.

[coolbreeze413]

Also, if you look at the invocation at the end, it is meant to basically transparently replace a local installation which we can directly invoke, just like calling ql_symbiflow directly.
This can only be achieved by using an entrypoint + cmd override.
If you could suggest an alternative approach which might be cleaner, we could look at the options.
I share your view of the flow not being clean enough, but unfortunately I haven't been able to devise a more efficient approach, to just be able to invoke tools in docker as if they were just like a local install.

[Thirsty2]

Coolbreeze413, thanks for clarifying the issues.

I took the time to simplify the Dockerfile in my pull request for the iverilog project, removing the extra target with the make entrypoint. Most of the remaining complexity is because it runs regression tests before the final default image target is built.

I have created an example repository that uses the iverilog image as a starting point, adds a user, and installs make, adding it as the entrypoint. There is a Makefile in the test directory that shows an example of a very simple project. That example project is here:
https://github.com/Thirsty2/iverilog-make/blob/main/alpine312/Dockerfile

My point here is that it may be better to keep the base/parent Dockerfiles simpler, and add user and entrypoints in a wrapper dockerfile project. I don't yet know whether we can adapt this approach for the Symbiflow project as it is.

Ideally, docker containers should be single purpose. There are some challenges with this approach with more complex projects like Symbiflow. I still think it is worth creating dockerfiles for Symbiflow for CI/CD purposes, but I think if we were designing this to run in containers from the beginning we would try to give each container a single purpose. Perhaps we can do some of that by factoring the installer into its separate components (and separate containers,) but it depends on whether the components are coupled to each-other in ways that require the components to access each other as they run. For instance, cosimulation with MyHDL and iverilog through the VPI interface is currently more easily done with both of those tools installed in the same container. It probably doesn't have to be that way, but right now it is. I suspect there may be similar situations with SymbiFlow components being coupled in ways that drive them to be installed in the same container, but if not, we may be better off separating them into separate containers.

[coolbreeze413]

In principle, I agree completely with the view that the docker structure is meant to be layered, so it would make sense to have a base image with the tools installed (ql_symbiflow) in this case, and then add a user-specific layer with the permissions, or probably just do all that in the docker-run rather than in the image.
However, the pain point seems to be that this will not solve the problem that I was going for.
The goal of this method is to:

1. make the toolchain encapsulated in a docker image, so that its dependencies are decoupled from the host system
2. transparently use the docker-based toolchain as a replacement for a local installation

If you would be able to try out the approach where we create a base docker image, with the ql_symbiflow installed and then add the user info in another image layer, you can see the permission problems this creates, and then maybe you could suggest a more docker-like solution to this goal.
I have tried that, and find that it does not work for the goal as above.
The installation of ql_symbiflow sets a lot of the file/directory permissions as root, because the default user for docker build is root.
This is precisely what I am aiming to fix, in the least ugly way, with the dockerfile, adding a user with UID, GID same as the current user at the build step.

I am not well versed with the complete FPGA toolchain and the various components used, and their interdependencies - hence I would not be able to say whether we can decouple those into separate docker layers.
I only approach the complete FPGA toolchain as one "black-box" package and use that to realize the end goal of having a dockerized toolchain directly usable by the end user to compile FPGA designs.

[Thirsty2]

Coolbreeze413,

I am hoping to put some time into the dockerfiles for the quicklogic toolchain again soon. A friend of mine is working with the fomu toolchain, so I took some time to create a similar dockerfile for that repo. One of the contributors (umarcor) had a number of comments on my (now merged) pull request, most of which I agree with, and some of which I have brought up in previous conversations in this repo. Umacor also pointed out a project I was not aware of, which is more inline with the way docker should be used for symbiflow, and ultimately for the quicklogic tools. This repo splits the tools into individual dockerfiles. It appears that it is still a work in progress but it is worth following.

https://github.com/eine/symbiflow-containers/

The fomu toolchain and new dockerfile, which also uses symbiflow is here:
https://github.com/im-tomu/fomu-toolchain

I added a user (fomu) to the dockerfile in that repo, and it seems to work fine for me, but my UID and GID numbers are both 1000, and happen to match the default for the user fomu in the dockerfile, which may explain why I didn't have any issues with this quicklogic toolchain dockerfile either.

[Thirsty2]

I submitted a PR on this proposal-for-effective-docker branch. Basically, it fixes the tools permissions after the installer runs in Dockerfile.use-installer which allows you to add users in other Dockerfiles that begin FROM symbiflow-ql. This keeps the parent/base symbiflow-ql a little cleaner (no users, no entrypoint) which allows it to be a little more re-usable for CI/CD use, or simply for personal preference. There are a couple of examples, Dockerfile.user and Dockerfile.entrypoint-make that show some ways of doing this. I also modified Dockerfile.use-installer-user to show how it could instead start FROM symbiflow-ql built from the new Dockerfile.use-installer.

[Thirsty2]

I have created a simple sample project that includes a Dockerfile for building a container that runs make. It contains your fpga_helloworldhw sample, so I hope that's okay. I modified the Makefile in a similar fashion as described in this thread.

The project can be found here:
https://github.com/Thirsty2/symbiflow-ql-make

It shows how to build a new container on top of the symbiflow-ql container, adding an entrypoint and a user. Feel free to use it as you see fit.

[coolbreeze413]

@Thirsty2 thanks for taking the time to create this. Glad to see that we were not far off on the idea of how to use docker for the tools.
I had tried chmod after installing the toolchain via installer, and because it didn't work, I ended up adding a user while building the docker image. Looks like I might have made a mistake while changing permissions.
I will check your changes and update back.

[Thirsty2]

You are welcome. I took time to create a new repo for running the symbiflow tools interactively, with the intent of allowing programming. It is similar to the symbiflow-ql-make repo, but doesn't call make, and it adds the programming tool. Currently, the programmer only works if I run the container in --privileged mode, but I hope to eliminate that requirement. I checked it in here:

https://github.com/Thirsty2/symbiflow-ql-user

[Thirsty2]

I forked the qorc-sdk repo, and created a new 'add-dockerfiles' branch that contains working dockerfiles for running qorc-sdk. I tested it with the qf_helloworldhw sample project, and was able to change the LED to red and rebuild/reload the quickfeather successfully.

The dockerfiles start from symbiflow-ql:latest, then add the gcc arm compiler. Dockerfile.user adds a user on top of the resulting image of Dockerfile.

After I do a little more testing, and write some documentation in the README, I will likely submit a pull request to qorc-sdk.

The new Dockerfile and Dockerfile.user are under the dockerfiles directory here:
https://github.com/Thirsty2/qorc-sdk/tree/add-dockerfile

[Thirsty2]

Coolbreeze413,

I submitted a pull request to the qorc-sdk repo, but I don't seem to have permission to request a review and it is blocked until it is reviewed. Can you help?

Doug