Docker Build Cache Reuse

Docker utilizes a layering system for its images. Each command in a Dockerfile creates a new layer that is built upon the previous one. This layering system is leveraged by Docker’s build cache. If Docker recognizes that a layer hasn’t changed (i.e., the command is the same and the context hasn’t changed), it reuses the existing layer instead of rebuilding it.

When building .NET applications in Docker, one common practice is to restore dependencies as a separate step before the application is built. This is done because restoring dependencies can be a time-consuming process, and it’s often the case that dependencies change less frequently than the application code. By restoring dependencies as a separate step, Docker can reuse the cached layer for the restore step, significantly speeding up the build process.

The recreate-sln-structure tool aids in this process by moving project files into the directories specified by the solution (.sln) file. This allows Docker to better track changes to individual projects and reuse cached layers when a project hasn’t changed, leading to more efficient builds.

The Need for recreate-sln-structure

When working with Docker, the COPY command is often used to copy files from the host into the Docker image. However, when using wildcards to copy multiple files, Docker flattens the directory structure. This means that all files, regardless of their original directory, are copied into the same destination directory.

This can be problematic when working with .NET solutions, as the structure of the solution can be important. For example, project references are often based on the relative paths of the projects within the solution.

This is where recreate-sln-structure comes in. After copying the project files into the Docker image, recreate-sln-structure can be used to move the project files into the directories specified by the solution (.sln) file, effectively recreating the original solution structure within the Docker image.

Working with Solution Filter (.slnf) Files

Solution Filter (.slnf) files are a feature in Visual Studio that allows developers to open a subset of a larger solution. This is particularly useful in large codebases where loading the entire solution can be time-consuming and resource-intensive.

The slnf-gen tool generates these .slnf files based on globs. For example, you could exclude all test projects, or only include projects in a specific directory. When you open the .slnf file in Visual Studio, only the included projects are loaded. This can significantly speed up solution load times and reduce the amount of memory used by Visual Studio, making it easier to work on large solutions.

In addition, .slnf files can also be used with the dotnet CLI. For example, you can restore or build only the projects included in the .slnf file. This can be useful in CI/CD pipelines where you might want to build or test a subset of a larger solution.

Using slnf-gen and recreate-sln-structure in Tandem

slnf-gen and recreate-sln-structure can be used together to create an extremely lean containerized .NET build process.

In this Dockerfile, we first copy the solution and project files into the Docker image. We then use recreate-sln-structure to restore the original solution structure within the Docker image.

Next, we use slnf-gen to generate a .slnf file that includes only the necessary projects for the build, excluding any test projects. We then restore these projects using dotnet restore.

After the necessary projects have been restored, we copy the rest of the application files into the Docker image. Finally, we build the necessary projects using dotnet build.

This process results in a Docker image that includes only what’s necessary for the application to run, making the image as lean and efficient as possible.

Here’s an example of what this might look like in a Dockerfile:

FROM bitnami/dotnet-sdk:6
# Copy solution and project files into Docker image
COPY ["MySolution.sln", "**/*.csproj", "./"]

# Recreate solution structure, generate .slnf file, and restore. 
# This step's cache is only invalidated when .sln or .csproj files are modified (or added)
RUN recreate-sln-structure MySolution.sln && \
    slnf-gen MySolution.sln --exclude **/*Test* && \
    dotnet restore MySolution.slnf

# Copy the rest of the source files
COPY ["./", "./"]

# Build the necessary projects. This steps cache is reused across usages with different values for PUBLISH_PROJECT
RUN dotnet build MySolution.slnf --no-restore

# Argument for the project to be published
ARG PUBLISH_PROJECT=MyProject

# Publish the specified project without building it again
RUN dotnet publish "${PUBLISH_PROJECT}/${PUBLISH_PROJECT}.csproj" --no-build --output /app/

...

FROM ...

In this Dockerfile, we first copy the solution and project files into the Docker image. We then use recreate-sln-structure to restore the original solution structure within the Docker image. This is necessary because Docker flattens the directory structure when copying multiple files.

Next, we use slnf-gen to generate a .slnf file that includes only the necessary projects for the build, excluding any test projects. We then restore these projects using dotnet restore. This step’s cache is only invalidated when .sln or .csproj files are modified or added, which means Docker can reuse the cached layer for the restore step in most builds, significantly speeding up the build process.

After the necessary projects have been restored, we copy the rest of the source files into the Docker image. We then build the necessary projects using dotnet build with the –no-restore option, which ensures that we don’t restore the projects again.

Finally, we use an argument to specify the project to be published. We then publish this project using dotnet publish with the –no-build option, which ensures that we don’t build the project again. The published files are output to the /app/ directory in the Docker image.

This makes the build process as lean and efficient as possible since we only build exactly what is necessary, when it is necessary.

In conclusion, slnf-gen and recreate-sln-structure are powerful tools that can help streamline your .NET development workflow. Whether used individually or in tandem, they offer practical solutions to common challenges in .NET project management and Docker containerization.