Navigating Dagger: A Tech Journey from Novice to Ninja

In the fast-paced realm of technology, our /mid team at Puzzle ITC recently devoted a week to exploring new horizons (/mid Week 2023 Blog). My groups spotlight fell on Dagger: CI/CD as Code that Runs Anywhere. From tentative first encounters to mastering advanced features, our journey through Dagger (version v0.9.3) promises insights for both beginners and seasoned developers. Join us as we unravel the layers of Dagger, sharing the highs, lows, and the secrets to the programmable CI/CD engine that runs your pipelines in containers. Get ready for a tech odyssey—from novices to masters. Let’s dive in!

Dagger@Midweek: Impressions of a newbie

… while encountering numerous intriguing topics. I chose to dedicate most of the time to exploring Dagger, a potential rising star in the CI/CD domain. Unlike traditional approaches that rely on declarative methods like YAML or domain-specific languages (DSLs) to define CI/CD pipelines (plus the necessity for a central CI/CD solution to execute them), Dagger adopts a developer-centric approach. It allows the creation of pipelines using any of its supported programming languages. Notably, in contrast to most other CI/CD solutions, Dagger enables local pipeline execution.

Hands-on Experience

I successfully set up the Dagger CLI and SDK on both OSX and Ubuntu Server. I completed the quick start tutorials for Go and Python, although I encountered some minor challenges with Python due to library dependency issues. I also connected a Dagger runner to my personal GitLab instance, which necessitated adding an internal CA certificate to the Dagger runner. I worked on building an existing sample project using an already existing Dockerfile, which involved building and pushing the image to a registry. This was part of an effort to reuse existing code while gradually transitioning to Dagger. Furthermore, I revamped the functionality of an existing Dockerfile and .gitlab-ci.yml into a Dagger pipeline written in Go, aiming to replace the Dockerfile and simplify the .gitlab-ci.yml.

Impressions

My impressions of Dagger are highly promising. Developing pipelines in a programming language leverages all its features and benefits. The ability to run pipelines locally eliminates the need for a remote CI/CD solution, though the possibilities for debugging currently seem limited. The feasibility of performing and testing pipelines might vary depending on the complexity of the pipeline, especially concerning external dependencies. The project’s adoption may depend on the level of investment in existing tooling, such as whether to replace current CI/CD chains or start afresh. Future development of the project will be crucial, considering factors like language availability, integration, and tooling. While the cloud offering of Dagger is still in its early stages and appears interesting, a comparable offline offering could also be valuable. Given that the project is still in an early stage, significant changes are to be expected, and it might be wise to keep an eye open for alternatives as well.

Daggerized pipeline

My aim was to re-write an existing Jenkins pipeline with Dagger.

Here we see the application build as Jenkins pipeline:

pipeline {
    stages {
        stage('Build App') {
            agent {
                docker {
                    image 'registry.access.redhat.com/ubi9/openjdk-11'
                    label 'docker'
                }
            }        
            steps {
                sh './mvnw clean package'
                stash includes: 'target/image/**', name: 'target'
            }
...

I choose the Dagger Python SDK because Python is a programming language that is easy to read for all developers.

This is my Dagger pipeline part for the application build:

import os
import sys
import anyio
import dagger

async def main():

    # initialize Dagger client
    async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client:
        # create a cache volume for Maven downloads
        maven_cache = client.cache_volume("maven-cache")

        # get reference to source code directory
        source = client.host().directory(".", include=["src/", "mvnw", "pom.xml", ".mvn/"])

        # build application
        build = (
            client.container()
            .from_("registry.access.redhat.com/openjdk/openjdk-11-rhel7:latest")
            .with_directory("/home/jboss/app",source, owner="185:185")
            .with_workdir("/home/jboss/app")
            .with_mounted_cache("/home/jboss/app/.m2", maven_cache, owner="185:185")
            .with_exec([
              "./mvnw", 
              "-Dmaven.repo.local=/home/jboss/app/.m2/repository", 
              "clean", 
              "package"
            ])
        )
...

Findings

• The commands for the build are still in Bash.
• For simple pipelines, I see no advantage over Jenkins or GitLab CI.
  • It is almost identical to a GitLab CI pipeline, just with different syntax.
  • Syntax with_exec is not elegant.
• Finding information is difficult.
  • Fragmentation with SDKs means that documentation is not equally good for everything.
  • Google search is difficult, beacuse there is an Android Dagger package.
  • The rapid changes make some things obsolete.

Go one step further

Having a build pipeline, how can we benefit from Dagger executed in containes?
We could package the application into an container image, start a database container besite the application container, connect them and run smoke tests.

This are the described steps inside the Python Dagger pipeline:

..
        # create database service container
        postgres = (
            client.container()
            .from_("registry.access.redhat.com/rhscl/postgresql-10-rhel7:latest")
            .with_env_variable("POSTGRESQL_USER", "demoapp")
            .with_env_variable("POSTGRESQL_PASSWORD", "demoapp")
            .with_env_variable("POSTGRESQL_DATABASE", "demoapp")
            .with_env_variable("POSTGRESQL_ADMIN_PASSWORD", "root")
            .with_exposed_port(5432)
            .as_service()
        )

        # create app as service container
        app = (
            client.container()
            .from_("registry.access.redhat.com/openjdk/openjdk-11-rhel7:latest")
            .with_directory("/deployments", build.directory("./target"))
            .with_workdir("/deployments")
            .with_service_binding("db", postgres)
            .with_env_variable("POSTGRESQL_USER", "demoapp")
            .with_env_variable("POSTGRESQL_PASSWORD", "demoapp")
            .with_env_variable("POSTGRESQL_DATABASE", "demoapp")
            .with_env_variable("POSTGRESQL_HOST", "db")
            .with_env_variable("JAVA_APP_JAR", "java-s2i-kustomize-pipeline-quickstart-0.0.1-SNAPSHOT-runner.jar")
            .with_exposed_port(8080)
            .as_service()
        )

        test = (
          client.container()
          .from_("registry.puzzle.ch/cicd/ubi9-base")
          .with_service_binding("app", app)
          .with_exec(["curl", "-I", "http://app:8080"])
        )

        await test.stdout()

anyio.run(main)

The development of the pipeline was simplified in the comparison to Jenkins and GitLab CI because that it can be run locally. This eliminates roundtrips over git and the ‘Push and Pray’ pipeline development.

Java SDK

We where very happy as we got the information over the availability of the dagger-java-sdk, even when it is still experimental.
Our company and lots of the developers have a Java background. Even most of our CI/CD team members know how to code and build Java applications.
That is a big change for Dagger to gain a broad support by the developers. Experienced Java devs can use thier tooling and knowledge to build pipelines!

Although a Java SDK exists, it is not yet public. Therefore, it must first be built. This makes its integration into Java projects somewhat cumbersome at the moment. The SDK Jars, including dependencies, must be locally available. However, once this initial hurdle is overcome, you can benefit from a well-documented and functional API for the early development stage.

Dagger Java Modules

Dagger started Project Zenith (Docs) to be able to create and use language-agnostic APIs, enabling seamless collaboration across different languages and tools.
In short, you can develop a functionality / module in the Dagger SDK of your choice and it can be used in any one of the Dagger SDK’s. As example you write a module in Go and you can integrate it into your Java pipeline.

On Daggerverse (hub for Dagger modules), Java/Maven and Gradle modules are already available. By using these modules, Java projects can be built without the need for Java and the not-yet-officially-available Java SDK on the build server. Additionally, the number of required lines of code dramatically decreases since the modules abstract away the “low-level” commands. However, integrating these modules into a “real-life” process as part of a pipeline can be challenging. One quickly realizes how limited they are compared to the possibilities that the SDK would offer. Since the modules are developed independently of Dagger, their functionality, code quality, and documentation vary greatly. Dagger’s documentation for modules is also sparse, mainly because the module concept itself is relatively new even for Dagger. Furthermore, standardizing module APIs would be desirable, as it would significantly simplify their use.

Test the Java SDK

To run a Java pipeline, the Java SDK is needed as a JAR file containing all dependencies. Get the Dagger repository and change to the sdk/java directory.

Then, this self-contained JAR file can be built with this command:

./mvnw clean package -Pbigjar,release

To run a sample, the classpath has to contain the Java SDK JAR file and the samples JAR file.

The following command uses the Dagger CLI to start the ListEnvVars sample:

dagger run java \
  -cp dagger-java-sdk/target/dagger-java-sdk-1.0.0-SNAPSHOT-jar-with-dependencies.jar:dagger-java-samples/target/dagger-java-samples-1.0.0-SNAPSHOT.jar \
  io.dagger.sample.ListEnvVars

Example to build with the Java module

This example shows how to build a Java application with Maven using the Dagger Java Module.

dagger call -m github.com/jcsirot/daggerverse/java \
  with-jdk --version 17 \
  with-maven --version 3.9.5 \
  with-project --source "." \
  maven --args clean,package

Pipeline Tests

Dagger offers many advantages, and a significant benefit is that with the SDK of a higher-level programming language such as Java, the concepts for testing logic are more mature. This provides many advantages over bash scripts or Groovy code.

In the OPS/CICD world, the concepts for automated tests are not yet applied to the same extent as in software development. Therefore, there are both opportunities and risks here. Shared functions and modules can be isolatedly tested. Additionally, the specific context and the associated path can be automatically verified.

Context

Environment variables configured through the build server are usually not or only sparsely validated. Often, besides credentials and hosts, items like git branches, tags, or commit hashes are used. With Dagger and a selected SDK, it is possible to validate and test the variables set via environment variables. An example would be the branch name or tag, where it can be checked if it matches an expected pattern.

Path

In CICD pipelines, steps are often distinguished based on the git branches or with context regarding which steps should be executed. Various paths are possible, where only builds, only deployments, both, or other steps are executed. In medium to large projects, this can evolve into complex logic. Certain functions are often reused in this logic. This means that, with a change that should only affect one path of the pipeline, it cannot be verified, or only manually verified, whether other paths are affected.

To check behavior in a specific context, there are different approaches. It is often validated whether a function is called, in what order, and with what parameters. The actual code of the function is not tested; instead, it is checked whether the expected functions have been called in the correct order with the correct parameters. The logic of the function is covered either by its own unit tests or by the library.

Bleeding Edge Technologies

Working with the latest technologies always comes with pros and cons. We dive into promising technologies early to stay ahead of our customers.

Dagger is still very new, posing some challenges. The support in Discord is exemplary, and we usually get prompt responses.

Challenges

We use Rocky Linux as Jenkins Build Agents. When starting Dagger pipelines, we found that the Dagger Engine doesn’t start. It’s likely due to nftables being used instead of iptables. The question to the community got answered and we could fix it.

Very new features, such as the Java SDK or the modules, have also presented some issues. The Java SDK is still experimental and needs to be built manually. After that, it works quite well. The modules are a great approach and will significantly improve and transform Dagger. Unfortunately, the modules are (still) not compatible with Podman, so I had to switch back to Docker. With the modules, Dagger is now in its third iteration of how it’s used. Dagger evolves quickly, so pipelines we created this year are already “deprecated.”

The Dagger engine has somtimes DNS problems when you swith networks. Finding out the reason can be time consuming. But it can be fixed by restarting it:

DAGGER_ENGINE_DOCKER_CONTAINER="$(docker container list --all --filter 'name=^dagger-engine-*' --format '{{.Names}}')"
docker restart "$DAGGER_ENGINE_DOCKER_CONTAINER"

Contributions

Working with new technologies also provides the opportunity to give back. While exploring and testing Dagger we did some contributions:

• Added Java SDK schema for Engine 0.9.3
  • feat: add API schema for dagger 0.9.3
• Documented Dagger CLI usage with the Java SDK
  • docs: Java SDK Dagger CLI example
• jcsirot/java-with-dagger-demo/ – extend usage doc

Conclusion

---

What is Dagger?
Dagger is an integrated platform to orchestrate the delivery of applications to the cloud from start to finish. The Dagger Platform includes the Dagger Engine, Dagger Cloud, and the Dagger SDKs.

This is the official text from the Dagger documentation.

---

Our impressions of Dagger’s potential were positive, emphasizing the benefits of developing pipelines in a programming language and the possibility to execute the pipeline locally, although debugging options seemed somewhat limited.

We were happy to have an experienced Dagger user and contributor on our side. With his help we were faster at the point where we could write and run our own pipelines. From there on we could use the language and the tooling around that we are accustomed and feel confident in.

Our Company has a majority of Java developer. The Java SDK would be a good match. This would help to overcome the barrier between development and CI/CD. We look forward that it becomes a fully supported Dagger SDK.

One more benefit that we see with Dagger is the help for migrating Build Server. The pipelines could be replaced part by part with Dagger steps (Strangler Pattern). The development can be done locally and will be faster than developing on the Build Server (‘No more Push and Pray’). When the pipeline is fully migrated to Dagger it can be moved unchanged to the new Build Server.

Want to read more about Dagger?

• A Mate with Christian Schlatter about Dagger.
• The state of CI Tools in 2023 – Dagger, the future (part 2/3)
• Part three of the blog post series will be published soon …

Despite the challenges associated with bleeding-edge technologies, Dagger’s rapid evolution, coupled with a supportive community in platforms like Discord, exemplifies the exciting possibilities and continuous growth within the Dagger ecosystem.
As we navigate the ever-changing landscape of CI/CD solutions, Dagger stands out as a compelling player, and our journey from novices to contributors has been both educational and rewarding.