Run App Builds in Parallel with Dynamic Matrix Builds on Jenkins

Jenkins is a widely used CI/CD tool for automated builds, test and deployments. It is usually very straight forward to setup Jenkins. You define a stage for static code analysis, test , build and deployment. Jenkins then runs each stage sequentially . However, when your app grows in complexity with multiple flavors and configurations, then it starts getting messy. Running builds sequentially for each build types and configuration combination can take hours. This is where dynamic matrix builds in Jenkins comes in, which allows to run builds in parallel. In this blog post, I will introduce matrix builds and how to implement it.

Photo by Jason Yuen on Unsplash

The Problem: Slow and Sequential Builds

Jenkins app builds usually start with a lint check, then static code analysis like sonarqube, then tests (unit tests, ui tests etc..) and then app build and lastly the app distribution to your distribution channel like firebase. Some stages needs to run sequential such as unit tests and static code analysis because output of unit tests is used in static code analysis for code coverage. It is all good until your apps complexity is low. When the complexity of app grows, such as adding new flavors, region specific builds etc…, it will take very long to run all builds sequentially. Usually, there are preprod, staging and prod build types. In addition to these, there might be market or region specific flavors. Furthermore, there might be free, paid, enterprise dimensions. Then you will need to generate preprodFree, preprodPaid, preprodEnterprise, stagingFree, stagingPaid, stagingEnterprise, prodFree, prodPaid, prodEnterprise builds. Consider adding region dimension to this, then it will increase the number of total combinations. Usually, it should be enough to run tests and static code analysis for only one type of combination. However, for builds and app distribution , builds needs to run for each combination. You can increase the hardware configuration (i.e memory, cpu) of the Jenkins configuration, but will be useful until a certain point, then you will have the same problem if a new configuration or dimension is added to your builds. This is not a scalable solution.

The Solution: Parallel Builds with Dynamic Matrix

The docs related to Jenkins matrix builds is here. Firstly, we need to define the axes of the matrix build. Each dimension becomes an axe of the matrix build such as build type or flavor :

stages {
        stage('Lint'){}
        stage('Test'){}
        stage('SonarQube'){}
        stage('BuildAndDistribute') {
            matrix {
                axes {
                    axis {
                        name 'BUILD_TYPE'
                        values 'preprod', 'staging', 'prod'
                    }
                    axis {
                        name 'FLAVOR'
                        values 'free', 'paid', 'enterprise'
                    }
                }
                stages {
                    stage('Build') {
                        steps {
                            echo "Do Build for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                    stage('Distribute') {
                        steps {
                            echo "Do Test for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                }
            }
        }
    }
}

The important problem here is that axis cannot be dynamic. Jenkins’ syntax doesn’t allow it. You can’t put a method or expression here. We need to list all possible values here. Jenkins will create all combinations of these axises. However, we may not need all combinations. In such cases, there are some solutions.

1.Exclude:

Suppose some combinations are not wanted, then these ones could be out in exclude section. For example: If preprodPaid is not a possible combination for your project, then it can be skipped with exclude. Then combinations will be created without preprodPaid.

stages {
        stage('Lint'){}
        stage('Test'){}
        stage('SonarQube'){}
        stage('BuildAndDistribute') {
            matrix {
                axes {
                    axis {
                        name 'BUILD_TYPE'
                        values 'preprod', 'staging', 'prod'
                    }
                    axis {
                        name 'FLAVOR'
                        values 'free', 'paid', 'enterprise'
                    }
                }
                excludes {
                    exclude {
                        axis {
                            name 'BUILD_TYPE'
                            values 'preprod'
                        }
                        axis {
                            name 'FLAVOR'
                            values 'paid'
                        }
                    }
                }
                stages {
                    stage('Build') {
                        steps {
                            echo "Do Build for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                    stage('Distribute') {
                        steps {
                            echo "Do Test for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                }
            }
        }
    }
}

2. when expression:

This part is the one that provides dynamic decisions. Exclude cannot be dynamic but we can use expressions in when expression which provides dynamism. Suppose you select the build types and flavors from github labels or from Jenkins parameters. And you don’t want all combinations to be build. Then we can use when expressions for that. Matrix build will start Build and Distribute stages but they will be skipped if when expression is not true. Sample:

stages {
        stage('Lint'){}
        stage('Test'){}
        stage('SonarQube'){}
        stage('BuildAndDistribute') {
            matrix {
                when { allOf {
                    expression { appBuildVariants.containsKey(env.FLAVOR) }
                    expression { appBuildVariants[env.FLAVOR].contains(env.BUILD_TYPE)           }
                } }
                axes {
                    axis {
                        name 'BUILD_TYPE'
                        values 'preprod', 'staging', 'prod'
                    }
                    axis {
                        name 'FLAVOR'
                        values 'free', 'paid', 'enterprise'
                    }
                }
                excludes {
                    exclude {
                        axis {
                            name 'BUILD_TYPE'
                            values 'preprod'
                        }
                        axis {
                            name 'FLAVOR'
                            values 'paid'
                        }
                    }
                }
                stages {
                    stage('Build') {
                        steps {
                            echo "Do Build for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                    stage('Distribute') {
                        steps {
                            echo "Do Test for ${BUILD_TYPE} - ${FLAVOR}"
                        }
                    }
                }
            }
        }
    }
}

In this sample, appBuildVariants is a map which is created and calculated depending on some parameters. Suppose it contains [free] = {preprod,staging} then Jenkins will build and distribute for only freePreprod and freeStaging.

Downside of this process is that you always need to update axes if a new build type or flavor is added. I think this is a negligible downside because it can’t happen very frequently.

Warning:

When i used matrix build, i received a warning about method size is too much and Jenkins build failed. I solved it by moving the code to a helper groovy class and used it from Jenkins file.

Conlusion

In conclusion, matrix build speeds up builds a lot and builds are scalable with the help of matrix builds. Each build runs on a separate agent. Because of that, memory and cpu usage for an agent doesn’t increase too much. If we run all builds in the same agent, then a lot of resources of the agent is used and can cause some errors such as out of memory exception. If the axes were calculated dynamically, that would be great, but it is not allowed by Jenkins syntax. However, we have workarounds with exclude and when expressions.

If you found this helpful, feel free to clap, share, or leave a comment!