At work we have enhanced conan.io with an own python build system wrapper, which handles full dependency chains, forward and backward, and is capable of generating Jenkins pipelines for building in the Jenkins CI system.

Such a build is quite simple, the Jenkinsfiles just needs two lines for bootstrap out of a jenkins pipeline library and then the CI build itself in it’s standard incarnation has 4 build phases:

1. Generation phase, here the build is bootstrapped, the pipeline is generated, loaded and executed
2. Prebuild phase, things to be done before the real build (reporting, or other stuff)
3. The build itself, parallel execution for all existing targets and variants, plus test execution, static code analysis etc.
4. Postbuild phase, reporting, metrics, Jira connectors, etc.

So such a generated pipeline can reach up to 1500 lines, depending on the project’s configuration, not a big deal typically.

Now we added the feature to build a complete dependency chain. Yes, you are reading right. If you have a Project A, B and C and the dependency chain is like A -> B -> C (-> means depends on) then if you build C the result is taken to build B and the result is taken again to build A. Still a rather simple thing, but can be a quite complex and huge pipeline to run that.

Method size

And there we hit our first problem, still a simple one. Executing that gives back an exception with the message “Method code too large!” coming from the underlying JVM, which allows a Method code size up to 64k.

Weeell, not that big deal, some small changes to the generation and every target build had it’s own function, problem solved.

Class size

Okay, fine. Then we got that one new huge project with something like 30 components or something. Normal build is fine, works good so far. And then someone tried to run a dependency build. BOOM!

Now we got an exception saying “Class size too large!”, seriously what the fuck?

Now it’s okay that a pipeline/Groovy script with 52.5k lines of code may be a bit oversized (1.6MB), but why the fuck is there a limit to the the class size? (We wondered already at the method code limit…)

Okay first step was to use generateable reusable methods in the pipeline, which reduced the size of that pipeline already to 22.7k lines of code – still too large for a class. (Yes I know that LOC are not the same as the bytesize of a class or method, but it is at least some indicator for the size of it)

What now? Splitting to multiple loadable groovy classes of course. Said and done, every build step is now it’s own little groovy file, 438 files to be exact. In the main pipeline script we generate now a map with an entry for each file and are loading the classes dynamically into that map.

Now guess what…

General error when generating a class: ArrayIndexOutOfBoundsException

That’s now a show stopper. Now we are trying to find out where this comes from and guess what the internet spats out about that error?

Nothing or just shit/brabble/rubbish.

[Update:]

A solution is of course to make the generation more intelligent and also the generated result more intelligent, which also brings a much higher complexity. But after all it is the best solution right now as long as you generate everything of a huge Jenkins pipeline.