The Best SBOM Generation Tools Compared (& How to Pick the Right One)

Software supply chain security is a higher priority than ever, and software bills of materials (SBOMs) are a key tool organizations use to keep their applications safe.

A good software application is always changing: software companies are constantly fixing bugs, adding capabilities, and improving the user experience. In addition to these changes, the third-party libraries these companies incorporate in their applications are also constantly being updated to improve performance and eliminate known vulnerabilities—which means many software companies sell applications built on out-of-date third-party code.

With both proprietary code and components in such a state of flux, it’s important for companies to generate comprehensive and accurate SBOMs—whether it’s for themselves or for hopeful vendors.

In order to do this, organizations invest in tools that can provide SBOMs for the software they create as well as for the software they consume. As part of our ongoing effort to help people choose SBOM generators, we’ve analyzed some of the most well-known SBOM generators on the market today to see which ones lend the biggest hand to software providers.

What to look for in an SBOM generator

When it comes to choosing an SBOM generator tool, there are five things that you must take into consideration:

Can the tool generate an SBOM from your build environment? At the very least, an SBOM generator should be able to reconstruct your software application using your source code.

Can the tool generate an SBOM independent of source code? Ideally, your SBOM generator will allow you to create accurate SBOMs for potential suppliers and vendors. These are companies who likely won’t show you their source code, but should still be vetted for transitive vulnerabilities.

Can the tool generate an SBOM from Docker images? Similar to above, it’s helpful if your SBOM generator can create an SBOM simply by using a Docker Image.

Does the tool capture all the dependencies in the target library? Most tools are built on libraries that are built on other libraries. When generating an SBOM, you want to use a tool that shows exactly how these third-party sources stack up within each other.

Is the tool easy to deploy? Although it’s important to find a tool that performs the above three capabilities, it’s also important to make sure your SBOM generator is easy for your dev team to implement into the shipping cycle.

4 SBOM-generation tools side-by-side

There are many SBOM generators available today, but I chose to focus on four that I already know perform well, incorporate dependency information into the SBOM, and support multiple SBOM formats.

Those tools are CyclineDX, Sfyt (by Anchore), Fossa, and, of course, our own Finite State.

A few key notes before we dive in:

• Every one of these tools can generate SBOMs in both XML & JSON formats.
• This report is based on my use of these tools using available documentation. (For example, if Fossa can generate an SBOM from a Docker image, I haven’t found any reliable documentation about it.)
• In addition to these tools, Snyk’s AppSec software can also generate SBOMs—however, this is only available at the enterprise plan level, so we left Snyk out of this comparison round.

SBOM tool comparison methodology

I started by making a small Java Maven project—one that only contains a single import. The project has a small main class that imports the Twillio library (which has the log4j dependency). I built an identical project with Gradle so we can see if there are any discrepancies in these tools’ reports.

Second, in order to assess each tool’s ability to analyze projects whether or not the source code is available, I separated these projects into four folders:

• Gradle compiled
• Gradle uncompiled
• Maven compiled
• Maven uncompiled

Some of the tools we’re assessing generate SBOMs from Docker images. To test this, I also put together a small Docker image and manually added a single vulnerable “jsch-0.1.55.jar” library under /opt/jsch/. This gives us a chance to see how well these tools work when you’re adding software using Docker’s “COPY” command instead of adding it via the traditional package manager route.

The repository also includes a file named “dependencies.txt,” which contains all the Twillio dependencies according to deps.dev.

With these samples ready, I ran each SBOM generation tool through three exercises:

1. Generate SBOM from source code (the Log4J transitive project, pre-build).
2. Generate SBOM from a deployed system (again, the Log4J transitive project, but post-build).
3. Generate SBOM from a Docker image.

After this, I checked each tool’s report to see if they captured every dependency in the dependencies.txt list. This informed the “Accuracy” rating on the table above. Each tool caught every transitive dependency except lombok, which makes me think this is more likely a deps.dev mistake than a flaw in the tools.

For each tool, I made a note of how easy it was to deploy—as well as any additional comments. You can find my notes on each below.

CycloneDX

[Gradle plugin | Maven plugin]

CycloneDX founder Steve Springett is profoundly proficient with Maven and Java—at the time of running these tests, he had made 324 commits to this project (out of 403 total commits to the tool). The result is an SBOM that’s as good as SBOMs get. It’s a document that can be read by machines and humans alike. Bravo, CycloneDX!

If you’re building a Java/Maven project, I can’t recommend CycloneDX highly enough: I’d enable it and start including it in your build as soon as possible.

In order to use CycloneDX, you’ll need to use the plugin compatible with your build tool. I’ve listed my notes from this test for each plugin below. This is the main drawback that I found in terms of ease of deployment and use: you need to find and install a different plugin for each language and build in each project’s dependency file, which could get onerous with large projects.

CycloneDX (Gradle plugin) notes

Generating an SBOM without source code:

• You need to run gradle wrapper in order to run the ./gradlew cyclonedxBom command, but don’t run ./gradlew build.

Generating an SBOM from build (source code available):

• When running the tool in build, run ./gradlew build first, then gradle wrapper in order to run the ./gradlew cyclonedxBom command.

Deploying CycloneDX for SBOM generation:

• All you need to do is add the “org.cyclonedx.bom” dependency to the build.gradle file, which is also where you can configure how you want CycloneDX to generate the SBOM.

CycloneDX (Maven plugin) notes

Generating an SBOM without source code:

• Do NOT run mvn clean install . Instead, just run mvn org.cyclonedx:cyclonedx-maven-plugin:makeBom

Generating an SBOM from build (source code available):

• This is a nice touch: every time you run mvn clean install it creates both the JSON and the XML SBOMs in the target folder.

Deploying CycloneDX for SBOM generation:

• Just add the dependency to the pom.xml file and you’re good to go.

Additional comments:

• Based on our own Maven and software-composition expertise, here at Finite State, we recommend the following configuration when using cyclonedx-maven-plugin:

<includeCompileScope>true</includeCompileScope>                
<includeProvidedScope>false</includeProvidedScope>                
<includeRuntimeScope>true</includeRuntimeScope>                
<includeSystemScope>false</includeSystemScope>                
<includeTestScope>false</includeTestScope>

Syft (by Anchore)

For a free, open-source tool, Syft is an effective SBOM generator. Syft is very easy to operate and will do exactly what you tell it to do—which means you need to be specific when telling Syft what to scan. (Be careful not to accidentally tell it to scan your complete filesystem!)

While Syft provided, on the whole, accurate analyses, I did have a few qualms. For example, Syft did find every transitive dependency. However, it only did so for the post-build directory scan, and presented the report as a flat list rather than a series of interconnected relationships. In addition to this, Syft returned more than 830 spurious entries in the resulting JSON file. When I reviewed the JSON SBOM, I found a long string of this nonsense:

{
  "name": "syft:cpe23",
  "value": "cpe:2.3:a:apache_software_foundation:log4j:2.14.0:*:*:*:*:*:*:*"
},
{
  "name": "syft:cpe23",
  "value": "cpe:2.3:a:apache_software_foundation:api:2.14.0:*:*:*:*:*:*:*"
},
{
  "name": "syft:cpe23",
  "value": "cpe:2.3:a:Activator:log4j_api:2.14.0:*:*:*:*:*:*:*"
}

Aside from this, Syft works rather well at generating SBOMs.

Fossa

Fossa is an easy enough tool to use. However, the CLI command doesn’t generate SBOM files directly into your project—you’ll have to move it there yourself.

By default, Fossa generates SBOMs using SPDX’s plain-text format, rather than in XML or JSON. This makes the file less machine-readable—but the resulting text is not very human-friendly, either:

SPDXVersion: SPDX-2.1 DataLicense: CC0-1.0 SPDXID: SPDXRef-DOCUMENT Creator: Organization: FOSSA, INC. Created: 2022-08-09T10:45:17Z PackageName: Apache Log4j API PackageVersion: 2.14.0

Unfortunately, the plain-text SBOM file Fossa generated was not particularly useful. I would much rather get a file that easily maps packages to Maven-Central coordinates (e.g., groupId=**org.apache.logging** and artifactId=**log4j-api**), or even an SBOM that used PURL to help me see where packages are. This is more aligned with how developers think and talk about dependencies, which would make Fossa’s SBOMs much more useful.

In addition to these issues, the license information in Fossa’s SBOM output seemed nonsensical to me:

PackageLicenseDeclared: Apache-2.0
PackageCopyrightText: ownership.
PackageLicenseInfoFromFiles: Apache-2.0
PackageLicenseInfoFromFiles: MIT
PackageLicenseInfoFromFiles: Apache-1.1

I manually analyzed the “Log4J-API-2.14.0” package to verify these claimed licenses, and found absolutely no evidence suggesting that MIT or Apache-1.1 were appropriate here. Not only is this inaccurate, but such an error could cause bigger problems, as Apache-1.1 is famously incompatible with GPL software.

Finite State

Finite State can generate an SBOM with a click. And because the tool can analyze binary applications (including Java and .NET), it doesn’t matter if you have access to the source code—Finite State can do it on command, or as part of an automated workflow. (And every time you generate an SBOM in Finite State, it automatically produces an VEX report.)

We built Finite State to be the most comprehensive SCA tool when it comes to SBOM management. Not only can it generate SBOMs, but it can also import and scan SBOM files you receive from suppliers and vendors. Finite State quickly turns an SBOM into a full risk analysis, showing you:

1. Known vulnerabilities in your software supply chain, and where they are
2. Legal risks associated with the licenses you’re using
3. Technical debt in your software supply chain

This is helpful in that you can cut straight to the risk implications of your entire software supply chain. Finite State gives clear developer guidance, helping you prioritize issues and directing your team to available patches.

The best SBOM generator

If you’re working in a Maven or Gradle environment on a shoestring AppSec budget, CycloneDX is the way to go.

But if you want an easy-to-deploy, highly accurate SBOM that tells you where your risks lie (and gives you guidance on how to fix them), Finite State is the tool for you.

If you’d like to see how Finite State can help you secure your software supply chain, start a free trial today!