Build Updates​

Step Resources Override​

You can now define stepResources in a Build or BuildRun to override the resource requests and limits of individual steps defined in a BuildStrategy or ClusterBuildStrategy. This gives you fine-grained control over CPU and memory allocation without having to modify the strategy itself.

RuntimeClass Support​

A new runtimeClassName field has been added to both Build and BuildRun resources. This allows builds to leverage alternative container runtimes available on your cluster, enabling use cases such as confidential computing or other runtime-specific capabilities.

PipelineRun Execution Mode​

This release introduces a foundational PipelineRun execution mode as an alternative to the existing TaskRun-based execution. This enables multi-pod builds using PVC-based workspace storage, laying the groundwork for more flexible build execution strategies in future releases.

Infrastructure and Dependency Updates​

• Updated to Tekton LTS v1.9.1
• Upgraded base image to Red Hat UBI 10
• Enhanced vulnerability scanning now uses both Grype and Trivy
• The minimum supported Kubernetes version is now v1.33.0
• The minimum supported Tekton version is now v1.0.0 (Tekton LTS v1.9.1 is recommended)

Bug Fixes​

• Fixed Tekton manifest URLs to point to infra.tekton.dev
• Fixed the Ko build strategy to correctly handle Go modules with dependencies but no vendoring

Check out the full release notes on GitHub for further details.

CLI Updates​

RuntimeClass Flag​

The shp build create, shp build run and shp buildrun create commands now supports a --runtime-class flag, allowing you to specify a RuntimeClass for the build pod directly from the CLI:

shp buildrun create my-build --runtime-class kata-containers

Package Extensibility​

This release also introduces initialization options that make it easier to import the CLI as a Go package and extend its functionality programmatically.

See the CLI release notes on GitHub for further details.

Installing Shipwright​

Build​

1. Install Tekton LTS v1.9.1:

   kubectl apply --filename https://infra.tekton.dev/tekton-releases/pipeline/previous/v1.9.1/release.yaml
   

2. Install v0.19.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.19.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.19.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.19.0/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.19.0/shp_0.19.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.19.0/shp_0.19.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.19.0/shp_0.19.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

The operator release for v0.19.0 is forthcoming. Stay tuned to the operator releases page for updates.

Build Updates​

In this release, we have made changes under the covers to run containers as much as possible with read-only root file system. Note that those changes are only active as long as you do not overwrite the step templates for out-of-the box containers in the configuration. If you had made customizations there, make sure to merge your changes with our extensions.

Beside that, we have updated Build to utilize the Tekton v1.6 APIs. The minimum supported Kubernetes version is now v1.32.0. The minimum supported Tekton version is now v0.68.0.

Check out the full release notes on GitHub of the v0.18.0 release for further details. Note that there is also already a v0.18.1 release that is a rebuild with the new Go version released recently that fixed vulnerabilities

CLI Maintenance Update​

The CLI has no new features but updated dependencies.

See the CLI release notes on GitHub

Operator Update​

The operator was upgraded to deploy Shipwright Build v0.18.0 components.

In our definition, we have removed the dependency on the cert-manager and Tekton Pipelines. The reason we looked at this was that cert-manager deprecated their operator and does not provide updates anymore. For cert-manager, you now have to use another installation method. See their installation instructions for your options.

To stay consistent, we also removed our other dependency on Tekton Pipelines. You may still deploy Tekton Pipelines using the operator, but you can now also chose a different installation method.

See the operator release notes on GitHub for details.

Installing Shipwright​

Build​

1. Install Tekton v1.6.0:

   kubectl apply --filename https://infra.tekton.dev/tekton-releases/pipeline/previous/v1.6.0/release.yaml
   

2. Install v0.18.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.18.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.18.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.18.0/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.18.0/shp_0.18.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.18.0/shp_0.18.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.18.0/shp_0.18.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.18.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

Build Updates​

In this release, we introduced a significant change to the status API for BuildRun objects. Status will now include an executor field indicating the name and kind of object used to execute the build. The taskRunName status field is officially deprecated, but will remain populated. In the future, this API change will let us use other Kubernetes objects to manage the build execution, such as Tekton PipelineRuns, Argo Workflows, and even plain Kubernetes Pods.

kind: BuildRun
metadata:
  name: buildah-build-x53sd
spec:
  ...
status:
  ...
  executor:
    kind: TaskRun
    name: buildah-build-x53sd-taskrun-43z3f

We also fixed the volume API for Build and BuildRun objects, where empty strings were incorrectly required for VolumeSource fields. This is now marked optional and can be left empty.

The Build project updated is containers to use UBI 10. This ensures we use the latest set of RHEL features in our builds, particularly for dependencies like git.

Finally, we have updated Build to compile with Go 1.24 and utilize the Tekton v1.3 APIs. With this change, the minimum supported Kubernetes version is now v1.31.0. The minimum supported Tekton version is now v0.65.0.

Check out the full release notes on GitHub for further details.

CLI Updates​

The CLI adds new commands to list and delete build strategies. To view the build strategies available on the cluster, run the following command:

shp clusterbuildstrategy list

We also added a --wide option when listing BuildRuns. This lets you see expanded information when viewing BuildRuns in the terminal:

$ shp buildrun list --wide
NAME                                    STATUS          AGE     SOURCE                                                  OUTPUT-IMAGE                                    BUILD-NAME              ELAPSED-TIME    SOURCE-ORIGIN
buildpack-nodejs-buildrun-bsnzp         Succeeded       58m     https://github.com/shipwright-io/sample-nodejs@main     quay.io/satyam16/sample-nodejs:latest           buildpack-nodejs-build  2m35s           Git
buildpack-nodejs-buildrun-gb79r         Succeeded       6d      https://github.com/shipwright-io/sample-nodejs          quay.io/satyam16/sample-nodejs:latest           buildpack-nodejs-build  1m26s           Git

Finally, we fixed two important bugs related to the streaming of local source code to Shipwright. The command line should no longer panic if the referenced Build object does not have source information set, nor should the build fail due to file permission issues on the uploaded source code.

See the CLI release notes on GitHub

Operator Updates​

The operator was upgraded to deploy Shipwright Build v0.17.0 components. It also fixes a bug where the operator reported itself "Ready" when the dependent Tekton Pipeline components were not ready.

See the operator release notes on GitHub for details.

Key Features​

Below are the key features in this release:

Improving Git Clone​

The source.git.depth was added to the Build resource to specify the depth of the Git history. If not specified the default value is 1 which means that no history is cloned at all, being the fastest way to clone a Git repository. Any value greater than 1 will create a clone with the specified depth. For a full git history clone, depth must be set to 0.

Optimizing Controller Memory Footprint​

Improve the Shipwright controller's memory efficiency by restricting the manager cache to only TaskRuns and Pods associated with the controller. This is achieved by leveraging the buildrun.shipwright.io/name label to identify relevant resources. By narrowing the scope of what the manager caches, we significantly reduce memory usage.

CLI: Maintenance Update​

The CLI was updated to support Build v0.16.0 APIs. Behind the scenes, this means it now uses the v1beta1 API instead of v1alpha1. As a result, CLI operations are slightly faster, since they no longer go through the conversion webhook.

Additionally, two new flags have been introduced to support node scheduling and node selection capabilities:

• --node-selector: Sets the .spec.nodeSelector field for Build and BuildRun resources during creation.
• --scheduler-name: Sets the .spec.schedulerName field for Build and BuildRun during both creation and execution.

Operator: Builds Upgrade​

The operator was updated to deploy Builds v0.16.0.

Installing Shipwright​

Build​

1. Install Tekton v1.0.0:

   kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v1.0.0/release.yaml
   

2. Install v0.16.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.16.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.16.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.16.0/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.16.0/shp_0.16.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.16.0/shp_0.16.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.16.0/shp_0.16.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.16.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

A new set of build scheduling features introduced in v0.15 allows users to specify node selectors, custom schedulers, and tolerations for builds.

These make it easier to schedule builds on clusters with nodes of multiple CPU architectures, use a scheduler that is tuned to a certain workflow, or just more general control of which nodes builds run on.

CLI flags to use build scheduler features​

With these features also comes new CLI flags (introduced in v0.16) that allow specifying nodeSelectors and custom schedulers on the command line when using shp with Builds or BuildRuns.

In the following commands, --node-selector and --scheduler-name sets these fields on the Build or BuildRun objects:

• shp build create
• shp build run
• shp build upload
• shp buildrun create

Example with shp build run and --node-selector​

We can specify a build to be scheduled to a node with certain labels by using a node selector. Here, we'll schedule a build to a node with an ARM CPU architecture. Starting with an example build:

$ shp build create test-golang-build \
  --output-image=kind.local/test/test-golang-build \
  --source-git-url=https://github.com/shipwright-io/sample-go \
  --source-context-dir=docker-build \
  --strategy-name=buildah-shipwright-managed-push

We'll run the build and specify the arm64 architecture in the node selector flag:

$ shp build run test-golang-build --node-selector=kubernetes.io/arch=arm64

and see that the created BuildRun now has a nodeSelector specified and has been scheduled on the arm64 node:

apiVersion: shipwright.io/v1beta1
kind: BuildRun
metadata:
  creationTimestamp: "2025-05-20T17:27:01Z"
  generateName: test-golang-build-
  generation: 1
  labels:
    build.shipwright.io/generation: "1"
    build.shipwright.io/name: test-golang-build
  name: test-golang-build-glghq
  namespace: default
  resourceVersion: "31512392"
  uid: 19f02c99-1d24-44c0-a5f9-ab544fdf55ae
spec:
  build:
    name: test-golang-build
  nodeSelector:
    kubernetes.io/arch: arm64

$ kubectl get pod test-golang-build-glghq-tfhnn-pod -o jsonpath='{.spec.nodeName} {.spec.nodeSelector}'

arm-node.compute.internal {"kubernetes.io/arch":"arm64"}

$ kubectl get node arm-node.compute.internal -o jsonpath='{.metadata.labels.kubernetes\.io/arch}'

arm64

Example with shp build create and --scheduler-name​

This time we'll specify a scheduler name test-scheduler that is assumed to be a custom scheduler that has been deployed to the cluster already.

Instead of specifying these options when running the build, we can also specify them when creating the build:

$ shp build create test-golang-build \
  --output-image=kind.local/test/test-golang-build \
  --source-git-url=https://github.com/shipwright-io/sample-go \
  --source-context-dir=docker-build \
  --strategy-name=buildah-shipwright-managed-push \
  --scheduler-name=test-scheduler

and see that the scheduler name appears in the Build yaml:

apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  creationTimestamp: "2025-05-20T17:49:07Z"
  generation: 1
  name: test-golang-build
  namespace: default
  resourceVersion: "31518385"
  uid: bd2333fb-71eb-4228-bc5f-5a466032fcc5
spec:
  output:
    image: kind.local/test/test-golang-build
  schedulerName: test-scheduler
  source:
    contextDir: docker-build
    git:
      url: https://github.com/shipwright-io/sample-go
    type: Git
  strategy:
    kind: ClusterBuildStrategy
    name: buildah-shipwright-managed-push
status:
  message: all validations succeeded
  reason: Succeeded
  registered: "True"

After running the build with shp build run test-golang-build, we see that the schedulerName got picked up by the build pod:

$ kubectl get pods test-golang-build-j9vbd-qr9cr-pod -o jsonpath='{.spec.schedulerName}'

test-scheduler

Example with setting tolerations on a BuildRun​

We'll start with the same example build as above:

$ shp build create test-golang-build \
  --output-image=kind.local/test/test-golang-build \
  --source-git-url=https://github.com/shipwright-io/sample-go \
  --source-context-dir=docker-build \
  --strategy-name=buildah-shipwright-managed-push

apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  creationTimestamp: "2025-05-20T18:29:43Z"
  generation: 1
  name: test-golang-build
  namespace: default
  resourceVersion: "31529978"
  uid: f393de53-e389-4e25-a29a-a434505bd82e
spec:
  output:
    image: kind.local/test/test-golang-build
  source:
    contextDir: docker-build
    git:
      url: https://github.com/shipwright-io/sample-go
    type: Git
  strategy:
    kind: ClusterBuildStrategy
    name: buildah-shipwright-managed-push

In this example we have a three node cluster, so let's taint all of the nodes with test-key and test-value. This will prevent any pod from scheduling on these nodes unless it tolerates this taint:

$ kubectl get nodes -o name
node/test-node-1
node/test-node-2
node/test-node-3

$ kubectl taint nodes test-node-1 test-key=test-value:NoSchedule
node/test-node-1 tainted
$ kubectl taint nodes test-node-2 test-key=test-value:NoSchedule
node/test-node-2 tainted
$ kubectl taint nodes test-node-3 test-key=test-value:NoSchedule
node/test-node-3 tainted

We see that if we create a BuildRun now, it will fail to schedule:

$ shp build run test-golang-build

$ kubectl get events
LAST SEEN   TYPE      REASON             OBJECT                                  MESSAGE
...
2m54s       Normal    Pending            taskrun/test-golang-build-bqw9s-hbkds   pod status "PodScheduled":"False"; message: "0/3 nodes are available: 3 node(s) had untolerated taint {test-key: test-value}. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling."
...

Let's patch the Build with the following toleration in order to have it tolerate the node taint:

tolerations:
  - key: "test-key"
    operator: "Equal"
    value: "test-value"
    effect: "NoSchedule"

$ kubectl patch Build test-golang-build --type=merge -p '{"spec":{"tolerations":[{"key":"test-key","operator":"Equal","value":"test-value"}]}}'

$ shp build run test-golang-build

Now we see that the build is successfully scheduled:

$ kubectl get events
LAST SEEN   TYPE      REASON             OBJECT                                  MESSAGE
...
109s        Normal    Scheduled          pod/test-golang-build-k7zc4-492d2-pod   Successfully assigned default/test-golang-build-k7zc4-492d2-pod to ip-10-0-2-69.us-east-2.compute.internal
...

Conclusion​

Shipwright's new build scheduling options make it much easier to control where builds get placed in the cluster and give more options to those who are operating in multi-arch environments or with other constraints.

Key Features​

Below are the key features in this release:

Build: More Control for Node Scheduling​

Builds v0.15 adds additional support for controlling which nodes a build can run on. In addition to specifying a node selector (introduced in v0.14), builds can now tolerate node taints and instruct Kubernetes to use a custom pod scheduler. The latter feature can be used with new projects like Volcano, which optimizes pod scheduling for batch workloads.

CLI: Maintenance Update​

The CLI was updated to support Build v0.15.0 APIs.

Operator: Builds Upgrade​

The operator was updated to deploy Builds v0.15.2.

Installing Shipwright​

Build​

1. Install Tekton v0.68.0:

   kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.68.0/release.yaml
   

2. Install v0.15.2 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.15.2/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.15.2/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.15.2/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.15.0/shp_0.15.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.15.0/shp_0.15.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.15.0/shp_0.15.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.15.2 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

What About That .z?​

Since v0.14.0 was released, we have done a lot of work behind the scenes to automate Shipwright's release process and security posture. Part of this includes a set of nightly GitHub Actions that scan our container images for vulnerabilities at the code and operating system level. This process covers our most recent release as well as the nightly builds that come out of the main branch.

Less than a day after Builds v0.15.0 was released, vulnerabilties in the golang.org/x/crypto and golang.org/x/oauth2 packages were disclosed. These were picked up by our nightly automation, which filed a GitHub issue notifying the community of the problem. The maintainers quickly sprung into action, submitting pull requests to patch the vulnerable code. The next night our automation detected these vulnerabilities were fixed, and drafted a security patch release.

Two days later, we patched the Build project all over again. All this happened as the cli and operator projects were preparing releases of their own.

Special thank you to @SaschaSchwarze0 for not only fixing these vulnerabilties, but also building much of the workflows that automate these security updates. Bravo!

We are happy to announce the v0.14.0 release of Shipwright. This is our first release since we have joined the Cloud Native Computing Foundation (CNCF) as a sandbox project.

In this release, we have put together some nice features:

Features​

Vulnerability Scanning​

Keeping your environments secure is key these days. For container images, scanning them is widely adopted. Shipwright now performs a shift left of those scans by incorporating image scanning into the image build itself. We'll ensure that a vulnerable image never makes it into your container registry (though, you'd still have to re-scan it regularly to determine when it becomes vulnerable). This is a great safeguard for example against base images you consume in your Dockerfile that suddenly are not updated anymore.

You can read more about it in the separate blog post Building Secure Container images with Shipwright.

Parameters in the CLI​

The Shipwright CLI finally received the first support for build parameters. You can use the --param-value argument to provide values for strategy parameters such as the Go version and Go flags in our ko sample build strategy like this: shp build create my-app --param-value go-version=1.23 --param-value go-flags=-mod=vendor.

The smaller but still nice things​

Often the small changes are what help you, here are some:

• If a step in your BuildRun goes out of memory, then it is now easier to determine that as the BuildRun status will have StepOutOfMemory as reason.
• A new sample build strategy has been added which orchestrates a multi-arch build using Kubernetes Jobs. See our documentation for more information.
• We started to implement node selection properties on Builds and BuildRuns with node selector support. Tolerations are planned to be added in v0.15.
• As usual, we have done our due diligence. DependaBot helped us to keep our dependencies secure. We are now building with Go 1.22. Kubernetes and Tekton dependencies have been updated. We are also in the process of establishing automation across our repositories that gives us a GitHub issue once our latest release becomes vulnerable. You'll probably see more patch releases in the future where we keep our release free of vulnerabilities.

Installing Shipwright​

Build​

1. Install Tekton v0.65.1:

   kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.65.1/release.yaml
   

2. Install v0.14.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.14.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.14.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.14.0/sample-strategies.yaml --server-side
   

If you are a long-standing Shipwright user that started to use us on our Alpha API (before v0.13.0), then we recommend you to run a storage version migration. It will update the stored version of all Shipwright resources in your cluster to the Beta API to unnecessary invocations of our conversion webhook in the future.

curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.14.0/hack/storage-version-migration.sh | bash

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.14.0/cli_0.14.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.14.0/cli_0.14.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.14.0/cli_0.14.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.14.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

In the modern software development era, containers have become an essential tool for developers. They offer a consistent environment for applications to run, making it easier to develop, test, and deploy software across different platforms. However, like any other technology, containers are not immune to security vulnerabilities. This is where vulnerability scanning for container images becomes crucial. In this blog, we will discuss how to run vulnerability scanning on container images with Shipwright while building those images.

Before jumping into this feature, let's explain what Shipwright is and why vulnerability scanning is important.

What is Shipwright​

Shipwright is an open-source framework designed to facilitate the building of container images directly within Kubernetes environments. It aims to streamline the development and deployment process by providing a native Kubernetes solution for creating container images from source code. Shipwright supports multiple build strategies and tools, such as Kaniko, Paketo Buildpacks, Ko, Buildkit and Buildah, providing flexibility and extensibility to meet various application needs. This Kubernetes-native solution helps ensure that container images are built efficiently and securely, leveraging the strengths of the Kubernetes ecosystem.

Shipwright consists of four core components:

1. Build - defines what source code are you trying to build from, and where the resulting container image should be publish(the what and where?).
2. BuildRun - defines the when to trigger the building mechanism, telling the Kubernetes cluster when to build your application(the when?).
3. BuildStrategy and ClusterBuildStrategy - defines how your application is assembled and which build tool to use(how to build?).

You can learn more by visiting this link.

Why is Vulnerability Scanning Important?​

Vulnerability scanning for container images involves examining the image for known security vulnerabilities. This is typically done using automated tools that compare the contents of the image against a database of known vulnerabilities. The key reasons for Vulnerability Scanning are:

• Security: Containers often include third-party libraries and dependencies, which might have known vulnerabilities. If these vulnerabilities are exploited, they can lead to data breaches, unauthorized access, and other security incidents.
• Compliance: Many industries have regulatory requirements that mandate regular security assessments, including vulnerability scanning. Ensuring your container images are free from known vulnerabilities helps in meeting these compliance standards.
• Stability: Vulnerabilities can also impact the stability and performance of your applications. By identifying and fixing these issues early, you can maintain the reliability of your software.

There are many popular tools available for vulnerability scanning of container images, such as Clair, Trivy, Aqua Security, and Snyk.

In Shipwright, we use Trivy under the covers for vulnerability scanning, our rational for choosing this tool can be found in our SHIP-0033.

Vulnerability Scanning in Shipwright​

Before we dive in how it works, lets explore the features offered by Shipwright for vulnerability scanning of container builds :

spec:
  output:
    vulnerabilityScan:
      enabled: true
      failOnFinding: true  #image won't be push to registry if set to true
      ignore:
        issues:
          - CVE-2022-12345 #specify list of cve to be ignored
        severity: Low | Medium | High | Critical
        unfixed: true #ignores the unfixed vulnerabilities

Configuration Options

• vulnerabilityScan.enabled: Specify whether to run vulnerability scan for image. The supported values are true and false.
• vulnerabilityScan.failOnFinding: Indicates whether to fail the build run if the vulnerability scan results in vulnerabilities. The supported values are true and false. This field is optional and false by default.
• vulnerabilityScan.ignore.issues: References the security issues to be ignored in vulnerability scan
• vulnerabilityScan.ignore.severity: Denotes the severity levels of security issues to be ignored, valid values are:
  • low: it will exclude low severity vulnerabilities, displaying only medium, high and critical vulnerabilities.
  • medium: it will exclude low and medium severity vulnerabilities, displaying only high and critical vulnerabilities.
  • high: it will exclude low, medium and high severity vulnerabilities, displaying only the critical vulnerabilities.
• vulnerabilityScan.ignore.unfixed: Indicates to ignore vulnerabilities for which no fix exists. The supported types are true and false.

Lets dive right in​

Now, let's see vulnerability scanning for a container image with Shipwright in action. If you want to try out in kind cluster, follow the steps from this section until you create the push secret. As a next step, create a build object with vulnerability scanning enabled, replacing REGISTRY_ORG with the registry username your push-secret have access to:

REGISTRY_ORG=<your_registry_org>
cat <<EOF | kubectl apply -f -
apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  name: buildah-golang-build
spec:
  source:
    type: Git
    git:
      url: https://github.com/shipwright-io/sample-go
    contextDir: docker-build
  strategy:
    name: buildah-shipwright-managed-push
    kind: ClusterBuildStrategy
  paramValues:
  - name: dockerfile
    value: Dockerfile
  output:
    image: docker.io/${REGISTRY_ORG}/sample-golang:latest
    pushSecret: push-secret
    vulnerabilityScan:
      enabled: true
      failOnFinding: true # if set to true, then the image won't be pushed to the registry
EOF

To view the Build which you just created:

$ kubectl get builds

NAME                   REGISTERED   REASON      BUILDSTRATEGYKIND      BUILDSTRATEGYNAME                 CREATIONTIME
buildah-golang-build   True         Succeeded   ClusterBuildStrategy   buildah-shipwright-managed-push   72s

Now submit your buildrun

cat <<EOF | kubectl create -f -
apiVersion: shipwright.io/v1beta1
kind: BuildRun
metadata:
  generateName: buildah-golang-buildrun-
spec:
  build:
    name: buildah-golang-build
EOF

Wait until your BuildRun is completed and then you can view it as follows:

$ kubectl get buildruns

NAME                            SUCCEEDED   REASON                 STARTTIME   COMPLETIONTIME
buildah-golang-buildrun-s9gsh   False       VulnerabilitiesFound   2m54s       98s

Here, you can see that the buildrun failed with reason VulnerabilitiesFound and it will not push the image to the registry as the failOnFinding option is set to true.

And one can find the list of vulnerabilities in the build run under the .status.output path:

apiVersion: shipwright.io/v1beta1
kind: BuildRun
metadata:
  creationTimestamp: "2024-07-08T08:03:18Z"
  generateName: buildah-golang-buildrun-
  generation: 1
  labels:
    build.shipwright.io/generation: "1"
    build.shipwright.io/name: buildah-golang-build
  name: buildah-golang-buildrun-s9gsh
  namespace: default
  resourceVersion: "19926"
  uid: f3c558e9-e027-4f59-9fdc-438a31c6de11
spec:
  build:
    name: buildah-golang-build
status:
  buildSpec:
    output:
      image: docker.io/karanjmu92/sample-golang:latest
      pushSecret: push-secret
      vulnerabilityScan:
        enabled: true
        failOnFinding: true
    paramValues:
    - name: dockerfile
      value: Dockerfile
    source:
      contextDir: docker-build
      git:
        url: https://github.com/shipwright-io/sample-go
      type: Git
    strategy:
      kind: ClusterBuildStrategy
      name: buildah-shipwright-managed-push
  completionTime: "2024-07-08T08:04:34Z"
  conditions:
  - lastTransitionTime: "2024-07-08T08:04:34Z"
    message: Vulnerabilities have been found in the image which can be seen in the
      buildrun status. For detailed information,see kubectl --namespace default logs
      buildah-golang-buildrun-s9gsh-w488m-pod --container=step-image-processing
    reason: VulnerabilitiesFound
    status: "False"
    type: Succeeded
  failureDetails:
    location:
      container: step-image-processing
      pod: buildah-golang-buildrun-s9gsh-w488m-pod
  output:
    vulnerabilities:
    - id: CVE-2023-24538
      severity: critical
    - id: CVE-2023-24540
      severity: critical
          .
          .
          .
    - id: CVE-2024-24791
      severity: medium
  source:
    git:
      branchName: main
      commitAuthor: OpenShift Merge Robot
      commitSha: 96afb4108fba22e91f42168d8babb5562ac8e5bb
    timestamp: "2023-08-10T15:24:45Z"
  startTime: "2024-07-08T08:03:18Z"
  taskRunName: buildah-golang-buildrun-s9gsh-w488m

Conclusion​

Shipwright offers a robust and flexible solution for building container images within Kubernetes environments. By integrating vulnerability scanning directly into the build process, Shipwright ensures that container images are secure and compliant with Industry Standards and gets closer to Supply Chain Security Best Practices.

After months of diligent work, just in time for cdCon 2024, we are releasing our v0.13.0 release. This significant milestone incorporates a bunch of enhancements, features and bug fixes. Here are the key highlights:

Action Required​

After upgrading from v0.12.0 to v0.13.0, you can run the following two commands to remove unnecessary permissions on the shipwright-build-webhook: kubectl delete crb shipwright-build-webhook && kubectl delete cr shipwright-build-webhook

We switched our CRDs storage version to v1beta1. We strongly advise users to migrate to our v1beta1 API, as we intend to deprecate v1alpha1 in a future release. Note that the v1alpha1 still served for all CRD's.

Features​

Build Output Timestamp​

SHIP 0037 is now implemented. Users can make use of the Build .spec.output.timestamp to explicitly set the resulting container image timestamp.

Tekton v1 API Adoption​

Our Shipwright controllers now use the Tekton v1 API when working with TaskRuns. See PR.

API Changes​

Switch Storage Version to v1beta1​

The storage version for all CRD's has been switched to v1beta1. Concurrently, our Shipwright controllers have been updated to use this updated API version. The v1alpha1 version continues to be supported.

Beta API Changes​

We've implemented minor adjustments to the existing v1beta1 API for the sake of consistency, incorporating feedback gathered from the v0.12.0 release. These are:

• Renaming certain Go Types, such as build.Spec.Source.GitSource to build.Spec.Source.Git, aligning them with their respective JSON tags.
• Designating build.Spec.Source.Type and buildRun.Spec.Source.Type as mandatory, addressing usability concerns raised by users.
• We made build.Spec.Source optional. Enabling the definition of a Build without any source. This feature proves valuable particularly when executing a Build only with local source.
• Requiring build.spec.source.git.url upon the specification of build.spec.source.git.

Further details can be found in PR 1504 and PR 1441.

Bugs​

Conversion Webhook​

As the usage of the webhook increased over the last months, we've made enhancements to address certain gaps:

• Previously, there was an issue with the proper conversion of BuildRuns from v1alpha1 to v1beta1 when a generated service account was utilized. This has been resolved.
• Additionally, a bug was identified where patching a completed v1beta1 BuildRun would inadvertently remove its status. This issue has been resolved.

OCIArtifacts​

When implementing prevention measures against path traversal during the extraction of an OCI artifact, we were too strict. We only needed to prevent /../ because this means to go one directory up. We still must allow .. because a directory or file can contain two subsequent dots in its name. You can now use files and directories with two subsequent dots in its name when using an OCI artifact as source.

Builds​

When a Build with an unknown strategy kind is defined, the Build validation triggers. However, it was failing to update the Build status to Failed, resulting in an endless loop during reconciliation. This issue has been resolved.

Miscellaneous​

Dependabot updates play a crucial role in keeping our go dependencies current with CVE's. Building upon this, we've recently implemented a similar automation to streamline the process of updating our CI Github actions, see PR 1516.

Furthermore, we have been consistently updating all of our Build tools, such as ko and buildpacks, using our custom automation, ensuring that our Strategies remain on the latest versions.

It's worth noting that our minimum supported Kubernetes version is now 1.27, while the minimum Tekton version is 0.50.*.

Additionally, Shipwright Build is now compiled with Go 1.21

Docs​

Our general Roadmap and Adopters doc has been updated.

Installing Shipwright​

Build​

1. Install Tekton v0.50.5:

   kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.50.5/release.yaml
   

2. Install v0.13.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.13.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.13.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.13.0/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.13.0/cli_0.13.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.13.0/cli_0.13.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.13.0/cli_0.13.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.13.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

Today, as part of our release v0.12.0, we are introducing our beta API. The beta API brings multiple changes as a result of accumulated experience operating the alpha API and incorporating valuable user feedback.

With the introduction of the beta API, users can have confidence that our core components have been battle-tested, and using our different features is considered a safe practice.

We want to thank our community for their contributions and support in redefining this new API!

Beta API​

The beta API is available starting from the v0.12.0 release. The release is available across our cli, operator and build repository.

Within the v0.12.0 release, a conversion webhook has been introduced to ensure backward compatibility between the v1alpha1 and v1beta1.

We strongly encourage both current and future users to adopt the beta API to benefit from its enhanced definition.

Migration guidelines​

Deprecated fields​

Changes to Build API fields​

See this example of the Git source type:

# v1alpha1
---
apiVersion: shipwright.io/v1alpha1
kind: Build
metadata:
  name: a-build
spec:
  source:
    url: https://github.com/shipwright-io/sample-go
    contextDir: docker-build
  strategy:
    name: buildkit
    kind: ClusterBuildStrategy
  output:
    image: an-image

# v1beta1
---
apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  name: a-build
spec:
  source:
    type: Git
    git: 
      url: https://github.com/shipwright-io/sample-go
    contextDir: docker-build
  strategy:
    name: buildkit
    kind: ClusterBuildStrategy
  output:
    image: an-image

Changes to BuildRun API fields​

Note: generated service accounts is a deprecated feature, and may be removed in a future release.

See example:

# v1alpha1
---
apiVersion: shipwright.io/v1alpha1
kind: BuildRun
metadata:
  name: a-buildrun
spec:
  buildRef:
    name: a-build
  serviceAccount:
    generate: true


# v1beta1
---
apiVersion: shipwright.io/v1beta1
kind: BuildRun
metadata:
  name: a-buildrun
spec:
  build:
    name: a-build
  serviceAccount: ".generate"

Changes to Strategies API fields​

References​

For more information, see SHIP 0035.

Some key points to consider:

• Starting with the v0.12.0 release, a conversion webhook is deployed to provide support for both v1alpha1 and v1beta1 API versions.
• Users are encouraged to adopt the v1beta1 API.
• Support for v1alpha1 will continue for some additional releases. Part of the v1alpha1 API is already deprecated and not available in v1beta1.

Please take a look at the following blog post to see some of our guidelines on moving your Shipwright Custom Resources from alpha to beta.

Installing Shipwright​

Build​

1. Install Tekton v0.47.4:

   kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.47.4/release.yaml
   

2. Install v0.12.0 using the release YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.12.0/release.yaml --server-side
   
   curl --silent --location https://raw.githubusercontent.com/shipwright-io/build/v0.12.0/hack/setup-webhook-cert.sh | bash
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply --filename https://github.com/shipwright-io/build/releases/download/v0.12.0/sample-strategies.yaml --server-side
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.12.0/cli_0.12.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.12.0/cli_0.12.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.12.0/cli_0.12.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first ensure the operator v0.12.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

Join us this October to contribute to Shipwright and earn your Hacktoberfest rewards. Check out our issues labeled hacktoberfest to get started.

What Should Shipwright Become?​

We concluded that Shipwright is and should remain a framework for building container images. Shipwright will continue to make it simple to build a container image from source, using tools that are actively maintained by a community of experts. Our separation of build strategy from build definition and execution will remain a cornerstone of the Shipwright framework.

However, we realized that building the image is just the starting point to delivering software on the cloud. Software supply chain security is a topmost concern of teams large and small. Artifacts like image scans, signatures, software bill of materials, and provenance are needed to build modern software for the cloud. Shipwright can, and should, rise up to meet these demands.

We also decided that Shipwright will continue to run on cloud-native infrastructure, powered by Kubernetes and Tekton. We can go further, though, and plug Shipwright into the vast cloud-native ecosystem, through integrations with CDEvents, ArgoCD, and more. Shipwright is just getting started in this effort through the Triggers and Image sub-projects.

Core Values​

Over the past two years, the Shipwright community has coalesced around three core values: simplicity, flexibility, and security.

Simplicity means that we provide an experience that is intuitive and consistent. It also means that we shouldn't be afraid to take an opinionated stance on common tasks, or features that we want to add to the project. We discovered in the Beta API workshop areas where our APIs were not consistent or intuitive, and we identified changes to fix these problem areas. These include single sources for builds and maintaining our opinionated steps to obtain source code.

Flexibility means that we provide space for teams to bend Shipwright to fit their needs. This started with the build strategy model itself, which we are keeping at the core of the API. We continued with the Parameters API, which provides avenues for customization between build strategies and build executions. We also took steps in our beta workshop to ensure our API is tool agnostic, such that we can help grow the ecosystem of build tools. This meant that some fields that were only used by specific build tools were dropped.

Lastly, Shipwright aims to meet the security needs for cloud-native applications. Security for Shipwright starts with the transparent pod security contexts built into the build strategy API. This encourages the continued evolution of tooling away from privileged and "root" containers, both of which are potential security risks. As a community, we have started experimenting with tools like Trivy to make the security of Shipwright-built images more transparent. We hope to continue these efforts with emerging software security tools in the future.

Bringing Shipwright to Beta​

Starting in version 0.12, Shipwright will introduce the beta Build API and begin phasing out the current alpha API. We encourage current and future users to provide feedback as we roll this new API out. You can provide feedback by filing an issue on GitHub, sending an email to our mailing list, or posting a message to the #shipwright channel on Kubernetes Slack. We look forward to hearing from you!

Join us this October to contribute to Shipwright and earn your Hacktoberfest rewards. Check out our issues labeled hacktoberfest to get started.

• We updated dependencies to mitigate security vulnerabilities.
• We updated our images to be based on the new Red Hat Universal Base Image 9 Minimal.
• We updated build strategy tools as usual. This includes a fix that re-enabled the ko build strategy. Congratulations to them for moving into the own organization. It is a great build tool that we do not just provide a sample for, but also use in our own builds.

Behind the scenes we are working on streamlining our API objects. As a first step, we deprecate the following fields:

• In Builds, spec.sources is deprecated. We will consolidate supporting a single source for a Build under spec.source. Also, HTTP sources are deprecated because we think that the focus that secure software supply chain brings, does not match with artifacts that are loaded from an HTTP endpoint.
• In Builds, spec.dockerfile and spec.builder are deprecated. Those fields were introduced at the very beginning of this project to support relevant build strategies. But, those fields only relate to specific build strategies. Since a couple of releases, we support strategy parameters for that purpose where we will move to.
• In BuildRuns, spec.serviceAccount.generate is deprecated. We think that the adhoc generation of service accounts does not fit very well in Kubernetes' security principals and will therefore move away from that concept.

Installing Shipwright​

Build​

1. Install Tekton v0.38.3:

   kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.38.3/release.yaml
   

2. Install v0.11.0 using the release YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.11.0/release.yaml
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.11.0/sample-strategies.yaml
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.11.0/cli_0.11.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.11.0/cli_0.11.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.11.0/cli_0.11.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first make sure the operator v0.11.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

Features​

Let's get back to v0.10. It comes with one big and long-wanted feature:

Volume support​

We extended our build strategy resource to contain volumes. Build strategy authors can "finalize" them - or make them overridable by Build users.

The most interesting scenario that this enables is the caching of build artifacts. Here is how the ko build strategy makes use of it:

• The build strategy defines a volume for the caching. The volume is of type emptyDir. Those are ephemeral volumes for the runtime of a pod. The default behavior therefore is that no caching happens just like the strategy behaves all the time. But, the build strategy defines the volume as overridable.
• Build users can therefore override that volume in their Build (or BuildRun, but for caching scenarios, the Build makes most sense), and point to a writable persistent volume. This allows reuse across BuildRuns.
• The ko build strategy references the volume in one of its steps and uses it for the GOCACHE.

Especially if you rebuild larger projects, the performance gain is enormous.

We will look at other sample build strategies in the future and will evolve them with volume support in the next releases.

Note: the feature comes with one breaking change that is relevant for Build Strategy authors. Previously, you were able to define volumeMounts on buildSteps. Shipwright then implicitly added the volumes with an emptyDir. Given we now support volumes in build strategies, we force build strategy authors to define the volume. We did that in other sample build strategies where such implicit volumes were used to share directories between buildSteps. For example, the source-to-image build strategy now has explicit emptyDir volumes to share directories between the source analysis and build step.

Smaller items​

Beside that, we invested in maintenance-related items:

• Dependabot now helps us to deliver secure releases by notifying us of necessary updates in our Go modules. We already merged its first pull requests.
• We now build with Go 1.18
• We're now supporting up to Tekton v0.35

Installing Shipwright​

Build​

1. Install Tekton v0.35.1:

   kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.35.1/release.yaml
   

2. Install v0.10.0 using the release YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.10.0/release.yaml
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.10.0/sample-strategies.yaml
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.10.0/cli_0.10.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.10.0/cli_0.10.0_macOS_$(uname -m).tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.10.0/cli_0.10.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first make sure the operator v0.10.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

We are pleased to announce that this year we will be hosting our first Shipwright Community Summit at the cdCon 2022 Conference.

The Shipwright Community Summit brings together our community members and future contributors, or any interested party. During the Community Summit we expect to get closer to you and address any questions you might have related to our Technology, Processes or Community.

If you would be interested to participate on this event, please refer to the next sections.

It goes without saying that we are very excited to meet you.

I want to participate, please tell me more​

The Shipwright Community will be available both on-site and virtually on Thursday June 9, 2022 2:00pm to 5:00pm CDT, you can find the schedule in here.

On-site: Find us at room 209 in the Conference Venue.

Virtually: We will be hosting a virtual session in Zoom. Join us on https://zoom.us/j/97487770299.

What to expect​

Additional Events​

Here are our two talks taking place during the conference.

DevSecOps with Shipwright and Tekton

Project Shipwright Update

Features​

Interested in what we have for you. Here are three larger items:

Standalone BuildRuns​

In Shipwright, in order to create a container image from source or a Dockerfile, you so far needed a Build and a BuildRun. Very simplified, it is the Build that contains all pieces of information required to know what to build and the BuildRun in this picture is the trigger to kick off the actual build process. This setup allows for a good separation of concerns and is ideal for use cases in which one builds a new image from the same source repository over the course of time.

However, frequent feedback we got is that there are also use cases where users just want to have a one-off build run and in these scenarios an additional Build just adds unnecessary complexity.

With this release, we introduced the option to embed a build specification into a BuildRun. That enables standalone build runs with just one resource definition in the cluster. The build specification is exactly the same configuration one would use in a Build resource, so it will immediately look familiar. Instead of buildRef in the BuildRun spec section, use buildSpec to configure everything you need.

apiVersion: shipwright.io/v1alpha1
kind: BuildRun
metadata:
  name: standalone-buildrun
spec:
  buildSpec:
    source:
      url: https://github.com/shipwright-io/sample-go.git
      contextDir: source-build
    strategy:
      kind: ClusterBuildStrategy
      name: buildpacks-v3
    output:
      image: foo/bar:latest

Some technical notes:

• You cannot use buildRef and buildSpec at the same time in one BuildRun, as this would be ambiguous. Therefore the respective BuildRun will reflect this with an error message in the status condition.
• The same rational applies also to the override options (e.g. timeout), which can only be used with the buildRef field. These overrides are not required, since you can define all build specific fields in the buildSpec directly.
• There is no preference for one or the other option, you can use buildRef or buildSpec in different build runs. It is up to your respective use case and liking.
• An embedded build specification does not count as a Build, it does not have a name and it does not affect the purely build related metrics (i.e. the build counter).
• Tekton TaskRun resources that are created by Shipwright will only have a BuildRun reference in their label for embedded builds, since there is no actual Build in the system. This is important for any label selector that might expect a TaskRun to have the build label.
• The buildSpec field and the Build spec field are technically the exact same definition. This means everything that is supported in builds is also supported as an embedded build specification. The same applies for potential field deprecation, it applies to both.

BuildRun cleanup​

In Shipwright, till now, we did not have a method to automatically delete BuildRuns. This release allows you to do that by adding a few fields to Build and BuildRun specifications. This feature can be used by adding the following fields:

• BuildRun specific time to live (TTL) fields:
  • buildrun.spec.retention.ttlAfterFailed: The BuildRun is deleted if the mentioned duration of time has passed after the BuildRun has failed.
  • buildrun.spec.retention.ttlAfterSucceeded: The BuildRun is deleted if the mentioned duration of time has passed after the BuildRun has succeeded.
• Build specific TTL fields:
  • build.spec.retention.ttlAfterFailed: The BuildRun is deleted if the mentioned duration of time has passed after the BuildRun has failed.
  • build.spec.retention.ttlAfterSucceeded: The BuildRun is deleted if the mentioned duration of time has passed after the BuildRun has succeeded.
• Build specific limit fields:
  • build.spec.retention.succeededLimit - Defines number of succeeded BuildRuns for a Build that can exist.
  • build.spec.retention.failedLimit - Defines number of failed BuildRuns for a Build that can exist.

Some technical notes:

• If both Build limits and TTL values are applied, the BuildRun will get deleted once the first criteria is met.
• In case TTL values are defined in BuildRun specification as well as Build specification, priority will be given to the values defined in the BuildRun specification.
• When changes are made to build.spec.retention.failedLimit and build.spec.retention.succeededLimit values, they become effective immediately.
• When changes are made to build.spec.retention.ttlAfterFailed and build.spec.retention.ttlAfterSucceeded values in Builds, they will only affect new BuildRuns. However, updating buildrun.spec.retention.ttlAfterFailed and buildrun.spec.retention.ttlAfterSucceeded in BuildRuns that have already been created will enforce the changes as soon as they are applied.
• If the above mentioned retention fields are not used, BuildRuns will not be deleted automatically.

Our command line interface is supporting these new fields when creating Builds, and BuildRuns:

$ shp build create my-build [...] --retention-failed-limit 10 --retention-succeeded-limit 5 --retention-ttl-after-failed 48h --retention-ttl-after-succeeded 3h

$ shp build run my-build --retention-ttl-after-failed 24h --retention-ttl-after-succeeded 1h

Local sources using the bundle approach​

In our v0.8.0 release, we enabled local source for builds using the streaming approach. A BuildRun is then waiting to get sources which the CLI streams using kubectl exec capabilities. This is an amazing feature that enables you to build container images without being required to commit and push your sources into a Git repository.

With v0.9.0, we support an alternative approach to transport sources into the BuildRun: using a container image, we call it the source bundle. You setup your build with some additional flags:

$ shp build create my-build [...] --source-bundle-image my-registry/some-image --source-bundle-prune Never|AfterPull --source-credentials-secret registry-credentials

And then you run it in the same way as with the streaming approach:

$ cd directory-with-my-sources
$ shp build upload my-build

The CLI will package the sources and upload it to the container registry. The BuildRun will download them from there. The --source-bundle-prune argument enables you to specify whether sources should be kept, or deleted after the BuildRun pulled them.

Why do we need two approaches? And which one should you use? Here are some criteria to decide:

• You want to use local sources without much configuration? Use the streaming approach.

• Your Kubernetes cluster does not permit you to perform a kubectl exec operation? Use the bundle approach.

• You want to keep the sources of your build to later access it? Use the bundle approach. The BuildRun status captures the digest of the source bundle image that was pulled:

  kind: BuildRun
  status:
    sources:
      - name: default
        bundle:
          digest: sha256:ecba65abd0f49ed60b1ed40b7fca8c25e34949429ab3c6c963655e16ba324170
  

You can read more about this in our CLI documentation.

Smaller items​

And that's not all, we have some smaller items that are worth to explore:

• We improved our sample build strategies to expose more parameters:
  • Buildpacks now has a platform-api-version parameter that allows to configure the CNB_PLATFORM_API version which is relevant to use features of newer Buildpacks implementations
  • The BuildAh sample strategy now exposes its parameters to configure default, blocked, and insecure registries as arrays. It also supports build-args in the same way as the BuildKit strategy.
  • The BuildKit strategy now supports to build multi-platform images.
• We added documentation about how to securely reference build strategy parameters in steps without allowing users to inject code in inline scripts. All sample build strategies are now secure.
• We added the shp version command to easily figure out which version of the command line interface is installed.
• Finally, we have now documentation for all command line interface commands which is automatically generated based on our commands, the flags and their description.

Installing Shipwright​

Build​

1. Install Tekton v0.34.1

   kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.34.1/release.yaml
   

2. Install v0.9.0 using the release YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.9.0/release.yaml
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.9.0/sample-strategies.yaml
   

CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.9.0/cli_0.9.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp version
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.9.0/cli_0.9.0_macOS_x86_64.tar.gz | tar -xzf - -C /usr/local/bin shp
shp version
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.9.0/cli_0.9.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp version
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first make sure the operator v0.9.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

What's next ?​

Some really cool features are under development these days:

• Volume support will make it in the v0.10 release. Much wanted to support for example layer caching.
• We will enable Shipwright to take care if the image push operation. This will at the beginning make build strategies simpler as they don't need to take care of that anymore, including capturing the digest and size of the image. In the future this enables more scenarios that we today cannot easily implement, such as SBOM creation, or vulnerability scanning.
• We work on trigger support which will enable you to configure your Builds to start right when you push changes to your Git repository.

You want to hear from us? Join us a cdCon in June in Austin where we speak about our project.

Features​

As promised in the v0.7.0 blog post, we closed last year developing three interesting features.

Array support in Parameters​

We introduced an extension to the parameter feature, by allowing users to define parameters in the form of a list. A list can be composed of values from secrets, configmaps or plain values.

Our main driver was the support for ARGS in Dockerfiles. This allows users to further customize their builds, by specifying variables that are available to the RUN command.

In addition, being able to use primitive resources (such as secrets and configmaps) to store key-values, allows users to protect confidential data or to share data when defining parameters values in their Builds or BuildRuns.

Note: For more details on this, please see the docs.

Surfacing Errors in the BuildRun Status​

Surfacing errors from different containers can be a challenging task, not because of technicality, but rather the question of the best way to represent the state. In case of failure or success during execution, we surface the state under the .status subresource of a BuildRun.

apiVersion: shipwright.io/v1alpha1
kind: BuildRun
# [...]
status:
  # [...]
  failureDetails:
    location:
      container: step-source-default
      pod: baran-build-buildrun-gzmv5-b7wbf-pod-bbpqr
    message: The source repository does not exist, or you have insufficient permission
      to access it.
    reason: GitRemotePrivate

In this release we concentrated on improving the state of errors that occur during the cloning of git repositories, by introducing .status.failureDetails field. This provides further details on why step-source-default failed.

In addition, this feature enables Build Strategy Authors to signalize what to surface under .status.failureDetails.reason and .status.failureDetails.message, in case a container terminates with a non-zero exit code. We will be gradually adopting this capability in our strategies, at the moment it is only used in the Buildkit strategy.

Now you do not need to worry if you have git misconfigurations in your Builds, we got you covered!

Note: For more details, please see the docs.

Local Source Upload​

At Shipwright, we've spent a lot of time trying to figure out the best ways to simplify the experience when building container images. In this release we are introducing a new feature that dramatically improves it, we call it Local Source Upload .

This feature allows users to build container images from their local source code, improving the developer experience and moving them closer to the inner dev loop (single developer workflow).

$ shp build upload -h

Creates a new BuildRun instance and instructs the Build Controller to wait for the data streamed,
instead of executing "git clone". Therefore, you can employ Shipwright Builds from a local repository
clone.

The upload skips the ".git" directory completely, and it follows the ".gitignore" directives, when
the file is found at the root of the directory uploaded.

 $ shp buildrun upload <build-name>
 $ shp buildrun upload <build-name> /path/to/repository

Usage:
  shp build upload <build-name> [path/to/source|.] [flags]

Go ahead and give it a try! The feature is now available in the v0.8.0 cli, look for shp build upload!

Installing Shipwright​

Build​

1. Install Tekton v0.30.1

   kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.30.1/release.yaml
   
   

2. Install v0.8.0 using the release YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.8.0/release.yaml
   

3. (Optionally) Install the sample build strategies using the YAML manifest:

   kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.8.0/sample-strategies.yaml
   

SHP CLI​

Windows​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.8.0/cli_0.8.0_windows_x86_64.tar.gz | tar xzf - shp.exe
shp help

Mac​

curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.8.0/cli_0.8.0_macOS_x86_64.tar.gz | tar -xzf - -C /usr/local/bin shp
shp help

Linux​

curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.8.0/cli_0.8.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
shp help

Operator​

To deploy and manage Shipwright Builds in your cluster, first make sure the operator v0.8.0 is installed and running on your cluster. You can follow the instructions on OperatorHub.

Next, create the following:

---
apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-operator
spec:
  targetNamespace: shipwright-build

Is it a big thing just like our previous v0.6.0 release? No, but we have still done a couple of nice things:

What's New​

Signed Images on ghcr.io​

We decided to go away from Quay as container registry for the images that we produce, and instead to consolidate things in GitHub. The new images can therefore be found in the Packages section of our repository. No worry, nothing that you need to bother so much. You can continue to install Shipwright through the Kubernetes manifest and that has the right image location in it.

What else has changed with our images? We now sign them with cosign! You can verify our controller image with the following command:

COSIGN_EXPERIMENTAL=1 cosign verify ghcr.io/shipwright-io/build/shipwright-build-controller:v0.7.0

Verification for ghcr.io/shipwright-io/build/shipwright-build-controller:v0.7.0 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - Any certificates were verified against the Fulcio roots.

[{"critical":{"identity":{"docker-reference":"ghcr.io/shipwright-io/build/shipwright-build-controller"},"image":{"docker-manifest-digest":"sha256:887b76092d0e6f3c4f4c7b781589f41fde1c967ae9ae62f3a6bdbb18251a562f"},"type":"cosign container image signature"}...

Our signing process takes advantage of the new keyless mode for cosign and support for OIDC tokens in GitHub actions.

shp - New buildrun delete Command​

In v0.7.0 we fixed a bug that prevented the shp buildrun delete command from being exposed to users. Thanks to new static analysis tooling, we were able to catch this bug and make this command available. This will be useful for clusters that rebuild their applications regularly with Shipwright, as pruning old BuildRun objects is essential to keeping your cluster healthy.

Revamped Operator Experience​

We have completely overhauled the operator for those who use Operator Lifecycle Manager (OLM) to install Shipwright. The operator now comes with its own Custom Resource - ShipwrightBuild - that allows administrators to control where the Shipwright build controller is installed. It also has the ability to automatically install Tekton Pipelines if Tekton's operator is present in one of OLM's catalogs.

Other Features​

So far, everything was mostly internal. No new features for our users at all? Of course not. :-) But smaller this time:

• We lifted the limit to specify image annotations and labels just on the Build. You can now also define them on the BuildRun. Our CLI finally knows them as well. You can specify them both when creating Builds and BuildRuns.
• We introduced a new configuration option on the build controller: GIT_ENABLE_REWRITE_RULE. This is false by default, but you can set it to true in the deployment. It addresses a problem that many of our users faced downstream when using Shipwright in IBM Cloud Code Engine: to access private code repositories, they managed to get their SSH key into the system and referenced it in the Build. But, they failed to specify the SSH URL but instead used the HTTPS one. The magic flag advises our Git step to setup an insteadOf configuration so that the HTTPS URL is internally rewritten to the SSH URL automatically when applicable. The same setting also helps if you have submodules configured using HTTPS in your .gitmodules file, but need to pull them with authentication.
• We worked further on our environment variable support: the Buildpacks sample build strategy now performs the necessary logic to propagate them to the Buildpacks build so that you can customize the behavior of your build using one of the many configurations that the Buildpacks support.
• We also further enhanced the BuildRun status: for your Git source, we now include the branch name in case you have not specified any revision in your Build. That way you have better insights on what was built.

Important Notes​

Dependency updates​

• We updated our projects to be current: we now build with Go 1.17.
• With Kubernetes 1.19 being out of support, we changed our minimum version to 1.20 and internally build with the libraries of 1.21.
• We now want you to use Tekton Pipelines 0.27 at a minimum but recommend to use the latest 0.30 version. The Shipwright team contributed a fix that optimizes reconciliations for TaskRuns that are similar to those created by Shipwright BuildRuns.

Deprecations and Breaking Changes​

• As previously noted, the minimum supported version of Kubernetes for Shipwright is now v1.20, and the minimum supported version of Tekton Pipelines is 0.27.
• In Shipwright build strategies, an emptyDir volume is implicitly created if the strategy uses volume mounts in the build stpes. This implicit behavior is now deprecated, and will be replaced with explicit support for volumes in an upcoming release.

How to install​

Shipwright Builds​

Nothing has changed on the installation. Assuming your Kubernetes cluster is ready, it's all done with just these three commands, which install Tekton, Shipwright, and the sample build strategies.

kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.30.0/release.yaml
kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.7.0/release.yaml
kubectl apply -f https://github.com/shipwright-io/build/releases/download/v0.7.0/sample-strategies.yaml

shp Command Line​

For the CLI, we still have no package manager support. :-( You therefore need to get the release again from GitHub:

Windows​

$ curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.7.0/cli_0.7.0_windows_x86_64.tar.gz | tar xzf - shp.exe
$ shp help

Mac​

$ curl --silent --fail --location https://github.com/shipwright-io/cli/releases/download/v0.7.0/cli_0.7.0_macOS_x86_64.tar.gz | tar -xzf - -C /usr/local/bin shp
$ shp help

Linux​

$ curl --silent --fail --location "https://github.com/shipwright-io/cli/releases/download/v0.7.0/cli_0.7.0_linux_$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -xzf - -C /usr/bin shp
$ shp help

Operator​

Coming soon! We are in the process of adding the new version of the Shipwright operator to OperatorHub. Once the new version has been added, you can follow the provided instructions on OperatorHub to install the operator with OLM. If you previously installed v0.1.0 of the Shipwright operator, you must remove it first.

Once the operator has been installed, you will be able to deploy the Build controller and APIs by creating an instance of the ShipwrightBuild custom resource:

apiVersion: operator.shipwright.io/v1alpha1
kind: ShipwrightBuild
metadata:
  name: shipwright-build
spec:
  targetNamespace: shipwright-build

What's next ?​

We have cool stuff being under development. So, let me list four items that are all cool and are in the makings.

Soon you should be able to see an evolution of parameters: build strategy authors will be able to define array parameters which is required for scenarios such as build-args for Dockerfile-based builds, and Build users will be able to define that the value should be retrieved from a ConfigMap or Secret.

We work on the next evolution of error reporting inside BuildRuns. Where you today just see a BuildRun being failed and must look at the Pod logs to determine the root cause, you will see further error reasons and messages in the BuildRun status directly – with those details provided internally in our Shipwright-managed steps (like the Git step, for example when a revision does not exist), but also by the build strategy author (for example if Buildpacks fail to detect any source code, or a Dockerfile-based strategy when the Dockerfile does not exist).

We are also looking to add support for volumes (such as PersistenVolumeClaims) in future releases. Our first pass will help build strategy authors add volumes in their build steps, enabling capabilities such as artifact and image layer caching.

Finally, we investigated and spiked on different approaches to support building from source code from your local disk rather than from a Git repository. Part of the code is even in the product already, but we need more time to get it done end-to-end.

Stay tuned!