To fail, or not to fail, that is the question

Have you ever pushed a commit to your CI server, waiting to see if all your tests pass only to discover an earlier step like linting failed before you ever got your answer?

This is super frustrating and has probably led to the death of more than a couple CI initiatives.

You can do something to make this a bit less painful.

Most build servers have some kind of flag to allow a pipeline to continue running even if an individual step fails.

This is how I allowed some steps to fail without immediately breaking the build using GitHub Actions.

Each step has a continue-on-error property.

- name: Lint
  id: lint
  continue-on-error: true
  run: yarn lint

Now even if the linting step fails, the following steps will run.

Unfortunately, GitHub does not show this well on the UI, and if nothing else fails, the job will succeed.

So we now need to make sure that the job correctly shows the failure, and we need to give feedback to the user about our step that failed.

First, let’s fail the job.

- name: Check linting
  if: steps.lint.outcome != 'success'
  uses: actions/github-script@v3
  with:
    script: |
      core.setFailed('The Linting step failed!')

Next, let’s write a message to the PR notifying the user of the failure. I use a service account for this, aka a GitHub account only used for programmatically doing stuff in GitHub Actions. If you want to just use the built-in GitHub token, you’ll need to adjust the permissions.

- name: Update Pull Request
  uses: actions/github-script@0.9.0
  if: github.event_name == 'pull_request'
  env:
    LINT: "linting\n${{ steps.lint.outputs.stdout }}"
  with:
    github-token: ${{ secrets.SERVICE_ACCOUNT_TOKEN }}
    script: |
      const output = `#### Linting\`${{ steps.lint.outcome }}\`

      <details><summary>Show linting output</summary>

      \`\`\`\n
      ${process.env.LINT}
      \`\`\`

      </details>`;

      github.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: output
      })

These code snippets are licensed under the Unlicense. Go forth and have fun!