Custom Distributed Task Execution on Github Actions
Using Nx Agents is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an enterprise license, you can set up distributed task execution on your own CI provider using the recipe below.
Run Custom Agents on GitHub
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
.github/workflows/ci.yml
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8# Needed for nx-set-shas when run on the main branch
9permissions:
10 actions: read
11 contents: read
12
13env:
14 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
15 NX_BRANCH: ${{ github.event.number || github.ref_name }}
16 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
17 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
18
19jobs:
20 main:
21 name: Nx Cloud - Main Job
22 runs-on: ubuntu-latest
23 steps:
24 - uses: actions/checkout@v4
25 name: Checkout [Pull Request]
26 if: ${{ github.event_name == 'pull_request' }}
27 with:
28 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
29 ref: ${{ github.event.pull_request.head.sha }}
30 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
31 fetch-depth: 0
32
33 - uses: actions/checkout@v4
34 name: Checkout [Default Branch]
35 if: ${{ github.event_name != 'pull_request' }}
36 with:
37 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
38 fetch-depth: 0
39
40 # Set node/npm/yarn versions using volta
41 - uses: volta-cli/action@v4
42 with:
43 package-json-path: '${{ github.workspace }}/package.json'
44
45 - name: Use the package manager cache if available
46 uses: actions/setup-node@v3
47 with:
48 node-version: 20
49 cache: 'npm'
50
51 - name: Install dependencies
52 run: npm ci
53
54 - name: Check out the default branch
55 run: git branch --track main origin/main
56
57 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done
58 run: npx nx-cloud start-ci-run --distribute-on="manual" --stop-agents-after=e2e-ci
59
60 - name: Run commands in parallel
61 run: |
62 # initialize an array to store process IDs (PIDs)
63 pids=()
64
65 # function to run commands and store the PID
66 function run_command() {
67 local command=$1
68 $command & # run the command in the background
69 pids+=($!) # store the PID of the background process
70 }
71
72 # list of commands to be run on main has env flag NX_CLOUD_DISTRIBUTED_EXECUTION set to false
73 run_command "NX_CLOUD_DISTRIBUTED_EXECUTION=false npx nx-cloud record -- nx format:check"
74
75 # list of commands to be run on agents
76 run_command "npx nx affected -t lint,test,build,e2e-ci --parallel=3"
77
78 # wait for all background processes to finish
79 for pid in ${pids[*]}; do
80 if ! wait $pid; then
81 exit 1 # exit with an error status if any process fails
82 fi
83 done
84
85 exit 0 # exits with success status if a all processes complete successfully
86
87 agents:
88 name: Agent ${{ matrix.agent }}
89 runs-on: ubuntu-latest
90 strategy:
91 matrix:
92 # Add more agents here as your repository expands
93 agent: [1, 2, 3]
94 steps:
95 - name: Checkout
96 uses: actions/checkout@v4
97
98 # Set node/npm/yarn versions using volta
99 - uses: volta-cli/action@v4
100 with:
101 package-json-path: '${{ github.workspace }}/package.json'
102
103 - name: Use the package manager cache if available
104 uses: actions/setup-node@v3
105 with:
106 node-version: 20
107 cache: 'npm'
108
109 - name: Install dependencies
110 run: npm ci
111
112 - name: Start Nx Agent ${{ matrix.agent }}
113 run: npx nx-cloud start-agent
114 env:
115 NX_AGENT_NAME: ${{ matrix.agent }}
116There are comments throughout the workflow to help you understand what is happening in each section.