- Continuous code generation - Automated Utopia
- Continuous code generation - Versioning
- Continuous code generation - TypeScript libraries
- Continuous code generation - .NET libraries
- Continuous code generation - Automatically set up new libraries and APIs
I got a little lazy and did not end up recording how the .NET library got released, but this is for the TypeScript setup, which is 95% the same as for a .NET package :smile:
The video shows from start to finish the current state of the continuous code generation for TypeScript libraries (and this is 95% the same setup for .NET). So what is happening in the video?
First, an API change happens, where we add a new channel for the public Rust API, through a PR (read more about the setup here). The API version is then bumped to 0.8.0 as it was a minor version change.
Once the version changes, it's time to remotely trigger the code generation process, in this example for the rust-ts-public-api library. The code generation process is what the previous blog post outlined. The CI system then provides a PR with the desired change based on the version change of the API and code template.
Once merged the library is then released with the appropriate new version and then afterward bumps the package version to 0.5.0. For .NET that would be bumping the project version in .csproj.
As the video shows, it's not quite fully automated, as I still need to manually merge the PRs, it is, however, an "easy fix", as you can get the CI to auto-merge and even approve PRs if it's a specific bot that creates it (we have a similar setup at AsyncAPI). I just have not come around to include it yet even though it's the idea.
Generate new code
Because I choose to host (or at least that's how it came to be, as initially it was hosted on Gitlab) it on GitHub, many of the automation steps are for GitHub actions. However, it should be "rather easy" to translate it to another CI system as the building blocks and processes are there.
Find the current version of the library
As part of the generation script, one step was to find the current version of the library to know our starting point. For the .NET libraries this is "slightly" more tricky then for TypeScript.
The reason is its XML :sweat_smile: I ended up using a support library which of course needs to be installed beforehand. Next, the .NET project file is piped into the xml-to-json converter to read the current version through jq. I guess you could use the GitHub API to read the latest release and use the version therein, but that has drawbacks as well.
1if [ -f "./${libary_name}/${libary_name}.csproj" ]; then
2if ! command -v xml-to-json &> /dev/null
3then
4 git clone https://github.com/tyleradams/json-toolkit.git tooling
5 cd ./tooling && make json-diff json-empty-array python-dependencies && sudo make install
6 cd ..
7fi
8library_last_version=$(cat ./${libary_name}/${libary_name}.csproj | xml-to-json | jq -r '.Project.PropertyGroup.Version')
9else
10library_last_version="0.0.0"
11fiThe steps to follow to retrieve the current version of a .NET library
Auto-generate the library
Next is to trigger the code generation when the API definition receives a version change. If you have read the TypeScript setup, this setup is pretty much the same.
To achieve this I choose to add a generation workflow in the API library and one where the AsyncAPI documents are located, to remotely trigger the generation workflow. This is not the only way to achieve it, but it was the one I found most suiting in this case.
For the generation workflow, it's rather simple as all it does is execute the generate.sh script, read the results of the generation process - and then create appropriate PR with the desired conventional commit. The workflow is then set to be triggered on workflow dispatch which can be done remotely.
1name: Auto update generated library
2on:
3workflow_dispatch:
4jobs:
5update-code:
6 runs-on: ubuntu-latest
7 steps:
8 - uses: actions/checkout@v3
9 - uses: actions/setup-node@v3
10 with:
11 node-version: '14'
12 - name: Generate code
13 run: chmod +x ./generate.sh && ./generate.sh
14 - name: Set Environment Variables from generator file
15 uses: ./.github/actions/setvars
16 with:
17 varFilePath: ./.github/variables/generator.env
18 # Follow conventional commit for PR's to ensure accurate updates
19 - name: Create pull request for major version if needed
20 if: ${{env.major_version_change == 'true'}}
21 uses: peter-evans/create-pull-request@v4
22 with:
23 title: "feat!: major version change"
24 body: ${{env.commit_message}}
25 - name: Create pull request for minor version if needed
26 if: ${{env.minor_version_change == 'true'}}
27 uses: peter-evans/create-pull-request@v4
28 with:
29 title: "feat: minor version change"
30 body: ${{env.commit_message}}
31 - name: Create pull request for patch version if needed
32 if: ${{env.patch_version_change == 'true'}}
33 uses: peter-evans/create-pull-request@v4
34 with:
35 title: "fix: patch version change"
36 body: ${{env.commit_message}}GitHub workflow that re/generate the .NET library and provide a PR with the change
To remotely trigger the code generation from where the AsyncAPI documents are located, we can then use the GitHub action benc-uk/workflow-dispatch to trigger the workflow remotely. This is run whenever the bundled AsyncAPI document change. This is my lazy way of detecting version changes, as the AsyncAPI documents are bundled on each release.
1name: Trigger remote code generation for Rust public API
2on:
3workflow_dispatch:
4push:
5 branches:
6 - main
7 paths:
8 - 'bundled/rust_public.asyncapi.json'
9jobs:
10generate:
11 runs-on: ubuntu-latest
12 steps:
13 - name: Update csharp public game API library
14 uses: benc-uk/workflow-dispatch@v1
15 with:
16 workflow: Auto update generated library
17 token: ${{ secrets.GH_TOKEN }}
18 repo: GamingAPI/rust-csharp-public-api
19 - # Update ... public game API libraryGitHub workflow that triggers the above workflow when the API definition changes
The only thing we cannot trigger upon is when the code template receives a change. You could fall back to a cronjob which could trigger the generation workflow at some specific time, so it's semi-automatic.
Release the new library
This is the part that has taken me the longest to achieve because I wanted to reuse the same kind of setup as for TypeScript, at least as much as I possibly can. The fewer differences between the setups for each language the better, less to remember. Having a hard enough time remembering what I did yesterday...
The reason this was the hardest to achieve, was because at the time I started playing around with this setup, I did not look deep enough into the sematic-release library. This meant that I ended up writing a library to analyze the commits and change the version accordingly. Instead, I should have used their plugin system to achieve this.
For this blog post, I will keep my current setup, but in the future, I will probably switch to a sematic-release plugin. The same for AsyncAPI versioning in practice, where I developed gh-action-asyncapi-document-bump, this needs to be re-created to a sematic-release plugin (copy-paste more or less, just have to find the time).
The way the release workflow works are somewhat similar to the TypeScript setup just more simplified. It will be less customizable, but with the change mentioned above, it does not make sense to fix those limitations. How it works is that on each commit to the main branch, we analyze the commits since the last release. If it's determined that a new version is needed, the library is then packed and released.
For the .NET libraries I am releasing them to GitHub, this could of course be changed to something like NuGet. Once released it's time to commit the changes i.e. the version change, and potentially other resources that are changed during the release. These changes are committed to a branch and then provided through a PR to the main branch. You could directly commit them to the main branch, but I like the control a PR gives.
1name: Release
2env:
3GH_USER: jonaslagoni
4GH_EMAIL: <jonas-lt@live.dk>
5on:
6push:
7 branches:
8 - main
9jobs:
10release:
11 runs-on: ubuntu-latest
12 permissions:
13 packages: write
14 contents: read
15 steps:
16 - uses: actions/checkout@v2
17 - uses: actions/setup-dotnet@v1
18 with:
19 dotnet-version: '5.0.x' # SDK Version to use.
20 source-url: https://nuget.pkg.github.com/GamingAPI/index.json
21 env:
22 NUGET_AUTH_TOKEN: ${{secrets.GITHUB_TOKEN}}
23 - name: Install dependencies
24 run: dotnet restore
25 - name: 'Automated Version Bump'
26 uses: 'jonaslagoni/gh-action-dotnet-bump@main'
27 id: version_bump
28 env:
29 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
30 with:
31 skip-tag: 'true'
32 skip-commit: 'true'
33 path-to-file: './RustGameAPI/RustGameAPI.csproj'
34 release-commit-message-regex: 'chore(release): v{{version}}'
35 - if: steps.version_bump.outputs.wasBumped == 'true'
36 run: dotnet build --configuration Release RustGameAPI
37 - if: steps.version_bump.outputs.wasBumped == 'true'
38 name: Create the package
39 run: dotnet pack --configuration Release RustGameAPI
40 - if: steps.version_bump.outputs.wasBumped == 'true'
41 name: Publish the package to GitHub registry
42 run: dotnet nuget push RustGameAPI/bin/Release/*.nupkg --api-key ${{secrets.NUGET_AUTH_TOKEN}}
43 - if: steps.version_bump.outputs.wasBumped == 'true'
44 name: Create Pull Request with bumped version
45 uses: peter-evans/create-pull-request@v3
46 with:
47 token: '${{ secrets.GH_TOKEN }}'
48 commit-message: 'chore(release): v${{steps.version_bump.outputs.newVersion}}'
49 committer: '${{env.GH_USER}} ${{env.GH_EMAIL}}'
50 author: '${{env.GH_USER}} ${{env.GH_EMAIL}}'
51 title: 'chore(release): v${{steps.version_bump.outputs.newVersion}}'
52 body: Version bump
53 branch: 'version-bump/v${{steps.version_bump.outputs.newVersion}}'GitHub workflow that releases the .NET library on changes, and provide a PR with the updated release changes (such as version, etc)
Next
That finalizes the .NET setup, and now all there is left is to automate the library creation in GitHub, as it's ever-changing how many languages are provided for each game, and how many games are supported. Therefore I need some kind of setup where it enables me to quickly change any of these parameters.
Photo by Iswanto Arif on Unsplash