Database Migrations with DbUp

Managing database schemas can be a challenging problem. It is also an area where there is no one size fits all solution (not sure that is ever really true for anything). Solutions range from manual script management, database vendor-specific options (such as DACPAC for Microsoft SQL Server), to migrations. This post will be looking at one of the options for a migration based approach using DbUp.

Sample Database

I needed a sample database to start off with so I dug up one of my posts for Getting a Sample SQL Server Database to guide me through getting Microsoft’s WideWorldImporters sample database downloaded and restored.

DbUp Console Application

There are quite a few ways DbUp can be used, for this example, we are going to be using it from a .NET Core Console application. From a terminal use the following command to create a new console application in the directory you want the application in.

dotnet new console

Now we can use the following command to add the DbUp NuGet package to the sample project. In this case, we are using the SQL Server package, but there are packages for quite a few database providers so install the one that is appropriate for you.

dotnet add package dbup-sqlserver

Next, open the project in Visual Studio (or any editor but part of how this example is setup is easier in Visual Studio). In the Program class replace all the code with the following. We will look at a couple big of this code that below.

using System;
using System.Linq;
using System.Reflection;
using DbUp;

namespace DbupTest
{
    class Program
    {
        static int Main(string[] args)
        {
            var connectionString =
                args.FirstOrDefault()
                ?? "Server=localhost; Database=WideWorldImporters; Trusted_connection=true";

            var upgrader =
                DeployChanges.To
                             .SqlDatabase(connectionString)
                             .WithScriptsEmbeddedInAssembly(Assembly.GetExecutingAssembly())
                             .LogToConsole()
                             .Build();

            var result = upgrader.PerformUpgrade();

            if (!result.Successful)
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine(result.Error);
                Console.ResetColor();
#if DEBUG
                Console.ReadLine();
#endif                
                return -1;
            }

            Console.ForegroundColor = ConsoleColor.Green;
            Console.WriteLine("Success!");
            Console.ResetColor();

            return 0;
        }
    }
}

The bit below is trying to pull the connection string for SQL Server out of the first argument passed from the terminal to the application and if no arguments were passed in then it falls back to a hardcoded value. This works great for our sample, but I would advise against the fallback value for production use. It is always a bad day when you think your migrations have run successfully but it was on the wrong database because of a fall back value.

var connectionString = 
    args.FirstOrDefault() 
    ?? "Server=localhost; Database=WideWorldImporters; Trusted_connection=true";

This next section is where all the setup happens for which database to deploy to, where to find the scripts to run, and where to log. There are a lot of options provided by DbUp in this area and I recommend checking out the docs under the More Info section for the details.

var upgrader =
    DeployChanges.To
                 .SqlDatabase(connectionString)
                 .WithScriptsEmbeddedInAssembly(Assembly.GetExecutingAssembly())
                 .LogToConsole()
                 .Build();

Finally, the following is where the scripts are actually executed against the database.

var result = upgrader.PerformUpgrade();

The rest of the function is dealing with displaying to the console the results of the scripts running.

Adding Scripts

Keep in mind that we are using the Embedded Scripts option so DbUp is going to find all the embedded files that end with ‘.sql’. I have added a Scripts directory to the project so the scripts don’t clutter the root of the project, but that isn’t a requirement. Do not that how the files are named will control the order in which the scripts get executed so make sure you establish a good naming convention upfront. For our sample, I added a file named 0001-TestScript.sql to the Scripts directory. Since we are using the embedded scripts provider it is critical that after adding the file we go to its properties and set the Build Action to Embedded resource.

The script file itself doesn’t do anything in this case except select a value.

Trying it out

At this point, we can hit Run in Visual Studio and it will execute our new script. The output will look something like the following.

If we were to run the application again it would tell us that no new scripts need to be executed. DbUp like all migration based solutions I have encountered use a table in the database to keep a record of what migrations have been run. The default table for DbUp is SchemaVersion and it consists of an Id, ScriptName, and Applied columns. Given this structure, it is important to keep in mind that if you rename a script that has already been executed on a database DbUp will see that as a different script and execute it again.

Wrapping Up

In the docs for DbUp there is a philosophy section and one of the points is that a database is a result of a set of transitions, not just its new state vs. its old state. This point is key to why I am a fan of a migration based approach.

Check out the DbUp docs and GitHub repo for more information.

Database Migrations with DbUp Read More »

Summer Ramblings

Welcome to the summer edition of ramblings. I tend to do this type of post once a quarter so I have added a ramblings category to keep them better separated from my normal tech style posts.

COVID-19

COVID-19 is still raging across the world. Here in the United States, the case count seems to be growing at a faster and faster rate as the push for reopening continues. Keep in mind I’m writing this in mid-July so things may have changed drastically by the time this publishes. We have multiple at-risk family members and the longer this goes on the more the stress of builds up. I’m sure that is true for most people. All we can do is to continue to be as safe as we can and hope others are doing the same.

Working from home

Working from home continues to go well. I do miss the random interactions of the office and have a much more limited set of people I interact with on a daily basis mostly based on what project I’m working on. I think this has added to my feelings of isolation. I would say my interactions these days are 80% to 90% Microsoft Teams based with the remaining being Zoom. It is amazing to me how much Teams have changed since this pandemic started, especially the video features. I highly recommend giving Teams a shot if you haven’t before.

The lack of a commute is still the best part of working from home. I also tend to get more focus time than I do in the office. It seems people are more likely to schedule a meeting for something vs just popping in.

Blogging

Not a lot has changed on the blogging front since the Spring Ramblings post. Posts have continued to be focused around Azure DevOps with a little bit of GitHub Actions mixed in. I want to get back to more programming topics. .NET 5 is fast approaching and I need to dig in and see if I can find some topics there. I have also been kicking around giving Python a shot and if I do I’m sure I will end up with some post from that.

Wrapping Up

Here’s to hoping by the time I do another post like this will thing have improved. Thanks for reading!

Summer Ramblings Read More »

Azure DevOps Multi-Stage Pipelines: Require Stage Approval

In last week’s post, we covered taking our existing build pipeline and making it a multi-stage Pipeline with a build stage and a deploy stage. This week we are going to add another stage to our pipeline for production. Since we don’t want the production stage deployed before it has been through QA we will need to hold the stage until it is verified ready, which is what this post is going to be about. If you haven’t read last week’s post, Azure DevOps Pipelines: Multi-Stage Pipelines, you might want to start there before reading the rest of this post if you are new to multi-stage pipelines.

 

Add an Environment

In order to require approval on a stage is to associate it with and environment and add the approval requirement to the environment. In Azure DevOps under Pipelines select Environments and then click the Create environment button.

On the New environment dialog fill in a Name. If you had actual resources associated with the environment they can be added to provide traceability, but in this example, we are going to stick with the None option.

Require Approval for an Environment

Now that the resource has been created on its details page we can use the three dots to open the menu and click Approvals and checks.

On the next screen click the +button in the upper right corner and then from the lists of check select Approvals and then click Next. As you can see from the partial list in the screenshot the range of check available for approvals is massive.

The next dialog is used to select the uses or groups who should be able to perform approves for the environment. When your approves are set click Create to finish.

Use an Environment in a Pipeline

If you are following along from a previous post not that the Deploy stage has been renamed to QA to make the Pipeline results clearer.

Before:
- stage: Deploy
  jobs:
  - job: Deploy
    steps:
      - script: echo Fake deploying code

After:
- stage: QA
  jobs:
  - job: DeployQa
    steps:
      - script: echo Deploying to QA

Now we are going to add a new stage for our production environment. Notice that instead of a normal job we are using a deployment job which enables us to specify our desired environment. Deployment jobs have a ton more features than we are using so make sure and check out the docs to see what other options are available.

- stage: Production
  jobs:
  - deployment: DeployProduction
    environment: 'Production'
    strategy:
     runOnce:
       deploy:
         steps:
          - script: echo Deploying to Production

Save and run the Pipeline and we will look at how this new state presents differently than the Build and QA stages.

Pipeline Results

As you can see in the following screenshot the results of the Pipeline run have has a section notifying that the Production stage can’t run until it has been reviewed. Also, notice in the Stages section that the Production stage shows a Waiting status.

Clicking the Review button will show a dialog that will allow you to approve or reject the stage that is waiting.

Wrapping Up

The extra layer of options provided environments enables most of the scenarios that I missed when I first started playing with multi-stage Pipelines. Having the build and release steps for an app in source control and the added ability to vary them by branch makes this worth it even if I do have to deal with more YAML.

Azure DevOps Multi-Stage Pipelines: Require Stage Approval Read More »

Azure DevOps Pipelines: Multi-Stage Pipelines

The last couple of posts have been dealing with Release managed from the Releases area under Azure Pipelines. This week we are going to take what we were doing in that separate area of Azure DevOps and instead make it part of the YAML that currently builds our application. If you need some background on how the project got to this point check out the following posts.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging
Azure DevOps Pipelines: Depends On with Conditionals in YAML
Azure DevOps Pipelines: PowerShell Task
Azure DevOps Releases: Auto Create New Release After Pipeline Build
Azure DevOps Releases: Auto Create Release with Pull Requests

Recap

The current setup we have uses a YAML based Azure Pipeline to build a couple of ASP.NET Core web applications. Then on the Release side, we have basically a dummy release that doesn’t actually do anything but served as a demo of how to configure a continuous deployment type release. The following is the current YAML for our Pipeline for reference.

name: $(SourceBranchName)_$(date:yyyyMMdd)$(rev:.r)

resources:      
  repositories: 
  - repository: Shared
    name: Playground/Shared
    type: git 
    ref: master #branch name

trigger: none

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: PowerShell@2
    inputs:
      targetType: 'inline'
      script: 'Get-ChildItem -Path Env:\'

  - template: buildCoreWebProject.yml@Shared
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp1.csproj
      artifactName: WebApp1

- job: WebApp2
  displayName: 'Build WebApp2'
  condition: and(succeeded(), eq(variables['BuildWebApp2'], 'true'))
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - template: build.yml
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp2.csproj
      artifactName: WebApp2
      
- job: DependentJob
  displayName: 'Build Dependent Job'
  pool:
    vmImage: 'ubuntu-latest'

  dependsOn:
  - WebApp1
  - WebApp2

  steps:
  - template: buildCoreWebProject.yml@Shared
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp1.csproj
      artifactName: WebApp1Again

- job: TagSources
  displayName: 'Tag Sources'
  pool:
    vmImage: 'ubuntu-latest'

  dependsOn:
  - WebApp1
  - WebApp2
  - DependentJob
  condition: |
    and
    (
      eq(dependencies.WebApp1.result, 'Succeeded'),
      in(dependencies.WebApp2.result, 'Succeeded', 'Skipped'),
      in(dependencies.DependentJob.result, 'Succeeded', 'Skipped')
    )
 
  steps:
  - checkout: self
    persistCredentials: true
    clean: true
    fetchDepth: 1

  - task: PowerShell@2
    inputs:
      targetType: 'inline'
      script: |
        $env:GIT_REDIRECT_STDERR` = '2>&1'
        $tag = "manual_$(Build.BuildNumber)".replace(' ', '_')
        git tag $tag
        Write-Host "Successfully created tag $tag" 

        git push --tags
         Write-Host "Successfully pushed tag $tag"     

      failOnStderr: false

The above setup works great, but in April of this year, Azure Pipelines got the concept of multi-stage Pipelines which gives us the ability to manage the Release side of things in the same YAML as our builds and allows releases to be source controlled and different per branch in the same way that builds in YAML can be.

Simplified Build YAML

The above is the full YAML for our sample builds, which is a lot of code. The following is a paired down version that we will be using for the rest of this post that only builds WebApp1 and should help the changes stand out.

name: $(SourceBranchName)_$(date:yyyyMMdd)$(rev:.r)

resources:      
  repositories: 
  - repository: Shared
    name: Playground/Shared
    type: git 
    ref: master #branch name

trigger: none

variables:
  buildConfiguration: 'Release'

jobs:
- job: WebApp1
  displayName: 'Build WebApp1'
  pool:
    vmImage: 'ubuntu-latest'

  steps:
  - task: PowerShell@2
    inputs:
      targetType: 'inline'
      script: 'Get-ChildItem -Path Env:\'

  - template: buildCoreWebProject.yml@Shared
    parameters:
      buildConFiguration: $(buildConfiguration)
      project: WebApp1.csproj
      artifactName: WebApp1

Adding Stages

Stages are an extra layer of grouping that help divide a Pipeline similar to how jobs work except at a higher level. Jobs are a group of Steps, but Stages are a group of Jobs. In the following YAML, you can see that our existing jobs have been grouped under a Build stage and a new Release stage has been added.

name: $(SourceBranchName)_$(date:yyyyMMdd)$(rev:.r)

resources:      
  repositories: 
  - repository: Shared
    name: Playground/Shared
    type: git 
    ref: master #branch name

trigger: none

variables:
  buildConfiguration: 'Release'

stages:
- stage: Build
  jobs:
  - job: WebApp1
    displayName: 'Build WebApp1'
    pool:
      vmImage: 'ubuntu-latest'

    steps:
    - task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: 'Get-ChildItem -Path Env:\'

    - template: buildCoreWebProject.yml@Shared
      parameters:
        buildConFiguration: $(buildConfiguration)
        project: WebApp1.csproj
        artifactName: WebApp1

- stage: Deploy
  jobs:
  - job: Deploy
    steps:
      - script: echo Fake deploying code

When adding stages watch your whitespace it is easy to miss spacing in your existing code when wrapping them in stages.

Results

After running the Pipeline with the above changes you will see on the Pipeline’s summary page that it will display the results of each stage.

In the detailed view of a specific Pipeline run, there will now be a Stages tab that shows the results by stage. If you hit the expander on a stage it will also give you an option to rerun a stage if you ever have that need.

Wrapping Up

Hopefully, this will help you get a jump start on setting up your own multi-stage Pipelines. While I’m still not in love with YAML it is nice to have builds and releases in source control with the ability to vary by branch when you have the need. This setup works great if you want all your stages to run every time. A follow-up post will look at how to make a stage that requires approval.

Azure DevOps Pipelines: Multi-Stage Pipelines Read More »

Azure DevOps Releases: Auto Create Release with Pull Requests

Last week we covered auto-creating Release when a build completes. This week we are going to cover how to create a release when a build from a pull request completes. This setup would be helpful for verification of changes before the actual make it into a releasable branch. The following posts will help you catch up if you’re new to the series.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging
Azure DevOps Pipelines: Depends On with Conditionals in YAML
Azure DevOps Pipelines: PowerShell Task
Azure DevOps Releases: Auto Create New Release After Pipeline Build

Build Validation Branch Policy

Before we can have a Release created with a pull request we have to make sure that the pull request process does a build. I’m going to review how to do this quickly, for more info see my Branch Policies post. To do this we are going to head over to the Repos section of Azure DevOps. In the Branches section on the branch we want a build on a pull request for select the three dots and then click Branch policies.

On the repo settings page scroll down to the Build Validation section and click the + button to add a build to the pull request process.

The Add build policy dialog has a few options, but we are taking the defaults. Do note that if you have multiple Build pipelines that you make sure and adjust that option to the correct build. Click Save when you’re done.

Release Changes to allow Pull Request Trigger

Based on how we set up our Release to trigger when a build complete you might think that using the same build to validate a Pull Request would automatically trigger a new Release, but that isn’t the case. In most cases, this is actually a good thing since you wouldn’t want to deploy change before they have been reviewed. In the case we are trying to cover our release is to a QA environment so the requested changes can be verified before they make it into a releasable branch.

To enable a Release to be created from a pull request we need to head over to the Pipeline > Release area in Azure DevOps. Once there with the release in question selected click the Edit button.

In the Artifacts, section click the lightning bolt to edit the continuous deployment triggers.

Near the middle of the dialog, we want to Enable the Pull request trigger. Doing this will also require you to enter Target Branch Filters which are the branches that will be allowed to trigger a release when they are a target of a pull request.

Next, we need to enable our sample stage to be deployed for releases based on pull requests. Click the lightning bolt on the left side of the stage to edit the pre-deployment conditions.

On the dialog that shows Enable the Pull request deployment setting.

After closing the dialog make sure and Save the release.

Results

To show the results I created a new branch with a small change and created a PR into the master branch. From the Pull Request, we can click the View all checks button to see the status of the required build.

On the Checks dialog, you can see at the bottom that our sample release ran successfully.

If you click on the release you can see in the artifacts section that the files being used are from the pull request’s merge branch, not the branch being PRed or the target branch.

Wrapping Up

Building and releasing on a pull request opens up a lot of options, especially around making sure your code is verified before making it into a release branch.

Azure DevOps Releases: Auto Create Release with Pull Requests Read More »

Azure DevOps Releases: Auto Create New Release After Pipeline Build

I hit a snag with my planned post for this weeks and decided to hope back over to Azure DevOps and show how to create a Release and have that Release be triggered anytime a build Pipeline complete successfully. This post is going to be using the same Azure DevOps Project as all the previous posts in this series which are linked below.

Getting Started with Azure DevOps
Pipeline Creation in Azure DevOps
Azure DevOps Publish Artifacts for ASP.NET Core
Azure DevOps Pipelines: Multiple Jobs in YAML
Azure DevOps Pipelines: Reusable YAML
Azure DevOps Pipelines: Use YAML Across Repos
Azure DevOps Pipelines: Conditionals in YAML
Azure DevOps Pipelines: Naming and Tagging
Azure DevOps Pipelines: Manual Tagging
Azure DevOps Pipelines: Depends On with Conditionals in YAML
Azure DevOps Pipelines: PowerShell Task

Creating a Release

In Azure DevOps to create a release go to Pipelines and then Releases. Since this is our project’s first Release we have a New pipeline button to click to start the creation process.

The New pipeline button will start the creation process by showing a Select a template dialog. Since our release isn’t really going to do anything yet we are going to click the Empty job option.

Next, we want to add the artifacts from our build Pipeline to this Release. Click Add an artifact to start the process.

The Add an artifact dialog will show which allows us a lot of options on the source of the artifact. For this setup, we are going to use a Source type of Build since our artifacts are the result of an Azure DevOps Build. The next option we need to select is Source which is where we select which Azure DevOps Build Pipeline we want to use artifacts from. In this sample case, we only have a single option. Once the Source is selected a couple more options will show up, but we are taking the default values for those and just clicking the Add button.

Now that we have the basic release setup click the Save button. There will be a prompt for a folder. Either enter a folder name or leave blank to keep the release in the root and then click OK to complete the save.

Auto Create After Pipeline Build

Now that we have our basic Release Pipeline we are going to set up a continuous deployment trigger on the artifact so that any time a new build is completed. In the Artifacts area click the Lighting Bolt on the artifact the trigger should be on, we only have one option in our sample.

A dialog will show different trigger options. We are going to Enable the Continuous deployment trigger. Notice that when the trigger is enabled there is an option to only trigger for specific branches. I have used this option pretty often to only trigger a release when a build is done on master. When done click the X to close the dialog and then Save the release. I also renamed the release to Web App 1 & 2 to make it clearer what the release is doing.

Now pop over to the Pipelines and run a build. After the build is complete go back to the Release and you will see that a new release was triggered, and deployed in this case based on how the rest of the Release was set up.

Wrapping Up

In this post, we created our first Release, which doesn’t actually do anything, and configured it to deploy automatically on the build of our application. I use a similar setup to automatically deploy to a QA environment. This setup can open up a ton of scenarios.

Azure DevOps Releases: Auto Create New Release After Pipeline Build Read More »

GitHub: Use Actions to Run Multiple Jobs

In the post, we are going to take our sample Workflow that builds two ASP.NET Core web applications and split it so each web application is built individually. This post is using the repo and Workflow built in the following posts if you need to catch up.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application
GitHub: Use Actions to Publish Artifacts

Starting Point and the Plan

Our Workflow currently contains a single job that just happens to build two ASP.NET Core web application based on the fact that the .NET CLI picks up and builds both applications. The following is the YAML for our current Workflow.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish 
    
    - name: Upload WebApp1 Build Artifact
      uses: actions/upload-artifact@v2
      with:
        name: WebApp1
        path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
    - name: Upload WebApp2 Build Artifact
      uses: actions/upload-artifact@v2
      with:
        name: WebApp2
        path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

This post is going to take this Workflow and split the build and publish of the two web applications into two jobs. By splitting the Workflow into multiple jobs we open the possibility that the jobs can run in parallel. One reason to do this would be to speed up the total Workflow run time if you have parts of your build that are independent. Another example of why you would need multiple jobs is if the different jobs need different needed to run on different operating systems such as one needing to run Windows and another a Linux.

Creating the Jobs

The following is the Workflow set up along with the job to build the first web application. The first change was of the ID from build to build_web_app1 since each job has to have a unique ID. Most of the rest of the highlighted changes are related to the .NET CLI commands that are now directed at a specific project. Do also note that we changed from a hardcoded path to using expression to get the workspace path which is the ${{ github.workspace }} bit instead of /home/runner/work/Playground/Playground/. See the expression syntax docs for more info.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/upload-artifact@v2
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/

Here is the full file with both jobs defined. As you can see the second job is basically the same thing as the first one with a different ID, name, and project.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build_web_app1:

    name: Build WebApp1
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp1/WebApp1.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp1/WebApp1.csproj
    
    - name: Upload Build Artifact
      uses: actions/upload-artifact@v2
      with:
        name: WebApp1
        path: ${{ github.workspace }}/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
        
  build_web_app2:

    name: Build WebApp2
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
      
    - name: Build
      run: dotnet build ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --configuration Release --no-restore
      
    - name: Test
      run: dotnet test ${{ github.workspace }}/src/WebApp2/WebApp2.csproj --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish ${{ github.workspace }}/src/WebApp2/WebApp2.csproj
       
    - name: Upload Build Artifact
      uses: actions/upload-artifact@v2
      with:
        name: WebApp2
        path: ${{ github.workspace }}/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After all the edits are done commit the changes to the repo to run the Workflow. From the results of the Workflow run, you will see that it now has two jobs and we still got the artifacts for both applications as we had before.

Wrapping Up

If your applications need it breaking them up into different jobs can be helpful not only with Workflow runtimes but it can also help your ability to reason about what each part of your build process is doing.

GitHub: Use Actions to Run Multiple Jobs Read More »

GitHub: Use Actions to Publish Artifacts

This post is going to take the GitHub Actions Workflow we set up in the last post and add a couple of steps that will provide us with access to our application’s binaries. If you are new to this series the following post will catch you up if needed.

GitHub: Import an Azure DevOps Repo
GitHub: Use Actions to build ASP.NET Core Application

Edit the Workflow

Our first step is to get back to the Workflow we want to edit. At the top of the repo click Actions.

On the left side of the screen select the specific Workflow, .NET Core in this case. Now that the list is filtered to just the Workflow we are interested in select the three dots on the most recent run and then click View workflow file.

On the next screen click the pencil above the Workflow file to edit the Workflow.

Add the end of the file adds a call to the .NET CLI to publish. The following is the full file and the last two lines are the publish step.

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101
 
    - name: Install dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

    - name: Publish
      run: dotnet publish

Click the Start commit button and commit the change to the branch of your choice. I committed the change directly to master which triggered the Workflow to run. From the logs of the Workflow, you can see that the publish step executed successfully.

Publish Build Artifacts

Now that we have the two web applications publishing we need a way to get the file for the applications. To do this we are going to use the Upload Artifacts Action. I used the output of the Publish step above to find the path to the publish folder for each application and then used an Upload Artifacts Action for each application. The following are the two steps added to the bottom of our existing Workflow.

- name: Upload WebApp1 Build Artifact
  uses: actions/upload-artifact@v2
  with:
    name: WebApp1
    path: /home/runner/work/Playground/Playground/src/WebApp1/bin/Debug/netcoreapp3.1/publish/
    
- name: Upload WebApp2 Build Artifact
  uses: actions/upload-artifact@v2
  with:
    name: WebApp2
    path: /home/runner/work/Playground/Playground/src/WebApp2/bin/Debug/netcoreapp3.1/publish/

After checking in the changes let the Workflow run. Once complete if you click on the details of the Workflow run you will see it now has Artifacts that can be downloaded.

Wrapping Up

We now have a Workflow that results in files we could actually deploy on top of verifying that the applications builds and the tests pass. I hope this has given you a good jumping-off point to build your own Workflows. We have only scratched the surface of what can be done with GitHub Actions and I’m looking forward to seeing what else we can do.

GitHub: Use Actions to Publish Artifacts Read More »

GitHub: Use Actions to build ASP.NET Core Application

In this week’s post, we are going to use GitHub’s Actions to build one of the applications that we imported from an Azure DevOps Repo. The sample repo we are using can be found here.

Create an Action Workflow

From the repo in GitHub click the Actions option at the top center of the screen.

The Actions page will make suggestions based on the contents of the repo you are working with. In our case, the suggested .NET Core workflow is the one we are interested in. Click the Set up this workflow button.

The next screen that shows will be an editor loaded with the YAML for the .NET Core workflow we selected. For now, we are going to keep the YAML that was defaulted in and click the Start commit button. This workflow may or may not work for our repo at this point we are still exploring and can change as needed after we get a feel for how Actions work.

The next dialog is the commit details. For this initial change, we are going commit directly to master with the default commit message. Click Commit new file to continue.

View Workflow Status

Now that we have a workflow set up click on the Actions tab of the repo again to view the list of workflows and their status. As you can see in this screenshot the commit queued our new workflow to run.

The workflow finished quickly so I didn’t get to see the details while it was running, but if you click on commit title, Create dotnetcore.yml in this example, it will take you to the detail of this workflow run. From this view, you will see the jobs for the workflow listed on the left side of the screen, we only have one job which is the build. When you click on a job you will see the logs from that job. The following screenshot is the sample build job with the details of the build step expanded to show that both WebApp1 and WebApp2 were built.

Wrapping Up

Hopefully, this post will give you a good jumping-off point to create your own GitHub Actions. I was impressed with how easy it was to get started and the wide verity of languages supported especially for a feature set that has been out for less than a year. Check back in next week for more exploration of Actions.

GitHub: Use Actions to build ASP.NET Core Application Read More »

GitHub: Import an Azure DevOps Repo

Over the last couple of months, we have been exploring some of the features of Azure DevOps around Pipelines and Repos. I thought it would be interesting to see how the same type of setup and process might look using GitHub instead of Azure DevOps. I haven’t used GitHub other than for a basic repo before so I’m not sure how much of the Azure DevOps post will carry over, but we are going to find out. In this first post, we are going to import the repo we used in the Azure DevOps series from our Azure DevOps Repo.

Importing a Repo

To start in the project click the New button in the Repositories section of your GitHub dashboard.

On the next page click the Import a repository link.

The first thing the import process wants to know is the URL of our old repository. To get this we need to head over to our Azure DevOps Repo. Once in the Azure DevOps Repo click the Clone button.

When the dialog for clone shows you will see the URL, but before copying the URL hit the Generate Git Credentials button. This will create a username and password we will also need to enter when importing the repo at GitHub.

Here is the dialog after generating credentials.

Use the button next to the URL to copy URL and head back to GitHub and paste it into the URL box.  Next, enter a name for the repo and click Begin import.

The following is the screen you will see next while GitHub works on the import. Since the repo we are importing from needs credentials it will fail after a couple of minutes and ask for a login and password. If you hit refresh it will prompt you immediately without having to wait.

Copy and paste the values we generated in Azure DevOps above into the Login and Password boxes and then click Submit.

The page will update when the process is complete. GitHub will also send you an email so don’t feel the need to keep the page open while the process is running.

GitHub Repo Cleanup

Now that our repo is in GitHub we can clean up some of the items that were specific to Azure DevOps. For the sample project, this would include the files that were used in/or defined our build pipeline. The following are the files that can be deleted.

  • azure-pipelines.yml
  • build.yml

Wrapping up

As you have seen the repo transition from Azure DevOps to GitHub is a simple process. I’m looking forward to exploring how GitHub handles some of the scenarios from the Azure DevOps series. I’m betting that GitHub has support for most of them especially since the introductions of Actions.

GitHub: Import an Azure DevOps Repo Read More »