Azure Nuget Restore From Private Artifact Feed

Question

I'm trying to test out the AspNetZero (ANZ) template (the Angular 16 flavor) and I'm trying to build the code as a docker image in an Azure pipeline. I have a need to use a set of custom DLLs that I have built for other purposes and those DLLs have been "nugetized" and placed in an Azure Artifact Feed. These DLL projects exist in a separate Azure repo we will call "shared". The artifact feed is a project-scoped feed in the "shared" Azure project. This feed is configured and the nuget packages for those DLLs are already being built and pushed successfully in a separate Azure pipeline from the "shared" repo. From Visual Studio I am able to communicate with that artifact feed via my Azure user account and I can install those DLLs into the ANZ project. Locally, the code for ANZ builds and runs fine.

The problem I'm having is when I try to build the ANZ code inside it's Azure pipeline, the "dotnet build" command that executes from my dockerfile continues to throw the following error:

error NU1301: Unable to load the service index for source https://my_valid_feed_url.

I'm pretty confident this error is related to the authentication of the artifact feed, but nothing I have tried so far has worked.

Here is the setup I currently have:

1. Folder structure of ANZ code base: rootfolder/aspnet-core. This structure is part of the ANZ template and was not created by me. The aspnet-core folder contains the .sln file for all the projects in the aspnet code base. The aspnet-core folder contains a src sub-folder with each project divided into it's own project specific folder inside src. For instance, aspnet-core/src/Test.Migrator. The Test.Migrator folder contains the Test.Migrator.csproj and the project's dockerfile.
2. The rootfolder contains the azure-pipeline.yml file. The contents of this file are listed below.
3. The rootfolder also contains a NuGet.config file to support the custom Azure Artifact feed. The contents of this file are listed below.
4. The dockerfile that exists inside the Test.Migrator folder that is failing is also listed below.
5. Based on several other StackOverflow Q/As, I added the "Project Collection Build Service" to the artifact feed's security settings and granted it "Contributor". I also added the "ANZ Build Service" account to that feed's security as a "Contributor". Neither of these have resolved the issue.
6. The azureNugetServiceConnection is of type Nuget with Authentication Method of "External Azure DevOps Server".

I used the following document for part of the setup, specifically the section on "Azure Pipelines". https://github.com/dotnet/dotnet-docker/blob/main/documentation/scenarios/nuget-credentials.md

Side note 1: One of my troubleshooting steps also had the following between the NuGetAuthenticate@1 and Docker@2 steps in the job. This task succeeded and appeared to have connected to the feed based on the output logs in Azure. I have since taken that step out.

task: NuGetCommand@2
      displayName: Restore NuGet packages
      inputs:
        restoreSolution: '$(Build.SourcesDirectory)/aspnet-core/src/Test.All.sln'
        feedsToUse: 'config'
        nugetConfigPath: '$(Build.SourcesDirectory)/NuGet.config'
        externalFeedCredentials: $(azureNugetServiceConnection)

Side note 2: The RUN echo $VSS_NUGET_EXTERNAL_FEED_ENDPOINTS command in the dockerfile does echo out the result of the variable, but the password appears to be empty. Not sure if that is Azure redacting it, or if that is the problem. Here is that log output:

#18 [build 10/11] RUN echo {endpointCredentials: [{endpoint:my_valid_feed_url, username:docker, password:}]} #18 0.285 {`endpointCredentials`: [{`endpoint`:`my_valid_feed_url`, `username`:`docker`, `password`:}]} #18 DONE 0.3s

Any help would be greatly appreciated.

• azure-pipeline.yml

• Test.Migrator dockerfile

• NuGet.config

Answer 1

You can try to check with the following things to fix the issue:

1. In your Dockerfile, before the steps to build the project, use the 'ENV' instruction to set the valid username (Email Address) and password (Personal Access Token) that can access your Private Artifact Feed.

   ENV FEED_USERNAME="{username}"
   ENV FEED_PASSWORD="{password}"
   

2. In your NuGet.config file, you can use the '<packageSourceCredentials>' node to set the Credentials of your Private Artifact Feed like as below.

   . . .
   <packageSources>
       . . .
       <add key="CustomFeed" value="xxxx">
   </packageSources>
   <packageSourceCredentials>
       <CustomFeed>
           <add key="Username" value="%FEED_USERNAME%" />
           <add key="cleartextpassword" value="%FEED_PASSWORD%" />
       </CustomFeed>
   </packageSourceCredentials>
   

With above method, when building the Docker image, it will map the provided username and password as two environment variables (FEED_USERNAME and FEED_PASSWORD) within the Docker container.

Then when the subsequent steps to restore the packages from your Private Artifact Feed, it will to reference the packageSourceCredentials set in the NuGet.config file. The expressions '%FEED_USERNAME%' and '%FEED_USERNAME%' will try to use the values of the environment variables that have the same names within the container.

---

Related documentations:

• nuget.config reference
• Dockerfile reference

---

EDIT_1:

If you do not want to use your username and password in the Dockerfile, you can try to use a service principal to authenticate.

Prerequisites in AAD:

1. Ensure the Azure DevOps organization where your Private Artifact Feed is in has connected to an AAD (Microsoft Entra ID).

2. On Azure Portal, go to Microsoft Entra ID > App registrations to create a Service Principal in the AAD if you do not have one.

Prerequisites in Azure DevOps:

1. Go to Organization Settings > Azure Active Directory, ensure the Organization has connected to the AAD where the Service Principal was created.

2. Go to Organization Settings > Users, search and add the Service Principal into the Organization.

3. After successfully adding the Service Principal into the Organization, add it into a security group so that you can manage the permissions of the Service Principal through that group in the Organization. Give the group the access to your Private Artifact Feed.

Generate the AAD Access Token for the Service Principal: Use below Bash script to generate the token.

access_token=$(curl -X POST -H "Content-Type: application/x-www-form-urlencoded" 'https://login.microsoftonline.com/{tenant_ID}/oauth2/v2.0/token' \
-d 'client_id={client_id}&client_secret={client_secret}&grant_type=client_credentials&scope=499b84ac-1321-427f-aa17-267ca6975798/.default' | jq -r '.access_token')
echo $access_token

• Replace {tenant_ID} with the Directory (tenant) ID of AAD.
• Replace {client_id} with the Application (client) ID of the Service Principal.
• Replace {client_secret} with the value of the client secret created in the Service Principal.

After above steps, in your Dockerfile, you can try to set the Service Principal name as the username, and the Access Token as the password.

Note:

• The AAD Access Token has an only 24-hour lifetime as AAD will regularly rotate the token. So, you may need to refresh a new token at least once every 24 hours.

• However, after generating the token, if you only need it when building the Docker image within the 24 hours and not need it when running the Docker container after completing the build, you should not need to refresh the token.

Related documentation:

• Use service principals & managed identities

---

EDIT_2:

Another one more way is using the System.AccessToken:

1. Follow this documentation to assign the build identities the access to your Private Artifact Feed.

2. When building the Docker image, pass the System.AccessToken into your Dockerfile as the password. The username can be any, you can use the Azure DevOps organization name.

---

EDIT_3:

Below are the detailed configurations of using System.AccessToken to access the Azure Artifacts private feed.

1. In the nuget.config.

   <configuration>
     . . .
     <packageSources>
       <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
       <add key="CustomFeed" value="{URL of feed}" />
     </packageSources>
     <packageSourceCredentials>
       <TestNewFeature>
         <add key="Username" value="%FEED_USERNAME%" />
         <add key="ClearTextPassword" value="%FEED_PASSWORD%" />
         <add key="ValidAuthenticationTypes" value="basic" />
       </TestNewFeature>
     </packageSourceCredentials>
     . . .
   </configuration>
   

2. In the Dockerfile.

   . . .
   FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
   . . .
   ARG BUILD_CONFIGURATION=Release
   ARG FEED_USERNAME
   ARG FEED_PASSWORD
   . . .
   

3. Then when building the Dockerfile using Docker@2 task in pipeline.

   - task: Docker@2
     displayName: 'Build image'
     inputs:
       containerRegistry: {Docker registry service connection}
       repository: '{repository}'
       command: build
       Dockerfile: path/to/Dockerfile
       tags: |
        latest
        $(Build.BuildId)
       arguments: '--build-arg FEED_USERNAME=vsts --build-arg FEED_PASSWORD=$(System.AccessToken)'
   

---