How to use symbol server with gitlab

Question

We are using Gitlab version 16.7 and I want to create a debug profile for my dotnet class library projects. But I couldn't find a detailed documentation on this topic. Below you can find my CI/CD settings. I couldn't find where to upload the .snupkg packages, I tried to upload it to "https://mygitlab.com/api/v4/projects/project-id/packages/nuget/symbolpackage" or I tried to upload it to "https://mygitlab.com/api/v4/projects/project-id/packages/nuget/index.json" but visual studio couldn't see the .snupkg file in both. How do you debug your projects with Gitlab?

stages:
- build
- deploy

image: mcr.microsoft.com/dotnet/sdk:7.0

variables:
  OBJECTS_DIRECTORY: 'obj'
  NUGET_PACKAGES_DIRECTORY: '.nuget'
  SOURCE_CODE_PATH: '*/*/'

cache:
  key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"
  paths:
    - '$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/project.assets.json'
    - '$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/*.csproj.nuget.*'
    - '$NUGET_PACKAGES_DIRECTORY'
  policy: pull-push

before_script:
  - 'dotnet restore --packages $NUGET_PACKAGES_DIRECTORY'
  - 'export VERSION_SUFFIX=beta$CI_PIPELINE_ID'

build:
  stage: build
  script:
    - 'dotnet build --configuration Release'

deploy:
  stage: deploy
  script:
    - 'dotnet pack -c Release /p:PackageVersion=1.0.0-$VERSION_SUFFIX --output $NUGET_PACKAGES_DIRECTORY --version-suffix $VERSION_SUFFIX --include-symbols -p:SymbolPackageFormat=snupkg'
    - 'dotnet nuget add source "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/nuget" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text'
    - 'dotnet nuget push $NUGET_PACKAGES_DIRECTORY/*.nupkg --source gitlab'
  environment: production

Answer 1

But I couldn't find a detailed documentation on this topic.

Maybe because it is not yet officially supported.
See gitlab-org epic 10825:

Nuget symbol package capability (.snupkg) was added in 14.1

The "Symbol packages" documentation stayes:

Consuming symbol packages is not yet guaranteed using clients such as Visual Studio or dotnet-symbol. The .snupkg files are available for download through the UI or the API.

A week ago (Dec. 19th, 2023), that epic says

The feature has been deployed to gitlab.com and can be used after enabling the symbol server capability in the group's package settings.

For now, only portable PDB symbol files are supported.

More details at "NuGet packages in the package registry / Use the package registry as a symbol server"

---

What don't I understand is if the pdb file and the .snupkg files are the same thing?

GitLab has two different endpoints, one is "/packages/nuget/symbolfiles" and the other is "/packages/nuget/symbolpackage": which one should I send my .snupkg files to?

To clarify:

• The PDB (Program Database) files store debugging information for a given assembly (like a DLL or EXE). They contain data like function names, variable names, and line numbers that help debuggers correlate executable code with the source code that produced it.

• The .snupkg files are NuGet symbol packages. They are essentially containers for PDB files and possibly other related debugging information. They are designed to be uploaded to a symbol server or a NuGet feed, allowing debuggers to automatically fetch the corresponding PDB files when needed.

In the context of GitLab, you have two different endpoints:

1. /packages/nuget/symbolfiles: This endpoint is likely meant for directly uploading PDB files.
   This assumes, as mentioned in the documentation, that you did enable the nuget_symbol_server_enabled namespace setting with the GraphQL API.

2. /packages/nuget/symbolpackage: This endpoint is for uploading .snupkg packages, which are the NuGet symbol packages containing the PDB files.

Since you are working with .snupkg files, you should use the "/packages/nuget/symbolpackage" endpoint. Your CI/CD pipeline needs to push these .snupkg packages to this endpoint, allowing GitLab to serve as a symbol server.

The relevant part of your CI/CD configuration for pushing the .snupkg files will be:

- 'dotnet nuget push $NUGET_PACKAGES_DIRECTORY/*.snupkg --source gitlab'

Make sure that the source 'gitlab' in the dotnet nuget push command is correctly configured to point to the /packages/nuget/symbolpackage endpoint on your GitLab instance.

---

Another question that confuses me is that the pipeline above saves the package output in the package registry.
When I look at the package details, .nupkg and .snupkg are next to each other: do I need to send the .snupkg files to one of the places I mentioned in my previous question?

The presence of both .nupkg and .snupkg files in your package registry is a good sign, indicating that your CI/CD pipeline is successfully uploading these packages.

1. .nupkg files are the NuGet packages containing your compiled code, resources, and metadata necessary for using your library.

2. .snupkg files are symbol packages contain the debugging symbols (PDB files) for your library. They are separate from the .nupkg files but are related to them.

When you upload these packages to GitLab, they should be stored in the package registry, which serves as a repository for both types of files. Your pipeline already seems to be configured correctly for this.

That means:

• You do not need to send the .snupkg files to a different place: The fact that they are stored alongside the .nupkg files in the package registry is the intended behavior. That setup allows users and tools that consume your NuGet packages to also find and use the corresponding symbol packages for debugging.

• The /packages/nuget/symbolpackage endpoint in GitLab is the correct destination for your .snupkg files. That is typically handled by your CI/CD pipeline script, as you have already set up with the dotnet nuget push command.

---

Based on your answers, it seems that I don't need to modify my pipeline file, I just need to place the code you suggested (dotnet push ..... "*.snupkg") at the bottom of my pipeline file.

But in my pipeline file we didn't give the endpoint that should be transferred for the symbolpackage, in this case which URL should I define for the symbolserver? According to the discussion, only pdb is currently supported, which means that the "symbolpackage" endpoint doesn't work.
How can I transfer pdb files to the "symbolfiles" endpoint?

Transferring the pdb file to the "symbolfiles" endpoint every time the pipeline runs and the packaging phase is completed seems to solve my problem. Do you have a pipeline code suggestion for this? How do I ensure that the pdb file is created during the pipeline run and how do I send it?

True: Since the .snupkg endpoint is not currently functional for your needs, you will be focusing on the /packages/nuget/symbolfiles endpoint to transfer PDB files.

So make sure your build process is creating PDB files. That is typically done by default in .NET projects, especially when building in the Debug configuration. However, if you are building in Release mode, make sure to include the necessary flags to generate PDB files. The DebugType property in your .csproj files controls this.

You need a script in your pipeline to identify and upload the PDB files to GitLab's /packages/nuget/symbolfiles endpoint. That process might involve using curl or a similar tool to make HTTP requests to GitLab's API. Your script will need to include authentication against your GitLab instance, typically using a CI job token or a personal access token.

That means your deploy stage might look like this:

deploy:
  stage: deploy
  script:
    - 'dotnet build --configuration Release /p:DebugType=portable'  # Make sure PDB files are generated
    - 'dotnet pack -c Release /p:PackageVersion=1.0.0-$VERSION_SUFFIX --output $NUGET_PACKAGES_DIRECTORY'
    - 'dotnet nuget push $NUGET_PACKAGES_DIRECTORY/*.nupkg --source gitlab'
    - for file in $(find $NUGET_PACKAGES_DIRECTORY -name '*.pdb'); do
        curl --request POST --header "JOB-TOKEN: $CI_JOB_TOKEN" --form "file=@$file" "https://mygitlab.com/api/v4/projects/$CI_PROJECT_ID/packages/nuget/symbolfiles"
      done
  environment: production

The /p:DebugType=portable flag means portable PDB files are created. Portable PDBs are generally recommended for .NET Core and later projects.

The find command locates all PDB files in your output directory. Adjust the path if your PDB files are located elsewhere.

The curl command uploads each PDB file to the GitLab API. Replace https://mygitlab.com with your actual GitLab instance URL, and make sure $CI_PROJECT_ID is correctly set in your CI environment. The script uses JOB-TOKEN for authentication. Make sure your CI job token has the necessary permissions to upload to the repository.

---

I get that error for every pdb files while post-process: {"error":"404 Not Found"}

The URL used in the curl command might not be correct. Double-check the API endpoint provided by GitLab for uploading symbol files. Make sure it matches the one you are using in the script.

And double-check that the $CI_PROJECT_ID in your script does correspond to your project's ID in GitLab. A mismatch here could result in the endpoint not being found.

Verify that the CI job token ($CI_JOB_TOKEN) has the necessary permissions to upload files to the repository. If the token lacks the required privileges, it might lead to a 404 error.

You can add debugging lines to help diagnose the issue (curl --verbose, and echo lines):

deploy:
  stage: deploy
  script:
    - 'dotnet build --configuration Release /p:DebugType=portable'
    - 'dotnet pack -c Release /p:PackageVersion=1.0.0-$VERSION_SUFFIX --output $NUGET_PACKAGES_DIRECTORY'
    - 'dotnet nuget push $NUGET_PACKAGES_DIRECTORY/*.nupkg --source gitlab'
    - for file in $(find $NUGET_PACKAGES_DIRECTORY -name '*.pdb'); do
        echo "Uploading $file"
        curl --verbose --request POST --header "JOB-TOKEN: $CI_JOB_TOKEN" --form "file=@$file" "https://mygitlab.com/api/v4/projects/$CI_PROJECT_ID/packages/nuget/symbolfiles"
      done
  environment: production