Module XXX found, but does not contain package XXX

Question

Not so familiar with Golang, it's probably a stupid mistake I made... But still, I can't for the life of me figure it out.

So, I got a proto3 file (let's call it file.proto), whose header is as follows:

syntax = "proto3";
package [package_name];
option go_package = "github.com/[user]/[repository]";

And I use protoc:

protoc --go_out=$GOPATH/src --go-grpc_out=$GOPATH/src file.proto

So far so good, I end up with two generated files (file.pb.go and file_grpc.pb.go) inside /go/src/github.com/[user]/[repository]/, and they are defined inside the package [package_name].

Then, the code I'm trying to build has the following import:

import (
    "github.com/[user]/[repository]/[package_name]"
)

And I naively thought it would work. However, it produces the following error when running go mod tidy:

go: downloading github.com/[user]/[repository] v0.0.0-20211105185458-d7aab96b7629
go: finding module for package github.com/[user]/[repository]/[package_name]
example/xxx imports
    github.com/[user]/[repository]/[package_name]: module github.com/[user]/[repository]@latest found (v0.0.0-20211105185458-d7aab96b7629), but does not contain package github.com/[user]/[repository]/[package_name]

Any idea what I'm doing wrong here? Go version is go1.19 linux/amd64 within Docker (golang:1.19-alpine).

Note: I also tried to only import github.com/[user]/[repository], same issue obviously.

UPDATE:

OK so what I do is that I get the proto file from the git repository that only contains the proto file:

wget https://raw.githubusercontent.com/[user]/[repository]/file.proto

Then I generate go files from that file with protoc:

protoc --go_out=. --go-grpc_out=. file.proto

Right now, in current directory, it looks like:

 - directory
 | - process.go
 | - file.proto
 | - github.com
   | - [user]
     | - [repository]
       | - file.pb.go
       | - file_grpc.pb.go

In that same directory, I run:

go mod init xxx
go mod tidy
CGO_ENABLED=0 go build process.go

The import directive in process.go is as follows:

import (
    "xxx/github.com/[user]/[repository]"
)

Now it looks like it finds it, but still getting a gRPC error, which is weird because nothing changed. I still have to figure out if it comes from the issue above or not. Thanks!

Answer 1

Your question is really a number of questions in one; I'll try to provide some info that will help. The initial issue you had was because

At least one file with the .go extension must be present in a directory for it to be considered a package.

This makes sense because importing github.com/[user]/[repository] would be fairly pointless if that repository does not contain any .go files (i.e. the go compiler could not really do anything with the files).

Your options are:

1. Copy the output from protoc directly into your project folder and change the package declarations to match your package. If you do this there is no need for any imports.
2. Copy (or set go_out argument to protoc) the output from protoc into a subfolder of your project. The import path will then be the value of the module declaration in your go.mod plus the path from the folder that the go.mod is in (this is what you have done).
3. Store the files in a repo (on github or somewhere else). This does not need to be the same repo as your .proto files if you "want it to be agnostic" (note that 2 & 3 can be combined if the generated files will only be used within one code base or the repo is accessible to all users).

Option 1 is simple but its often beneficial to keep the generated code separate (makes it clear what you should not edit and improves editor autocomplete etc).

Option 2 is OK (especially if protoc writes the files directly and you set go_package appropriately). However issues may arise when the generated files will be used in multiple modules (e.g. as part of your customers code) and your repo is private. They will need to change go_package before running protoc (or search/replace the package declarations) and importing other .proto files may not work well.

Option 3 is probably the best approach in most situations because this works with the go tooling. You can create github.com/[user]/goproto (or similar) and put all of your generated code in there. To use this your customers just need to import github.com/[user]/goproto (no need to run protoc etc).

Go Modules/package intro

The go spec does not detail the format of import paths, leaving it up to the implementation:

The interpretation of the ImportPath is implementation-dependent but it is typically a substring of the full file name of the compiled package and may be relative to a repository of installed packages.

As you are using go modules (pretty much the default now) the implementations rules for resolving package paths (synonym of import path) can be summarised as:

Each package within a module is a collection of source files in the same directory that are compiled together. A package path is the module path joined with the subdirectory containing the package (relative to the module root). For example, the module "golang.org/x/net" contains a package in the directory "html". That package’s path is "golang.org/x/net/html".

So if your "module path" (generally the top line in a go.mod) is set to xxx (go mod init xxx) then you would import the package in subfolder github.com/[user]/[repository] with import xxx/github.com/[user]/[repository] (as you have found). If you got rid of the intervening folders and put the files into the [repository] subfolder (directly off your main folder) then it would be import xxx/[repository]

You will note in the examples above that the module names I used are paths to repo (as opposed to the xxx you used in go mod init xxx). This is intentional because it allows the go tooling to find the package when you import it from a different module. For example if you had used go mod init github.com/[user]/[repository] and option go_package = "github.com/[user]/[repository]/myproto";" then the generated files should go into the myproto folder in your project and you import them with import github.com/[user]/[repository]/myproto.

While you do not have to follow this approach I'd highly recommend it (it will save you from a lot of pain!). It can take a while to understand the go way of doing this, but once you do, it works well and makes it very clear where a package is hosted.