Trouble Writing Data to Google BigQuery Using Storage Write API - Maven Dependency Issue

46 Views Asked by At

I am encountering difficulties while attempting to write data into Google BigQuery using their Storage Write API in Java. I have followed the documentation, but I am facing issues with the Maven dependencies they suggest. I am not able to find some classes with the provided dependency versions. Dependency Issue: The Maven dependencies provided in the documentation are not working as expected. I have included the following dependencies in my pom.xml file(https://cloud.google.com/bigquery/docs/reference/storage/libraries), but I am facing compatibility or version issues:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.30.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-bigquerystorage</artifactId>
  </dependency>

I also tried without BOM dependency.

I am looking for a working code example that demonstrates how to use the Google Cloud Storage Write API to stream data into Google BigQuery. The code snippets provided in the documentation do not seem to be compatible with the latest dependencies.

Specifically, I am interested in initializing the Google Cloud clients, creating a BigQuery table for streaming, writing streaming data to Google Cloud Storage using the Storage Write API, and ingesting that data into BigQuery.

If anyone has successfully implemented the Storage Write API with the latest Maven dependencies, could you please share a code snippet or provide guidance on resolving these dependency issues? Additionally, any links to GitHub repositories or relevant documentation would be greatly appreciated.

Which maven dependency I should add, as mentioned dependency does not have all the classes mentioned in the samples provided by Google github profile

Thank you for your help!

0

There are 0 best solutions below