Spring Boot - Java: Check backward compatibility of two given APIs by comparing two openapi contract files

Question

I have a spring boot application and there I receive openapi contract files from the client side. So, I have to compare the newly received contract file with the latest contract file (from the DB) and should check whether they are backward compatible or not.

To do this, first I checked pact.io. But as I found, using pact.io we can only check backward compatibility by testing with the real endpoint. (By generating a contract file against the real endpoint and test it later using consumer driven contract testing). But, in my case I can't test it with the real endpoint. I only have old contract file and new contract file to check that. When I searched for this issue, I found this link: https://specmatic.in/comparisons/specmatic-vs-pact-io-and-pactflow-io and it says pact.io does not support Contract vs Contract comparison. And also, it says specmatic supports Contract vs Contract comparison. Check this image: pact vs specmatic

Then, I checked about specmatic and found we can compare two contract files using specmatic and know whether they are backward compatible or not. But we can do this only in command line using specmatic.jar file. We can do it by running java -jar specmatic.jar compare contract_file-v1.1.yaml contract_file-v1.1.yaml.

But, still I couldn't find a way to integrate this into my spring boot application. Also, in specmatic doc, it says as it supports 'Backward Compatibility Verification #NOCODE'.

So can anyone help me to solve this issue...

My questions are...

1. Can't we use pact.io to compare two given openapi contract files?
2. Can't we use specmatic to compare two given openapi contract files in a spring boot application?
3. If above both are not possible, do you know any other way to do this? (Compare two given openapi contract files in a spring boot application to check backward compatibility)

Please help regarding this.

Thanks.

Answer 1

Can't we use pact.io to compare two given openapi contract files?

No, Pact is not an OpenAPI/Swagger based tool - it's an API contract testing tool. Its purpose is orthogonal to what an API specification aims to achieve.

Pact can help you address this problem though, albeit it seems like you want a simpler solution where you don't understand what behaviour the consumers need of your API, and simply want to check that your spec is backwards compatible (these are very different things and provide different guarantees and benefits - see https://pactflow.io/blog/schemas-are-not-contracts/ for more). For what it is worth, PactFlow does have a feature that works with OpenAPI and gives you many additional benefits over comparing two specs.

We can do it by running as java -jar specmatic.jar compare contract_file-v1.1.yaml contract_file-v1.1.yaml.

But, still I couldn't find a way to integrate this into my spring boot application. Also, in specmatic doc, it says as it supports 'Backward Compatibility Verification #NOCODE'.

Well, it depends on what you mean by "no code". You need to find a way to generate the new OAS and get your hands on the old OAS. Once you have these, you can run the CLI check. I would run that as part of a CI build, and fail it if it's not backwards compatible.

Answer 2

Co-founder and CTO of Specmatic here.

May I know what you mean by below statement

integrate this into my spring boot application

As per my understanding you are building a Springboot application that has to compare two API specifications (one received from client and another that is in your DB). Please correct me if I am wrong.

If you are just looking for how you can invoke specmatic backward compatibility feature programmatically I can suggest reading through the compare command code for reference.

Also to clarify why we have a #NoCode approach to this is because we treat contract as code and thereby prefer having it stored in a version control system like Git. And backward compatibility verification is heavily used in shell / cmd like environments such as Pre-merge checks (Github and Gitlab workflows, Azure pipelines and more) of pull / merge requests of central contract repository. Other capabilities of Specmatic such as contract as test and contract as stub have both command line and programmatic interfaces becase.

Your use case seems interesting where you have a version of your API specification in a database and another being sent to you over an API endpoint (if my understand is correct).

It would be great if you can share more context on what you are trying to achieve. This will help me provide a much more contextualised answer.

Hope this helps.