CFHTTP unable to find valid certification path to requested target

Question

I'm looking to scrape data off a website, other https sites work and this was working last week but now fails

<cfhttp url="https://www.cliftoncameras.co.uk/all-brands-and-types-of-used-cameras/"></cfhttp>

If I run a dump of cfhttp

Exception: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I have tried running with the latest JRE version 12 - no change

https://helpx.adobe.com/coldfusion/kb/import-certificates-certificate-stores-coldfusion.html

Reverted back to CF original JRE, downloaded the target SSL certificate and installed it using the keytool - no change

c:\ColdFusion2018\jre\bin\keytool -import -keystore cacerts -alias cliftoncameras -file c:\ColdFusion2018\jre\lib\security\cliftoncameras.cer

I changed the websocket in the CFAdmin to proxy - no change

I did restart the CF Application Server each time.

What else can I do?

Answer 1

On my development platform I added

-Dcom.sun.security.enableAIAcaIssuers=true

To the java.args in the file in ColdFusion2018\cfusion\bin\jvm.config

Then restarted the CF Application Server, and now my CFHTTP call is successful.

Thanks to @agreax for this solution

Thanks to @sevroberts who's answer was probably the correct one, even though I couldn't get it to work. The production host installed the SSL certificate to the keystore and successfully resolved it this way. They said:

If you use FireFox browser and click on the lock icon when browsing the URL you are wanting to have the cfhttp request access you can then get the more info and click the View Certificate option. You will need to download the PEM (cert) not the Chain. Once downloaded, you need to run the keytool in order to import it to the keystore.

If you are using the default JRE within your JVM for ColdFusion you will need to install a JDK to your development machine. You can see the details and steps we have listed on our wiki regarding the commands from the command prompt to import the SSL into the store. https://wiki.hostek.com/ColdFusion_Tips_%26_Tricks#Fixing_cfhttp_Connection_Failures_with_Keytool

Thanks to @alexbaban his workaround, whilst it worked, it was a solution I could not implement due to requiring the use of the tag cfexecute.

Answer 2

I have also seen this java.security.cert.CertPathBuilderException error before from Java and Coldfusion on sites that load ok in a regular browser, but which still error from cfhttp even after adding the certificate to the CF keystore and restarting.

This happens when the target site server certificate configuration has a trust chain issue - when one or more trust chain paths requires the browser to perform an "extra download" of a certificate. This can be because of a missing intermediate certificate in a single trust chain path, or because there are multiple branches in the trust chain with different fingerprints and one or more certificates from one or more of those branches is not being served.

If you run the target site through an SSL Analyzer like ssllabs.com - eg https://globalsign.ssllabs.com/analyze.html?d=www.cliftoncameras.co.uk&hideResults=on - you'll see that their intermediate certificate Starfield Secure Certificate Authority - G2 is not being served by their server, which forces the client to do an "extra download" - which won't be a problem for most proper browsers, but the Java client used by cfhttp needs the server to provide pretty much every intermediate and root cert directly. It used to be the same for most mobile OSs up until a few years ago.

So the ideal solution is to contact cliftoncameras and have their server admin install the correct Starfield Intermediate certificate so that it is served correctly.

A possible workaround on your side is to install the Starfield Secure Certificate Authority - G2 intermediate certificate in your CF keystore.

Answer 3

If you can not get the keystore thing working maybe you'll want to try this.

Create a dedicated command line executable (.exe) which will read the web page and save the source to a text file. You can then use ColdFusion to read the file and work with the data.

Here is the ColdFusion code:

<cfscript>

    _execPath = "c:/bin/clifton.exe";
    _filePath = "c:/bin/clifton.txt";

    // run your command-line app (clifton.exe)
    cfexecute(name="#_execPath#");  

    // wait for the file
    do {
        sleep(100);
    } while ( not fileExists(_filePath) ) 

    // wait for write to finish
    do {
        sleep(100);
        _fileInfo = getFileInfo(_filePath);
        writeOutput(" ## ");
        cfflush();
    } while ( _fileInfo.size eq 0 || dateDiff("s", _fileInfo.lastmodified, now()) lte 3 )

    writeOutput("<hr />")

    _result = fileRead(_filePath);
    writeDump(_result);

</cfscript>

As you can see it depends on clifton.exe and reads clifton.txt (clifton.txt is the result of executing clifton.exe).

---

How to make clifton.exe

You will use the Dart SDK and the dart2native tool to create the executable on your development computer. You can deploy the executable on your production server as a standalone (You don't need the Dart SDK installed on production).

• Create a bin folder on your C drive.
• From https://ssl-ccp.secureserver.net/repository/?origin=CALLISTO download the certificate sfig2.crt.pem (PEM) and save it inside c:\bin.
• Inside c:\bin create a text file clifton.dart with the following code:

// clifton.dart

import 'dart:convert';
import 'dart:io';

main() {
  //
  const String _certFilePath = 'c:/bin/sfig2.crt.pem';
  const String _responseFilePath = 'c:/bin/clifton.txt';
  const String _uri =
      'https://www.cliftoncameras.co.uk/all-brands-and-types-of-used-cameras/';

  final File _file = new File(_responseFilePath);
  final IOSink _sink = _file.openWrite();

  final SecurityContext _context = new SecurityContext();
  _context.setTrustedCertificates(_certFilePath);

  final HttpClient _client = new HttpClient(context: _context);
  saveSourceToFile(_client, _uri, _sink);
  _client.close();
  //
}

// get web page source then write it to file
void saveSourceToFile(HttpClient _client, String _uri, IOSink _sink) {
  //
  _client
      .getUrl(Uri.parse(_uri))
      .then((req) => req.close())
      .then((res) => res.transform(Utf8Decoder()).listen((data) {
            // as data is received write to file
            _sink.write(data);
          }, onDone: () {
            _sink.close();
          }));
  //
}

• Download and install the Dart SDK from https://dart.dev/
• Open a terminal window and test the installation of Dart with dart --version (you should be able to run dart from any folder, if needed add dart to your PATH)
• In a terminal window, change directory to c:\bin with cd c:\bin
• Next, run dart2native clifton.dart -o clifton.exe
• If compilation goes well you should have inside c:\bin the three files: clifton.dart, clifton.exe and the certificate sfig2.crt.pem.
• If you wish you can test run clifton.exe in the terminal window, which should create the clifton.txt file.
• Test the ColdFusion page which calls clifton.exe, waits for clifton.txt then outputs the content.

If you deploy in production you need both files clifton.exe and sfig2.crt.pem (the certificate).

Good luck!