AR.js distorted perspective: How to use a personalized camera calibration file `camera_para.dat` so that the "floor" plane is horizontal?

Question

I'm looking into AR.js for an augmented reality use case where 3D objects do not appear directly on the hiro marker, but somewhere around the marker.

When I view my AR scene through my iPhone 7 from the top, everything looks fine, but when I tilt my camera to get more perspective, AR.js doesn't apply the same perspective to the AR world so that distant virtual objects appear as if they were located on an inclined plane.

I created an example page to illustrate this behaviour: When viewed from above, the grids match perfectly, but when viewed from the side, the planes mismatch.

Are there any settings I can apply to configure AR.js (or ARToolKit, which it depends on)? Maybe there's a way to define the field of view of my webcam there?

[EDIT] One week later, I would reword my question to: How can I use a device-specific camera_para.dat ARToolkit camera calibration file in AR.js without generating side effects such as a distorted rendering?

Answer 1

Updating the intrinsic optical characteristics of the camera, also known as calibration, might help!

The artoolkitx-calibration app is made for calibrating cameras. Unfortunately, the app is not available on the App Store currently. You can, however, deploy it to your development device using Xcode.

Alternatively, the ARToolKitX calibration server might contain camera calibration results for your smartphone - Unfortunately, it returns 204 (no content) for the iPhone 7 (a.k.a. apple/iPhone/iPhone9,3, camera 0, aspect ratio 16:9).

By the way, camera_para.dat for several older iOS devices can be found on GitHub:

• iPad 2: 0,7 MP
• iPad Air 2: 8 MP, f/2.4
• iPad Mini 3: 5 MP, f/2.4
• iPhone 4: 5 MP, f/2.8
• iPhone 4s: 8 MP, f/2.4
• iPhone 5: 8 MP, f/2.4
• iPhone 5s: 8 MP, f/2.2 (similar to iPhone 6, iPhone 6 Plus)
• iPhone 6s Plus: 12 MP, f/2.2 (similar to iPhone SE, iPhone 6s)

Unfortunately, newer iPhone cameras have different specs (e.g. iPhone 7 or iPhone 8: 12 MP, f/1.8) so I doubt that any of these camera calibration settings would fit perfectly for them...