Introduction
I had often wondered what exactly would be the best file format to export images in when it comes to photogrammetry. Because photogrammetry takes so long I don’t usually like to do it much in between projects. Though I do like to experiment and improve my workflow, when I am in the middle of a 10-month project, tried and true is usually my mantra. When I did have some time recently I decided to give it a go. I processed the same photogrammetry scan of a semi-nude body with black garments in RealityCapture with different image file formats to figure out which one would yield the best results with regard to surface quality and color fidelity. Having finished the first tests here I do need to point out that any variations are very minor.
Pre-processing Images
With the exception of the raw camera data (Nikon NEF file format) I processed the images in lightroom as follows:
- For each camera series (D810, D5600, D800), I applied a unique linear color profile. These were generated from the X-Rite Colorcheckers using Colorchecker Camera Calibration and Adobe DNG Profile Editor.
- In the Detail tab I turned the default Sharpening to 0 and enabled Remove Chromatic Aberration. I leave all other settings to their defaults. It is always advised to let RealityCapture do the image distortion removal as it tends to yield better results.
- Using the Colorchecker image as a guide, I used the White Balance Selector and pick color 22, which is the third one from the bottom right (medium grey) as your neutral white balance color.
- I then changed the Exposure slider (or use the histogram) until color 22 had a value of 47.8%, or as close as possible. That is the absolute value that this color should have for R, G and B. This number is based on the actual values of the color chart as defined by X-Rite. You can find these values online. Hovering the cursor over the image should display the R, G and B values as percentages on the top right, just below the histogram. It should be noted that matching a single color on that chart is a good start, but to match up all the colors other software is required, such as 3DLutCreator.
- I saved these settings as the camera default by ALT-Clicking the Reset button, which turns to Set Default…. Before doing so I also make sure to enable Make defaults specific to camera serial number under Edit > Preferences > Presets. This will ensure the exact same processing parameters for all future images taken by that camera. You can always reset these settings in the future but it is helpful if you are setting up a permanent workflow with a photogrammetry rig that doesn’t change.
Exporting Images
|
File type |
RAW (.NEF) |
TIFF |
JPEG |
|
Bit depth |
12 and 14 bit |
16 bit |
8 bit |
|
Compression |
Standard |
ZIP |
Highest Quality |
|
File size |
3.45GB |
16.1GB |
1.25GB |
RealityCapture Alignment Settings
- Maximum features per mpx: 200,000
- Maximum features per image: 600,000
- Image overlap: Medium
- Image downscale factor: 1.0
- Maximum feature reprojection error: 2.0
- Preselector features: 200,000
- Detector sensitivity: High
Reconstruction
Here are the results of the alignment and reconstruction process:
|
File type |
RAW (.NEF) |
TIFF |
JPEG |
|
Parts (/remove floating) |
55 (44) |
52 (47) |
55 (44) |
|
Points (/remove floating) |
69.3m (69.1m) |
68.3m (68m) |
69.6m (69.3m) |
|
Texture |
Not color-corrected (exr) |
Oversaturated* |
OK |
|
Texture Bit Depth |
32 bit |
8 bit |
8 bit |
*At the time of writing this document Lightroom did not support OpenEXR image export. Similarly, OpenEXR support was not yet implemented in RC for exporting textures derived from 16 bit tiffs, so I am unable to test a 16-bit workflow.
Initial Observations
It would appear that there isn’t much difference between the three formats when it comes to camera alignment and mesh reconstruction. Only a few areas show misalignment “cracks” and surface noise. All three formats generate roughly the same amount of points and polygons. However, the first problem is that (at least at the time of writing this) RealityCapture appears to be unable to properly interpret the 16 bit tiffs and generates very oversaturated 8 bit images. Apparently RealityCapture converts 16 to 8 bit internally for mesh and texture calculation. On the other hand, the NEF images did generate a 32 bit OpenEXR image albeit not color corrected or linear.
So as of right now we need to consider that the three file types are 12 and 14-bit NEF, 8-bit TIFF and 8-bit JPEG.
Surface Quality
Although there doesn’t appear to be much difference between the various formats, I would argue that the tiff images yield a slightly better result, particularly visible in the tiny hairs in the stomach area and by the crispness and noise-level in the finger and toe nails. The jpegs and raw images appear to generate a slightly softer mesh and the nails have more surface errors such as dips. Also note the better surface results on the dark/black clothing.
In some areas such as the hands alignment errors (“cracks”) appear shorter in the tiff and raw versions compared to jpeg. Areas where surface noise is particularly bad the tiffs and raws seem to create smaller patches. It is difficult to tell, but in my opinion the areas between the fingers are better defined in the tiff and to a lesser degree the raw images.
Though minor, the facial area is slightly cleaner in the tiffs. I am looking specifically at the flanks of the nose where the skin is more consistent with less breaks and dips compared to the jpg version. Less noise is also present on the bumps on her skin in the tiffs.
Conclusion
As predicted the 16 bit tiffs (though internally down-converted to 8 bit) generate a slightly better result than the raw images, which in turn generate a much better result than the jpgs. For dark clothing I would definitely advise to use linear tiffs instead of jpegs. It must be noted that the 16 bit tiffs take up substantially more disk space than the other formats and may not be an efficient workflow for larger projects. In all likelihood, 8 bit tiffs will generate a similar if not identical result as their 16 bit counterparts.
Because I was unable to come up with a straightforward and consistent 16 bit workflow, I decided to keep this test to an analysis of just the raw geometry. I must also point out that, in all likelihood, these results may differ greatly from software to software. If time permits I’d love to do similar tests in Zephyr 5, Metashape and Meshroom.
Software Used
- Adobe Lightroom Classic 8.4.1
- Adobe Camera Raw 11.4.1
- RealityCapture 1.0.3.6310 Steam RC
- Adobe DNG Profile Editor 1.0.4
- X-Rite Colorchecker Camera Calibration 2.0.1
