Tutorial – 3DF Zephyr parameters tuning

Welcome to the 3DF Zephyr tutorial series.

In this recipe, you will learn how to get the best results possible with 3DF Zephyr.

  • Introduction

3DF Zephyr is a powerful yet simple tool: we put a lot of effort in making it automatic and user friendly, however, this advanced guide will significantly improve both your workflow (i.e. by enabling you to save processing time) and the final result (i.e. getting a more accurate mesh). The default settings will work for most cases, but you can always get a better result.

Before going into the actual parameters tuning guide, we need a deeper understanding on what is happening under the hood. The typical 3DF zephyr workflow is composed by these phases

· Structure from motion (New Project Wizard)
· · OrthoMosaic Generation
· · Multiview Stereo (Dense point cloud Generation)
· · · Surface Reconstruction
· · · · Orthophoto or Texture generation

Different settings in each phase can have huge impact in the phase itself and its following phases as well. Keep this in mind: if a phase result looks good, there’s a good chance the following one will be good as well. However, if you’re having odd results in a phase, you might want to tinker with the settings before going further down the pipeline. We will now go through each phase and understand what can be changed to achieve better results.

This guide focuses on the advanced settings.For a the custom/XML parameters guide, follow this link.

  •  Step 1 – Structure from motion
3DF Zephyr tuning 1
click to view fullscreen
3DF Zephyr tuning 1
click to view fullscreen

The goal of the structure from motion phase is to automatically extract internal and external camera parameters (long story short, these parameters are data needed by 3DF Zephyr to adjust the input depending on the camera objective, rotation and translation). This phase can be further divided into four steps:

  • Keypoint extraction
  • Pairwise image Matching
  • Neighbor image search
  • Geometric Reconstruction

When presented with the “new project” wizard, you have the chance to change the behaviour on each one of these sub steps.

As you can see, the structure from motion phase will first extract the feature of each image (keypoint extraction); after that, Zephyr will go through the images to understand for each image which other images are connected via overlapping (neighbor image search).

Once that is done, Zephyr will need to understand, for each point in each image, where these points are located in other images: when all this is done, the actual reconstruction can begin. We can operate on each one of these steps for the best result possible.

· Keypoint density (Keypoint extraction)

Keypoints are specific points in a picture that zephyr can understand and recognize in different pictures. The more doesn’t always mean the better, as more points means more ambiguities during the matching phase. The keypoint density also controls the scale of points (points with higher scale are extracted, thus increasing the probabilty to match distant photos to closer ones)

Very Low: will extract up to 2500 keypoints per image
Low: will extract up to 5000 keypoints per image
Medium: will extract up to 7500 keypoints per image
High: will extract up to 10000 keypoints per image
Very High: will extract up to 15000 keypoints per image

– Decrease When
»» you need fast processing
»» You are not varying much the distance between the cameras and the subject
»» You have many overlapping photos

+ Increase When
»» Cameras have not been correctly oriented
»» You have photos in the same dataset that are close and distant to the subject
»» You have few photos, not overlapping or that have uniform textures

· Matching type

Controls which optimizations (if any) should be used when computing matches between images. Increase or decrease accordingly depending on the desired accuracy versus speed.

· Matching stage depth (Pairwise image matching)

Controls how many pairwise image matching to perform. Usually, the more is the better, however, this comes at a computational cost.

Automatic: euristically computed from the number of images
Low: 2 * number of images
Medium: 6 * number of images
High: 12 * number of images
Full : (number of images * number of images – 1 ) / 2 (ALL couples, O(N^2)

– Decrease When
»» you need fast processing
»» You have many overlapping photos with good texturing

+ Increase When
»» Not all cameras have been oriented
»» You have few photos, not overlapping or uniform textures

· Photo ordering (Neighbor image search)

If you know the photo ordering, you can skip the neighbor search phase (improving speed and accuracy). If you don’t know the photo ordering, pick “unordered“. However, if you took the pictures in a sequence or in a circular motion pick respectively “sequential” or “circular” (the circual squence is intended as a closed loop sequence)

· Reprojection error tolerance (Geometric Reconstruction)

Controls the maximum allowed reprojection error (in pixels) for each point during the geometric reconstruction phase and the minimum number of points needed to merge two clusters togheter.

Very strict: 1.0 pixels and 10 minimum matches
Strict: 2.0 pixels and 10 minimum matches
Normal: 3.0 pixels and 10 minimum matches
Wide: 3.0 pixels and 6 minimum matches

– Decrease When
»» Reconstruction failed to converge

+ Increase When
»» Not all cameras have been oriented

· Reconstruction Engine (Geometric Reconstruction)

You can tell Zephyr how cameras have to be reconstructed. With sequential, cameras are reconstructed one by one, growing a cluster of cameras, while if you pick global, cameras are reconstructed in i single step.

The globa pipeline scales very very well when there are a lot of cameras and in some cases can help with drifting errors. Nevertheless, the incremental pipeline can sometime guarantee a lower level of discarded cameras in some cases.

· Bundle adjustment (Geometric Reconstruction)

The bundle adjustment (BA) is a non linear minimization procedure that adjust points and cameras by minimizing the reprojection error of points. It’s performed at the end of each merge and a final bundle adjustment with all points and cameras is always performed at the end of the procedure.

Local: avoids redundant information by not using points that have already been fully adjusted before. This is the default option and works fine for most cases.

Local and restrained: BA is computed locally and it uses a maximum number for each camera (use this if you have very large datasets)

Full: always adjusts everything. This should be used only when you need more precision. This option comes at a computational costs and usually does not improves results locally.

  •  Step 2 – Multiview Stereo
3DF Zephyr tuning 1
click to view fullscreen
The goal of the multiview stereo phase is to extract a dense point cloud by matching the image pixels. As you already understood, this is what is happening during the “dense point cloud generation” phase.

It’s important that what we input to each phase is a good starting point. If you think that the output from the structure from motion phase can be improved, you might want to change the settings and try again. This come both with picture shooting experience and 3DF Zephyr experience. After a while, you will understand which parameters you should change to get better results in each phase!

MVS tuning option Description Hint
Output type Controls wheter to use raw data or refined data. Use raw when
»» you just want the raw dense point cloud with no automatic post processing whatsoever

Use refined when
»» You want a smooth point cloud

»» You want a sharp point cloud

Number of nearest cameras Controls the number of near cameras used to comput each depth map – Decrease When
»» You don’t have many overlapping photos
»» You need fast processing

+ Increase When
»» You have many overlapping photos

Resolution Controls the input image resolution. Eg, if the input image is shot at 4000×3000 pixels, if the resolution is set to 50% images are then rescaled to 2000×1500 pixels for processing.

Not always the more the better: sometimes resizing the images can lead to better results as jpegs or camera artifacts may be present at pixel level.

– Decrease When
»» You need fast processing

+ Increase When
»» You need accuracy
»» You need to preserve detail

Noise filtering Removes noise from the result. – Decrease When
»» the final results lacks parts

+ Increase When
»» the final results has noise

Discretization Level Controls the discretization level furing the depth maps merging phase. In general, this should be dependant to the image resolution parameter, meaning that increasing or decreasing the resolution should mirror what you pick as the discretization level.

It’s not really useful to have the discretization level set to huge if you are processing images at a very low resolution.

– Decrease When
»» You have not a lot of system memory
»» You need fast processing
»» There is noise in the final results

+ Increase When
»» You need accuracy
»» You need to preserve detail

Shape from silhouette

(Pro only)

Controls the shape from silhouette (voxel carving) settings. Masks are obviously required for this option to be enabled. SfS can output the point cloud directly or use the resulting point cloud as intitalization for the MvS step. Usually, looseness should be left to 0. – Decrease loseness when
»» You want to be more strict about the masks coherence

+ Increase loseness when
»» You think the masks are not correct

  •  Step 3 – Surface generation
3DF Zephyr tuning 1
click to view fullscreen
As you probably already imagined, this next tuning step is about the surface generation (the mesh extraction from the dense point cloud).

The surface generation computes and indicator function from oriented points by solving a Poisson problem.

We can tune the surface generation by acting on parameters of the surface generation algorithm itself and on the parameters required in the post processing step.

Poligon count controls the discretization parameter inside surface reconstruction. Intuitively, you want to increase this value when you want to preserve high detail and/or you want a high poly mesh. Decreasing this parameter is useful if don’t have much system memory, if you have noisy results or just – of course – if you want a low poly mesh.

The smoothness slider controls the smoothness of the mesh. Increase it when you have noisy results and decrease it when you want to preserve high detail.

The reconstruction type option controls the algorithm that will be used when generating the mesh, depending on your needs, pick smooth or sharp.

The min and max vertices options controls the number of the final vertices. This step is computed after the generation algorithm so it does not add detail to the output mesh: when this option is enabled, a mesh simplification (or densification) will be applied to reach the desired number of vertices.

  •  Step 4 – Texture and ortophoto generation

Both these procedures share similar parameters (Inverse mapping -> from 3D to image pixel) and are driven by the desired size of the ortophoto/texture. Since the color information for each pixel comes from different sources/cameras you can always specify the maximum number of cameras used for the computation.

In most cases, the best results should be obtained by leaving 1 maximum camera per triangle with the color balancing option enabled.

There are, however, other available options that require more manual control.

There are two generation methods that allows you to mix the color information from different sources: blending and multiband, plus a third mode.

blending Blending

This method weights set according to view angle and distance. While robust to illumination changes, mean does not preserve high frequencies.

In this example, blending isn’t a good choice: the edges of the cars are very blurred.

multiband Multiband

This method uses the mean value for the low frequencies and the best image to compute the high frequencies. This is usually better than blending since it’s still robust to illumination changes (less than blending) but preserving the high frequency, resulting in a sharper and cleaner image.

You can clearly see how multiband is better than blending in this example.

lossless Lossless

You can set the number of cameras per pixels to 1 and Zephyr will use the color information coming from one camera only, without any blending. This will preserve all the image information, however it’s 0% robust to illumination changes and the final image might look like a stitched mosaic.

In this example, the illumination doesn’t change much in between photos and the area is well seen by the nearest camera, so all in all the results between lossless and multiband are very similar.

You can better understand what we mean by “robust to illumination changes” by comparing not a detail, but an overview of the ortophoto.

Blending - ortophoto overview
click to view fullscreen
Multiband - overview
click to view fullscreen
Lossless - overview
click to view fullscreen
  • Final notes

This guide should have given you enough insights on how 3DF Zephyr works under the hood. Feel free to use the forum to ask further questions if you need clarification!