FlowEngine  3.769
Photogrammetry Software Development Kit
FlowEngine Documentation

FlowEngine: Structure from Motion and Multiview Stereo libraries by 3Dflow

FlowEngine is the perfect photogrammetry Software Development Kit: a powerful, fully customizable photogrammetry reconstruction engine written in C++ for Windows and Linux platforms.

General Interface

FlowEngine is a thin layer on top of our technologies. As such, the interface is very simple. Abstractions are left to the users to be built on top.

Some very basic utilities are distribuited with the SDK (see section Utilities)

Objects

Flow Engine exposes a set of Objects that are used throughout various computations.

Some example of these Objects are: Camera, BoundingBox, StereoMesh.

Objects must be created and destroyed with their respective function (e.g. CreateCamera/DestroyCamera)

Objects are created and destroyed by the user. Flow Engine never creates Objects.

UniqueObjectPtr< T > can be used to facilitate automatic storage for Object classes.

Function parameters

The documentation specifies what kind of parameters are expected in a function call.

  • In parameters: the parameter is expected to in a valid state and, when applicabile, in a valid value range as described in the documentation. The passed-in parameter will not be modified (at least in its observable state) by the function.
  • Out parameters: the parameter must point to a valid memory location that will be filled with data by the function.
  • In,Out parameters: the parameter is expected to in a valid state and, when applicabile, in a valid value range as described in the documentation.

When passing references to Objects, the object must be in a valid state (i.e. at least succesfully created). Functions specify if the object must be in a specific state in the documentation.

When passing Buffers, the buffer must point to a valid memory region. Objects never store buffers directly. Instead memory is copied if needed. When appropriate, the documentation specifies how big the buffer must be before calling the function.

The parameter will also be modified by the function.

General notes

Passing a reference to NULL is undefined behavior.

Passing parameters in an invalid range is considered a broken precondition and therefore undefined behavior. Note that some functions will still check for invalid arguments when possible and will fail in an orderly manner by reporting an error code, but this should not be relied on.

Settings

You can set all the computation settings through the settings object and the setSetting function. For a full list of pipeline processing parameters, please refer to this page: https://www.3dflow.net/technology/documents/3df-zephyr-tutorials/3df-zephyr-parameters-tuning-custom-xml-settings/

Additionally, you can set the following general processing settings:

"Cuda","DeviceList": list of the devices to be used. If set to "0", the first Cuda device will be used. If set to "0 1" the first and the second Cuda device will be used together. If the list is empty, computation will run on CPU. "Workspace","TempPath": Path to the cache folder. Please note that the cache folder is not created or deleted automatically after the processing. "Workspace","ImagesPath": Path to the image folder. Used only in the FlowDriver example and not by the FlowEngine pipeline. "Workspace","ExportPath": Path to the export folder. Used only in the FlowDriver example and not by the FlowEngine pipeline.

A couple of examples configuration settings are included in the SDK redist/ExampleSettings folder.

Error Handling

Almost every function in the Flow Engine returns an error code

The result indicates if the operations was succesfull or what error caused the operation to fail.

The utility function CheckResult can be used to print an error message in string form to the standard error stream and terminate the application.

Object creation functions return nullptr if the object creation has failed.

See also
FlowEngine::Result
FlowEngine::CheckResult

Class utilities

The SDK is shipped with a small set of basic utilities to aid in the passing of data between client code and the engine.

Geometry and imaging

FlowEngine::Point2, Point3 hold respectively a couple and a triplet of doubles that represents a 2-dimensional and 3-dimensional position in space

FlowEngine::Normal is a triplet of floats that represents a normal (unit-sized)

FlowEngine::Color specifies a packed (32 bits) RGB color

FlowEngine::Triangle is a triplet of point indices that represents the vertices of a triangle specified in counter-clockwise order.

FlowEngine::TexCoords is a pair of floating point values holding texture coordinates.

FlowEngine::Quaternion is used to represent an axis and a rotation around that axis

Buffers

Buffers are very simple structs that aid in the passing of parameters that are generally coupled together as pointer and size.

They hold a typed pointer to an array of values and the number of elements they point to.

They never own the data they point to. This also implies trivial copying.

Various implicit constructors help the user code to call functions without having to manually convert their data first, so generally they are almost invisible to the user.

See also
FlowEngine::Buffer
FlowEngine::ConstBuffer

Quick example

Here's a superquick example to compute everything:

using namespace FlowEngine;
// Setup settings
UniqueSettingsPtr settings( CreateSettings() );
if ( argc > 1 ) // Try to load the xml from file
settings->load( argv[ 1 ] );
// Prepare the progress bar
FlowEngine::ProgressBarEmpty progressBar;
// Prepare the log listener to let the application write the log to file
FlowEngine::LogListenerOStream logListener;
std::ofstream myfile( "log.txt" );
if ( myfile.good() )
logListener.mFileStream = &myfile;
// Prepare data to be filled
std::vector< UniqueCameraPtr > cameras;
UniqueSparsePointCloudPtr sparsePointCloud( CreateSparsePointCloud() );
UniqueStereoPointCloudPtr stereoPointCloud( CreateStereoPointCloud() );
UniqueStereoMeshPtr stereoMesh( CreateStereoMesh() );
UniqueStereoTexturedMeshPtr stereoTexturedMesh( CreateStereoTexturedMesh() );
// Load all the cameras in the folder
std::string imagesPath;
imagesPath.resize( settings->getValueLength( "Workspace", "ImagesPath" ) );
settings->getValue( "Workspace", "ImagesPath", imagesPath );
UniqueCamerasLoaderPtr camerasLoader( CreateCamerasLoader() );
for ( Size i = 0; i < camerasLoader->getImageCount( imagesPath ); ++i )
cameras.emplace_back( CreateCamera() );
CheckResult( camerasLoader->loadImages( imagesPath, true, cameras ), logListener );
// Main flowengine object
FlowEnginePtr flowengine( CreateFlowEngineObject() );
// Compute Structure from motion
CheckResult( flowengine->computeStructureAndMotion( *settings, progressBar, logListener, cameras, *sparsePointCloud ), logListener );
// Compute the bounding box
UniqueBoundingBoxPtr boundingBox( CreateBoundingBox() );
CheckResult( boundingBox->computeFromPoints( *sparsePointCloud, true ), logListener );
// Compute Dense point cloud with Multiview Stereo procedure
CheckResult( flowengine->computeDensePointCloud( *settings, progressBar, logListener, *boundingBox, cameras, *sparsePointCloud, *stereoPointCloud ), logListener );
// Compute Mesh
CheckResult( flowengine->computeMesh( *settings, progressBar, logListener, *boundingBox, cameras, *stereoPointCloud, *stereoMesh ), logListener );
// Finally compute the textured mesh
CheckResult( flowengine->computeTexturedMesh( *settings, progressBar, logListener, cameras, *stereoMesh, *stereoTexturedMesh ), logListener );
// Export output as .obj
UniqueWorkspaceSaverPtr workspaceSaver( CreateWorkspaceSaver() );
CheckResult( stereoTexturedMesh->saveToObj( "test.obj" ), logListener );

For a more complete example you can refer to the main example included in the solution

Initial camera calibration parameters

Exif grouping

Grouping cameras calibration means that the cameras with the same EXIF information about camera lenses and model or without EXIF information but with the same resolution will share the same calibration parameters during the reconstruction. Grouping should be always used when taking photos with the same camera device as it makes the reconstruction more stable. In the case of a rig of cameras, it's usually better to keep it disabled, unless the results are unstable (like in the autocalibration case).

If you are using the FlowEngine::CamerasLoaderInterface class, grouping in the FlowEngine::CamerasLoaderInterface::loadImages method is automatically handled by the boolean parameter groupCalibrations. If you are loading the images and the calibration of the camera manually, grouping must be handled manually. If you want to set the same camera calibration for two cameras simply use the FlowEngine::CameraInterface::setCameraCalibration and FlowEngine::CameraInterface::getCameraCalibration to link them. By default, newly created cameras have a unique camera calibration object.

See also
FlowEngine::CamerasLoaderInterface

Autocalibration, Adjusted, and Fixed internal parameters

In the camera calibration object, you can control how the internal parameters will be handled by the reconstruction pipeline. The FlowEngine::CamerasLoaderInterface utility class will automatically use the EXIF information (when available) to set up the cameras calibration object in the most appropriate way. When using .png files or images that don't contain any EXIF information, the pipeline will try to perform the autocalibration routine. If the EXIF information is present, an initial parameters guess will be set and the parameters will be only adjusted by the pipeline, The type of adjustment is defined by the FlowEngine::CameraCalibrationInterface::setAdjustmentType function.

Autocalibration: No initial focal length or radial distortion parameters are given. By default, a newly created camera calibration object will be treated as autocalibrated. The pipeline will try to find the internal parameters from images matches. The procedure can be unstable if the same camera calibration is used in very few cameras (less than 5-10). You can additionally set if you want to adjust the radial and tangential distortion or keep it fixed.

Adjusted internal parameters: The pipeline will use the parameters defined by the user as initial guesses, but it will adjust them during the reconstruction. You can set to adjust the focal length, optical center, tangential and radial distortion. While the optical center and the radial distortion can be set to the default values (respectively zero and half the image size), it's important to set a sensible guess for the focal length. The optional precalibration step performs a quick reconstruction with adjustment followed by a Euclidean reconstruction step and should be used especially when the initial focal length value is not very accurate.

Fixed internal parameters : Should be used in calibrated environments only. Even if you know all the calibration parameters and set them in the camera calibrations object, it's usually better to let the pipeline slightly adjust them to the specific dataset.

See also
FlowEngine::CameraCalibrationInterface::setAdjustmentType
FlowEngine::CameraCalibrationInterface::AdjustmentType

Compilation notes

The included sample can be compiled with CMAKE.

Here are the basic commands for compilation:

cd build
cmake ..

The default output path is the "redist" folder

Notes for Linux environment:

  • you may need to install the FlowEngine library in the user lib folder with:
    sudo cp lib/libFlowEngine.so /usr/lib
  • FlowEngine depends on ImageMagick version >= 7.0.0 and it should be installed on your system.

We suggest that you follow one of the many tutorials available. However, these basic steps should be sufficient for most cases:

1) Install pkg-config (very recommended so that ImageMagick configure step will easily find any additional image delegates)

sudo apt-get install pkg-config

2) Install additional ImageMagick delegates

sudo apt-get install libpng-dev
sudo apt-get install libjpeg-dev

3) Download and compile ImageMagick

wget http://www.imagemagick.org/download/ImageMagick.tar.gz
tar -xvf ImageMagick.tar.gz
cd ImageMagick-7.*
./configure --with-quantum-depth=16 --enable-hdri=yes CXXFLAGS=-D_GLIBCXX_USE_CXX11_ABI=0
make -j8
sudo make install
sudo ldconfig /usr/local/lib

Activation (Full version only)

You need to activate FlowEngine before using it on a specific machine.

To activate the product you can use the related API (FlowEngine::LicenseManager class) or the included utility.

For a standard activation, all you need to do is to invoke:

FlowEngineLicenser activate XXXX-XXXX-XXXX-XXXX-XXXX-XXXX

Here's the full command list:

Usage: FlowEngineLicenser <command>
where <command> is one of the following:
help -> Prints this help message
check -> Prints the current state of the license
activate <key> [filePath] -> Activates the key on this computer.
If [filePath] is specified, a license activation
request file is generated instead.
activate <filepath> -> Activates the license on this computer using
the provided offline activation response.
deactivate [filePath] -> Deactivates the current license from this computer.
If [filePath] is specified, a license deactivation
request file is generated instead.

Free Version limitations

If you are using the free version of the SDK, some export functions are not available and the processing is limited to 50 pictures / cameras.

If you try to use a function that is not available, you will get a Result::FeatureNotAvailable.

Licensing

The FlowEngine license is included in the main SDK directory in the file license.txt.

Unless you have already discussed and agreed a specific licensing model with 3Dflow srl, the same licensing terms of FlowEngine Free applies.