Implementation of robust algorithms and softgauges for dense point clouds partitioning: application to traceable 3D scanners

Analysis of scanning systems needs

3D laser scanning systems are vital for the digital transformation of geometric measurements. They are widely used in economically significant fields as diverse as mechanical engineering, construction, or cargo logistics, for example. Data acquisition can be well integrated and fully automated into industrial workflows. They are often the surface measurement tool of choice to create digital representations of manufactured parts or structures of interest – a technique that the Directorate “Prosperity” of the European Commission’s Directorate-General for Research and Innovation identified as one of six enabling technologies supporting industry 5.0 [1]. But the fundamental output of scanning systems is, compared to classical dimensional measurements, not more than large amounts of 3D coordinate data points whose individual measurement uncertainty is, in general, to be considered worse than those of more classical approaches. A sophisticated large data volume post-process incorporating data fusion, partitioning, analysis and interpretation is then needed to extract the application specific product. A sound quantitative assessment of the uncertainty of the measurement product based on a reasonable time and analysis effort is considered the holy grail, enabling an even broader use also in more critical application fields. Fundamental advance in fields such as mechanical engineering, construction, or cargo logistics only be ensured, if generalisable, holistic and thus practical solutions for the metrological characterisation of the 3D scanning measurement can be achieved.

Point clouds are the intermediate product between the physical measurement and the feature measurement. In 3D scanning, routine scans can comprise millions of data points. Fusion or registration are further processing steps on the primary data. They are often captured from multiple scans made from different locations to unspecified surface locations. This introduces novel sources of uncertainty while potentially reducing the impact of the individual coordinate uncertainty on the final measurement result. Application of classical approaches to uncertainty modelling of these datasets lead to variance-covariance matrices with excessive dimensions of millions by millions and are impractical to handle. Less specialised users desperately need a more practical approach, possibly focussing on the metadata relevant for the final measurement uncertainty (e.g. the uncertainty of individual point of point clouds).

The property of interest is in most cases is not the point cloud itself, but derived characteristics such as inclination, volume, or manufacturing defects in in-line inspection, using either geometric or radiometric features. The amount of data generated is often challenging and requires dedicated computing architecture as well as sophisticated data-analysis approaches, often using proprietary software tools. Data analysis complexity, as well as limited interoperability of different laser scanning platforms are considered major challenges to a broader application, e.g., in architecture, engineering, and construction [2]. In their draft strategic research agenda, the EMN Advanced Manufacturing calls for further reference datasets for software validation in manufacturing QC and sees the need for data driven software validation against complex geometrical surfaces and components [3]. Since a good number of available computational software give differing results, reference data for validation are urgently needed, not only for industrial applications, but also at NMIs/DIs.

Context and state of the art

Due to the availability of several commercial and open-source software solutions, automated 3D reconstruction methods are becoming popular. Nonetheless, the metrological and reliability aspects of the resulting 3D scanning and modelling should not be ignored, particularly when the adoption of such solutions are not only dedicated to rapid 3D modelling and visualisation, but also to accurate scanning purposes related to tangible industrial applications. For many software solutions, performance in terms of computational costs is the prime target followed closely by visual impression. Stability and robustness to 3D scanning errors are of lesser concern. Stability means that the underlying numerical operations are numerically stable, while robustness refers to the software ability to handle extreme cases, for example systematic bias due to laser reflections when scanning cylindrical holes. Established software tests in coordinate metrology are currently based on small datasets with simple geometric features and limited number of data points. These tests do not sufficiently cover the challenges of a laser scanner generated 3D point cloud evaluation, for example where surfaces being captured are not Lambertian in their reflectance.

Within the selected European project 24DIT04-ScanClouDT, two approaches will be pursued to develop reference datasets for the metrological verification of point cloud processing software packages. Procedures and good practice will be developed to generate empirical reference datasets from experimental data generated employing dedicated artefacts and high-quality scanning systems. The focus of the project with respect to this objective will be on the development of a numerical generator of validation reference data for huge 3D datasets. Independent knowledge of the ground truth, for example from tactile probing systems, allows the evaluation of completeness and accuracy of respective software analyses. At least three verification datasets for the three different use cases (i.e., aerospace industry, logistics, and geodesy) will be developed. While the first experimental approach enables users to generate verification data close to the application by themselves according to established rules, the second approach can pave the way to a respective verification service provided by European NMIs.

Reference data generation and robust point cloud partitioning algorithms

Recent advances in 3D laser scanning systems allow complex objects and structures to be surveyed and reconstructed using a large set of scanned cloud points (millions of points). This thesis aims to investigate and make available robust reference methods for the whole trustworthy processing of dense scanned cloud points. The addressed and implemented reference processing methods could be validated on numerous reference datasets to ensure fusion, 3D reconstruction, partitioning, classification, evaluation, etc., with uncertainties well below the achieved scanning uncertainty. Reference data for validating reference, commercial and open-source software for dense cloud points processing defined by sets of data points for which the exact value of deviation is known with an associated uncertainty. The method for reference data generation enables the control of several parameters in the generated datasets including but not limited to, the density and distribution of the points, the value of the error/deviation, the number of datasets, the initial poses, etc. The aim of flawless reference data use is to make sure that the software returns correct values with an uncertainty below the nanometre level. Even if the establishment of a good reference dataset generator with an accuracy which is two to three times better than the expected software results is not trivial, it remains an essential and unavoidable task for the validation of software.

Partitioning is a fundamental operation defined in ISO GPS standard which aims at decomposing a part into independent features or surface portions for further processing and analysis. Default partitioning process is used to decompose the surface into independent features from invariance classes while non-default partitioning is used to create compound features [4]. Attributes of the point cloud such as curvature, normal, slippage, etc. are calculated in order to provide criteria for different methods [5]. Classical methods for default partitioning include edge detection, region growing, clustering, spectral analysis, shape fitting and statistical evaluation [6]. Then, non-default partitioning can be solved by invariance subgroup reasoning [7]. However, user interaction is often needed to improve the robustness and avoid over- or under-segmentations. With the rapid development of ANNs related techniques in computer vision, data-driven methods are developing increasingly for automated and efficient point cloud processing without user interaction [7] [8].

Specific objectives of the thesis

• To investigate partitioning strategies and software measurement standards. Literature will be reviewed on existing partitioning strategies and ISO 5436-2 2012 (in the field of surface texture as “Type F1” and “Type F2” software measurement standard, also called “softgauges”) for reference data generation. Research developed in EMPIR project 15SIB01 FreeFORM on reference software and the generation of reference data for validating min-max and least-squares fitting algorithms for complex surfaces (the subject of the ISO 10360-6:2001) will also be considered. Based on the conclusion of the literature review, the computational aim and mathematical formulations of the dense cloud points partitioning problem will be defined. The problem could be formulated as the minimisation of deviations between each point and their projection on the underlying surface. The main idea behind reference data generation here is to state the optimality conditions for the considered mathematical formulation and then derive datasets that perfectly meet these conditions.
• To investigate and develop robust reference algorithms for dense point clouds partitioning for complex geometries. 3D laser scanning systems provide dense point clouds and allow complex surfaces to be reconstructed and partitioned. To detect compound features and relations among complex surface patches, it is necessary to account for transition features such as chamfers and fillets which may not be explicitly represented in the nominal model for verification purposes. Non-default partitioning not related to invariance classes will also be investigated with data-driven methods for efficient and automated identification of complex compound features on the basis of semantic segmentation. While data-driven methods need no explicitly defined knowledge, large datasets need to be developed for the training of DL models. Moreover, hybrid method combining the advantages of various partitioning methods will be considered.
• To investigate and establish theoretical foundations on the assessment of the accuracy such as to distinguish between the numerical uncertainties associated with the software from those associated with the numerical standards. The proposed approach for generating reference data addresses the dual problem of the software: For predefined parameters, footpoints and normal, how to generate point deviations for which the objective function is minimum? Two optimality conditions are required to obtain a strict local minimum: (1) the Gradient should be equal to the null vector, and (2) the Hessian matrix should be strictly positive definite. It will be possible to introduce random noise and intentional geometric errors to the reference data to study the impact of synchronous and asynchronous errors on the processing chain. The accuracy of generated datasets will also be validated by calculating the difference between pre-defined and optimal parameters.
• To investigate and implement ‘degree of difficulty’ that aims at defining a quantity associated to each set of data. This quantity indicates at which extent the generated reference dataset ‘challenges’ the algorithm under test. Most of the relevant requirements will be summarised in a single metric (if possible), and the methodology for testing metrology software derived will consist of generating datasets with an increasing difficulty number. For each set, the output of the software under test will be assessed using performance measures. A survey involving at least 20 institutions both from within the consortium and the stakeholder community (such as software developers, large scale manufacturers and civil engineers) will be conducted. The aim is to achieve a response rate of at least 15 questionnaires. Afterwards, the diagram of performance measure against the degree of difficulty for each algorithm under test will infer conclusions about the fitness for purpose. The regions corresponding to reference algorithms, operating algorithms as well as rejected algorithms will be transposed in one graph and discussed with all project participants and the Stakeholder Committee. A wide selection of typical partitioning software solutions in relation to the project use cases in geodesy, logistics and aerospace will be investigated. Datasets will comprise classical geometries such as spheres, planes, cylinders and more complex geometries typical for the three use cases.
• To study the transferability of the software performance with respect to the reference data on real application results. For this, predictions based on the numerical validation results will be compared to scan results obtained from artefacts traceable to the SI definition of the metre of well-known dimensions and features. Experimental datasets representative of daily work will be submitted to the selected software. This will lead to methods and guidelines on the generation and establishment of empirical reference datasets physically collected by real scanners. Thus, compared to the uncertainty of nm gained by mathematical simulations, this will build the bridge to physical measurements acquired in reality having an uncertainty of a few mm or better.

Reference

[1] M. Breque et al., ‘Industry 5.0 – Towards a sustainable, human-centric and resilient European industry’, Publications Office of the European Union, https://data.europa.eu/doi/10.2777/308407 (2021)

[2] A. Waqar et al., ‘Complexities for adopting 3D laser scanners in the AEC industry: Structural equation modelling’, Applications in Engineering Science 16, 100160 (2023)

[3] ‘EMN for Advanced Manufacturing Key Industry Sector Topics: Draft Strategic Research Agenda December 2022’, https://www.euramet.org/european-metrology-networks/advanced-manufacturing/strategic-research-agenda

[4] ISO 18183-3 Geometrical product specifications (GPS) — Partitioning — Part 3: Methods used for Specification and Verification. International Organization for Standardization, Geneva (2021)

[5] A. Shamir, ‘A survey on mesh segmentation techniques’. Comput. Graph. Forum 27(6), 1539-1556 (2008)

[6] Anwer N., Scott P.J., Srinivasan, ‘Toward a Classification of Partitioning Operations for Standardization of Geometrical Product Specifications and Verification’, Procedia CIRP, 75, 325-330 (2018)

[7] Y. Qie, N. Anwer, ‘Toward non-default partitioning for compound feature identification in engineering design’, Procedia CIRP, 100, 852–857 (2021)

[8] Y. Qie, N. Anwer, ‘Invariance Class based Surface Reconstruction using CNN Transfer Learning’, Computer-Aided Design & Applications, 20(6), 1204-1220 (2023)

[9] Y. Qie, L. Qiao, N. Anwer, ‘A Framework for Curvature-Based CAD Mesh Partitioning’, Lecture Notes in Mechanical Engineering, 228-234 (2021)

La thèse se déroulera essentiellement au LNE de Paris et LURPA de l’ENS Paris-Saclay. Le cadre du projet européen (24DIT04-ScanClouDT) permettra des échanges scientifiques avec le consortium du projet ainsi que des déplacements en Europe sur la période de thèse.

Les candidats doivent avoir d'excellentes capacités d'analyse, un grand intérêt pour les modèles mathématiques pour la métrologie dimensionnelle, une expérience de la programmation et des méthodes numériques, et de solides compétences en communication.

La maîtrise de l'anglais écrit et oral est requise pour ce poste.

Une connaissance pratique des outils de calcul pour la métrologie dimensionnelle et la qualité est souhaitable.

Les candidats doivent être titulaires d'un master (ou équivalent) en ingénierie mécanique, en mathématiques appliquées ou en informatique, avec une solide expérience en analyse numérique, en optimisation ou en informatique, mais d'autres profils appropriés seront également pris en considération.