对3D对象进行形变分析

1，目的

分析3D实例对象相对标准参照物的形变。 一般用于质地较软的材质（例如橡胶，布料）查找，检查等。

标准参考模型
需匹配的实例：
形变后的模型：*

形变后的模型：

实例+形变后的模型

在这里插入图片描述

最终分析的结果(颜色编码表示，红色代表较大变形)：

2，关联的数学知识

2.1，右手坐标系

右手系(right-hand system) 是在空间中规定直角坐标系的方法之一。此坐标系中x轴，y轴和z轴的正方向是如下规定的：把右手放在原点的位置，使大拇指，食指和中指互成直角，把大拇指指向x轴的正方向，食指指向y轴的正方向时，中指所指的方向就是z轴的正方向。也可以按如下方法确定右手(左手)坐标系：如果当右手(左手)的大拇指指向第一个坐标轴(x轴)的正向，而其余手指以第二个轴(y轴)绕第一轴转动的方向握紧，就与第三个轴(z轴)重合，就称此坐标系为右手(左手)坐标系。
在这里插入图片描述

2.2，单位向量

单位向量(Unit Vector)：也被称为归一化向量(Normalized Vector)也就是长度为1的向量。

归一化向量(单元向量)关系：

$(\hat{e}_x)^2+(\hat{e}_y)^2+(\hat{e}_z)^2=1$
$\hat{e}_x,\hat{e}_y,\hat{e}_z$ 是单位向量在x axis，y axis，z axis上的分量。

2.3，向量反转

向量反转指的是向量的长度不变，但是向量的方向旋转180度。向量乘以-1即可得到一个长度相同方向相反的向量。
$-\vec{a}=\vec{a}.-1$

2.4，点积运算

点积运算（Dot Product）: 两个向量的乘积运算，它们的运算结果是一个标量。表达了两个向量之间的关系。

以下等式只对二维三维空间有效

点积运算：
$\vec{a}.\vec{b}=a_xb_x+a_yb_y+a_zb_z=|\vec{a}||\vec{b}|cos\theta$
当两向量的夹角为0度时：
$theta=0^o=>\vec{a}.\vec{b}=|\vec{a}||\vec{b}|$
当两向量垂直时：
$\perp b => \vec{a}.\vec{b}=0$
当两向量夹角为180度时
$\theta=180^o=>\vec{a}.\vec{b}.cos\theta=-|\vec{a}||\vec{b}|$
$\theta$ 为 $\vec{a}$ 与 $\vec{b}$ 的夹角，若当 $\vec{b}$ 为单元向量时（即长度为1）,可推导出
$\vec{a}.\vec{b}=a_xb_x+a_yb_y+a_zb_z=|\vec{a}||\vec{b}|cos\theta=|\vec{a}|.1.cos\theta=|\vec{a}|cos\theta$
在此种情况下可作出如下推断：
在a,b非零的前提下，点积如果为负，则a,b形成的角大于90度；如果为零，那么a,b垂直；如果为正，那么a,b形成的角为锐角。

3，代码分析

3.1，创建可形变的匹配模型

3.1.1，设置标准参考模型。

* 参考案例库：find_deformable_surface_model.hdev* 可变形表面匹配模型应用，适用于弹性物体（如橡胶，布料）的形变检测与定位。
* 
* 描述：
* 变形表面匹配模型将标准的模型 形变为与 3D 场景下的实例相识的模型* This example shows how to use HALCON's deformable surface-based
* 3D matching to find potentially deformed objects in 3D scenes.
* In this example, a CAD model is used as reference model. Note that it
* would also be possible, to created the reference model from (multiple)
* views of the object (for more details, see the HDevelop example program
* reconstruct_3d_object_model_for_matching.hdev).
* To extend the supported range of deformations, deformed variants
* of the 3D object model are added to the deformable surface model.
* Finally, with the created deformed surface model, the object is searched
* for in a number of 3D scenes.
* 
* The example demonstrates how to use the operators
*   create_deformable_surface_model, and
*   find_deformable_surface_model,
*   add_deformable_surface_model_sample,
*   get_deformable_surface_model_param, and
*   get_deformable_surface_matching_result.
* 
* Initialization
dev_update_off ()
* 配置显示参数
VisCamParam := [0.01,0,6e-6,6e-6,400,300,800,600]
VisPose1 := [0.00,0.01,0.62,122.0,357.0,120.0,0]
VisPose2 := [0.08,-0.77,0.35,298.0,353.0,315.0,0]
VisRefParName := ['alpha_0','color_0']
VisRefParValue := [0.75,'blue']
dev_close_window ()
dev_open_window_fit_size (0, 0, VisCamParam[|VisCamParam| - 2], VisCamParam[|VisCamParam| - 1], -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
* 
Message := ['This example shows how to use HALCON\'s deformable surface-based 3D matching to find potentially deformed objects in 3D scenes.',' ','In this example, a CAD model is used as reference model.','To extend the supported range of deformations, deformed variants of the 3D object model are added to the deformable surface model.','Finally, with the created deformed surface model, the object is searched for in a number of 3D scenes.']
MessageWrapped := regexp_replace(Message + ' ',['(.{0,60})\\s','replace_all'],'$1\n')
disp_message (WindowHandle, MessageWrapped, 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* 
* Define headings for the individual steps
Steps := 'Step 1: Create model'
Steps[1] := 'Step 2: Extend model (optional)'
Steps[2] := 'Step 3: Search for (deformed) instances of the object in 3D scenes'
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0], 'window', 12, 12, 'white', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* 
* =============================part01,创建可变形表面匹配模型* 1.1,读取 用作参照物的 3D 模型
read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_model', 'm', 'convert_to_triangles', 'true', ObjectModel3DReference, Status)

参照物模型ObjectModel3DReference

在这里插入图片描述

3.1.2，设置可变形表面模型的搜索点。

* 1.2,定义用于可变形表面模型搜索所需的搜索点
ReferencePointX := [4.2,4.2,4.2,-4.2,-4.2,-4.2] * 0.01
ReferencePointY := [7.75,0,-7.75,7.75,0,-7.75] * 0.01
ReferencePointZ := [3.0,3.0,3.0,3.0,3.0,3.0] * 0.01
* 
* Visualize the 3D object model together with the reference points
Title := 'Step 1: Create model\n \nCreate deformable surface model from a 3D object model of a\nsilicone baking mold (blue) and some reference points (green)'* 1.3，由搜索点组成的模型
gen_object_model_3d_from_points (ReferencePointX, ReferencePointY, ReferencePointZ, ObjectModel3DReferencePoints)
visualize_object_model_3d (WindowHandle, [ObjectModel3DReference,ObjectModel3DReferencePoints], VisCamParam, VisPose1, [VisRefParName,'color_1','point_size_1'], [VisRefParValue,'green',10], Title, [], [], PoseOut)

ObjectModel3DReference,ObjectModel3DReferencePoint呈现效果

在这里插入图片描述

3.1.3，创建可变形表面匹配模型

clear_object_model_3d (ObjectModel3DReferencePoints)
* 
* Create deformable surface model from the 3D object model* 1.4，创建可变形表面匹配模型
* stiffness:模型的刚性程度，控制模型在匹配中可以变形的程度
* 0.85:值越接近1模型的刚性越好（变形越小）,值越接近0模型越灵活（允许更大的变形）
create_deformable_surface_model (ObjectModel3DReference, 0.03, 'stiffness', 0.85, DeformableSurfaceModel)
* 
* Optionally, add the reference points that will be transformed to the
* geometry of the deformed search object.* 1.5，给变形表面匹配模型添加搜索的参考点
* 可选项，添加搜索的参考点，用于定义匹配过程中的关键位置（如抓取点，特征点）
add_deformable_surface_model_reference_point (DeformableSurfaceModel, ReferencePointX, ReferencePointY, ReferencePointZ, ReferencePointIndex)
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0:1], 'window', 12, 12, ['dim gray','white'], 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()

3.2，添加样本，扩大变形的支持范围（可选）

3.2.1，读取样本实例。

```
* ================================= part02，添加样本，扩大支持变形的支持范围（可选）
* 
* 可选项，将3D对象模型的变形实例添加到可变形表面模型
* 扩展变形的支持范围。
* 为此，使用上面创建的可变形表面模型在不同的场景中查找变形对象
* 从匹配结果中提取确定的变形，并随后添加到可变形表面模型中，以扩展支持范围
*的变形。
for Index := 1 to 3 by 1* * 2.1，读取3D对象实例read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_scene_' + Index$'02d', 'm', [], [], ObjectModel3DScene, Status)

ObjectModel3Dscene显示效果：

在这里插入图片描述

3.2.2，对样本进行预处理。

    * 2.2，消除背景，平滑，法线归一化，使匹配速度更快，鲁棒性更强。preprocess_object_model_3d (ObjectModel3DScene, 'true', 'true', 'true', ObjectModel3DSearchSceneForDef)

本地函数：preprocess_object_model_3d

* 预处理使程序更健壮，鲁棒性更好
* Define some parameters
* 用于分割背景的平面位姿
PlanePose := [0.047653983464843647,-0.0070959727421774418,0.93882498603198883,340.58369618439872,1.8517665823479257,0.0,0]
* 与平面的最小距离
MinDistanceToPlane := 0.015
* 断开连接域的最大两点距离
MaxDistanceBetweenPoints := 0.001
* 大于该阈值的点云保留
MinNumPoints := 1000
* 单位向量，归一性向量,这里表示指向Z轴负方向的向量，用于计算法向量的方向
NormalDirection := [0,0,-1]
* 移除背景与噪声
if (RemoveBackgroundPlane == 'true')* Create a plane from the given approximate plane posegen_plane_object_model_3d (PlanePose, [], [], ObjectModel3DPlane)* 计算点云各点到平面的距离visualize_object_model_3d (3600, [ObjectModel3DPlane,ObjectModel3D], [], [], ['color_0','color_1','alpha_0','disp_pose'], ['blue','green',0.8,'true'], ['分割平面与3D对象'], [], [], PoseOut)distance_object_model_3d (ObjectModel3D, ObjectModel3DPlane, [], 0, [], [])* 获取 3D 对象的点到平面的距离get_object_model_3d_params (ObjectModel3D, '&distance', Distances)* 去掉背景点云，噪声* predefined thresholdselect_points_object_model_3d (ObjectModel3D, '&distance', MinDistanceToPlane, max(Distances), ObjectModel3DThresholded)* 显示筛选后的点云visualize_object_model_3d (3600, [ObjectModel3DPlane,ObjectModel3DThresholded], [], [], ['color_0','color_1','alpha_0','disp_pose'], ['blue','green',0.8,'true'], ['分割平面与3D对象（去掉背景后）'], [], [], PoseOut)

ObjectModel3DPlane,bjectModel3D显示效果：

在这里插入图片描述

distance_object_model_3d (ObjectModel3D, ObjectModel3DPlane, [], 0, [], [])* 获取 3D 对象的点到平面的距离get_object_model_3d_params (ObjectModel3D, '&distance', Distances)* 去掉背景点云，噪声* predefined thresholdselect_points_object_model_3d (ObjectModel3D, '&distance', MinDistanceToPlane, max(Distances), ObjectModel3DThresholded)* 显示筛选后的点云visualize_object_model_3d (3600, [ObjectModel3DPlane,ObjectModel3DThresholded], [], [], ['color_0','color_1','alpha_0','disp_pose'], ['blue','green',0.8,'true'], ['分割平面与3D对象（去掉背景后）'], [], [], PoseOut)

ObjectModel3DPlane,ObjectModel3DThresholded显示效果：

在这里插入图片描述

* 断开连接域connection_object_model_3d (ObjectModel3DThresholded, 'distance_3d', MaxDistanceBetweenPoints, ObjectModel3DConnected)select_object_model_3d (ObjectModel3DConnected, 'num_points', 'and', MinNumPoints, |Distances|, ObjectModel3DSelected)union_object_model_3d (ObjectModel3DSelected, 'points_surface', UnionObjectModel3D)* clear_object_model_3d (ObjectModel3DThresholded)clear_object_model_3d (ObjectModel3DConnected)clear_object_model_3d (ObjectModel3DSelected)
elsecopy_object_model_3d (ObjectModel3D, 'all', UnionObjectModel3D)
endif
* *  visualize_object_model_3d (3600, [ObjectModel3DPlane,UnionObjectModel3D], [],[], ['color_0','color_1','alpha_0','disp_pose'], ['blue','green',0.8,'true'], ['分割平面与3D对象(去掉背景与噪声后)'], [], [], PoseOut1)
* 平滑对象模型
if (Smooth == 'true')* 平滑后的模型会基于新拓扑结构自动生成法向量，用于光照和渲染smooth_object_model_3d (UnionObjectModel3D, 'mls', [], [], ReferenceModelExt)clear_object_model_3d (UnionObjectModel3D)
elseReferenceModelExt := UnionObjectModel3D
endif
* 
visualize_object_model_3d (3600, ReferenceModelExt, [], [], ['color_0','disp_pose'], ['green','true'], ['3D对象平滑'], [], [], PoseOut2)

ReferenceModelExt显示效果：

在这里插入图片描述

* 将法线重新定向到预定义的方向
if (ReorientNormals == 'true')orient_normals_towards_direction (ReferenceModelExt, NormalDirection)
endif
return ()
本地函数：orient_normals_towards_direction
* 用于法线重定向的方法
* 平滑可以自动生成法向量，如果没有进行平滑则需要另行计算法向量
* surface_normals_object_model_3d (Smooth0, 'mls', [], [], Smooth0)get_object_model_3d_params (Smooth0, 'point_normal_x', NX)get_object_model_3d_params (Smooth0, 'point_normal_y', NY)get_object_model_3d_params (Smooth0, 'point_normal_z', NZ)
* 点积运算，法向量与单位向量的点积运算
* Dot>0则法向量与z axis 负方向的夹角小于90度
* Dot=0则法向量与z axis 负方向的夹角等于90度
* Dot<0 则法向量与z axis 负方向的夹角大于90度Dot := NX * Target[0] + NY * Target[1] + NZ * Target[2]
* 与z axis 负方向的夹角小于等于90度时为1，大于90度时为-1
* 向量*-1则向量方向旋转180度,所以如果对应的元素为-1时表示该法向量需要旋转180度Sign := (Dot [>=] 0) * 2 - 1tuple_find (Sign, 1, Indices)
* 设置法向量set_object_model_3d_attrib_mod (Smooth0, 'point_normal_' + ['x','y','z'], [], [NX,NY,NZ] * [Sign,Sign,Sign])return ()
释疑解惑：

‌变量含义‌

[NX, NY, NZ]：表示第一个向量的三个分量（如法向量）
Target[0], Target[1], Target[2]：表示第二个向量（目标向量）的三个分量
Dot：存储点积结果的标量变量

‌几何意义‌
点积结果可反映两向量的方向关系：

正值表示两向量夹角小于90°（方向相近）
负值表示夹角大于90°（方向相反）
零值表示两向量垂直
在用户代码中，该结果用于后续法向量方向调整（通过Sign变量）。

‌物理意义‌：检测法向量 [NX, NY, NZ] 的 ‌Z 分量方向‌：
- 若 NZ > 0（法向量朝 Z 正方向），则 Dot < 0（与 Target 方向相反）。
- 若 NZ < 0（法向量朝 Z 负方向），则 Dot > 0（与 Target 方向相同）。
‌用途‌：通过 Sign 变量调整法向量方向，使其统一朝向 Target 方向（或相反方向）。

Target := [0, 0, -1] 表示一个‌单位向量‌，其方向指向 ‌Z 轴的负方向‌（即垂直于 XY 平面向下）。具体解析如下：

‌1. 数学定义‌

‌**[0, 0, -1]**‌ 是一个三维向量，其分量分别为：
‌X 分量 = 0‌（不朝 X 方向延伸）
‌Y 分量 = 0‌（不朝 Y 方向延伸）
‌Z 分量 = -1‌（完全指向 Z 轴负方向）

‌单位向量‌：该向量的长度为 1（即 √(0² + 0² + (-1)²) = 1），常用于表示‌纯方向‌而非位置或大小。

2. 几何意义‌

在常见的 ‌右手坐标系‌（如 OpenGL、Unity）中：
‌Z 轴正方向‌（[0,0,1]）通常指向屏幕‌外‌（或观察方向）。
‌Z 轴负方向‌（[0,0,-1]）则指向屏幕‌内‌（或远离观察者）。

在 ‌左手坐标系‌（如 DirectX）中，Z 轴方向可能相反，需根据具体应用场景确认。

3. 典型应用场景‌

‌法向量统一‌
在三维建模中，确保所有面法线朝向一致（如全部指向模型外部），避免光照计算错误。

‌**背面剔除（Backface Culling）**‌
在渲染时，通过法向量与视线方向（如 [0,0,-1]）的点积判断是否剔除背面多边形。

‌碰撞检测‌
判断物体表面是否朝向某个特定方向（如重力方向 [0,-1,0]）。

3.2.3，获取根据样本而变形的模型。


* Visualize the preprocessed 3D scene that shows a deformed instance of the objectif (Index == 1)dev_clear_window ()Title := 'Step 2: Extend model\n \nProvide 3D scenes with deformed variants of the 3D object model'visualize_object_model_3d (WindowHandle, ObjectModel3DSearchSceneForDef, VisCamParam, VisPose2, [], [], Title, [], [], PoseOut)endif* * Find the position of the reference model in the 3D sceneTitle := 'Step 2: Extend model\n \nProcess scene ' + Index + ' of 3...'dev_clear_window ()disp_message (WindowHandle, Title, 'window', 12, 12, 'black', 'true')* 2.3，在3D场景中搜索可变形表面匹配的最佳匹配实例，适用于弹性物体（如橡胶，布料）的形变检测与定位find_deformable_surface_model (DeformableSurfaceModel, ObjectModel3DSearchSceneForDef, 0.03, 0, [], [], Score, DeformableSurfaceMatchingResult)* * Visualize matching resultsTitle := 'Step 2: Extend model\n \nDeformed reference object model (blue) aligned with deformed\nobject in 3D scene (white)'* 2.4,返回变形后的 3D 模型get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_model', 0, ObjectModel3DDeformed)visualize_object_model_3d (WindowHandle, [ObjectModel3DDeformed,ObjectModel3DSearchSceneForDef], VisCamParam, PoseOut, VisRefParName, VisRefParValue, Title, [], [], PoseOut)*

ObjectModel3DDeformed,ObjectModel3DSearchSceneForDef显示效果：

在这里插入图片描述

3.2.4，将形变样本添加到可变形表面模型。

  dev_clear_window ()* 2.5,获取变形样本(点云形式呈现)disp_message (WindowHandle, 'Step 2: Extend model\n \nAdd deformation from found instance to deformable surface model...', 'window', 12, 12, 'black', 'true')* 获取的是点云样式。样本均是点云get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_sampled_model', 0, ObjectModel3DDeformedSampled)* 2.6，向可变形表面匹配模型添加样本数据add_deformable_surface_model_sample (DeformableSurfaceModel, ObjectModel3DDeformedSampled)* visualize_object_model_3d (WindowHandle,  ObjectModel3DDeformedSampled, [], [], [], [], [], [], [], PoseOut1)* Cleanupclear_object_model_3d (ObjectModel3DScene)clear_object_model_3d (ObjectModel3DSearchSceneForDef)clear_deformable_surface_matching_result (DeformableSurfaceMatchingResult)clear_object_model_3d (ObjectModel3DDeformed)clear_object_model_3d (ObjectModel3DDeformedSampled)
endfor
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0:2], 'window', 12, 12, ['dim gray','dim gray','white'], 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
*

ObjectModel3DDeformedSampled显示效果(样本总是为点云形式)

在这里插入图片描述

3.3，在3D场景中匹配实例，颜色渲染形变区域。

3.3.1，读取并预处理3D场景中的实例。

* 在不同的3D场景中找到对象，即使它看起来变形了
for Index := 4 to 7 by 1* dev_clear_window ()disp_message (WindowHandle, 'Step 3: Search for deformed model instances\n \nLoad scene ' + (Index - 3) + ' of 4...', 'window', 12, 12, 'black', 'true')* * 3.1, 读取包含（变形）对象实例的3D场景read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_scene_' + Index$'02d', 'm', [], [], ObjectModel3DScene, Status)**todo:如果对象模型缺少triangles与surface_attributes等关键信息会被降级为变量窗口？* The background of the 3D scene is not eliminated to show that the* (deformed) instance of the object can be found in the complete scene* 3.2,3D场景的背景没有被消除，以显示（变形）对象的实例可以在完整的场景中找到preprocess_object_model_3d (ObjectModel3DScene, 'false', 'true', 'true', ObjectModel3DSearchScene)* * Visualize the 3D scenedev_clear_window ()Title := 'Step 3: Search for deformed model instances\n \n3D scene in which the deformed object is searched for'* 3.3,显示增强后的实例visualize_object_model_3d (WindowHandle, ObjectModel3DSearchScene, VisCamParam, VisPose2, [], [], Title, [], [], PoseOut)*

ObjectModel3DSearchScene显示效果

在这里插入图片描述

3.3.2，获取实例对应的变形模型。

 * 3.4，进行基于表面的可变形匹配find_deformable_surface_model (DeformableSurfaceModel, ObjectModel3DSearchScene, 0.03, 0, [], [], Score, DeformableSurfaceMatchingResult)* * 3.5，获取变形的模型get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_model', 0, ObjectModel3DDeformed)* * 3.6，获取变形后6个搜索点的位置get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_x', 'all', ReferencePointXDeformed)get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_y', 'all', ReferencePointYDeformed)get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_z', 'all', ReferencePointZDeformed)* * Visualize the matching resultgen_object_model_3d_from_points (ReferencePointXDeformed, ReferencePointYDeformed, ReferencePointZDeformed, ObjectModel3DReferencePointsDeformed)Title := 'Step 3: Search for deformed model instances\n \nMatched and deformed reference object model (blue)\nand deformed reference points (green)'* 3.7，显示匹配结果（变形后的3D模型+变形后的6个搜索点+3D场景中的实例）* ObjectModel3DDeformed：变形后的3D模型；ObjectModel3DSearchScene：3D场景中的实例;* ObjectModel3DReferencePointsDeformed:形变后的6个模型搜索点组成的模型visualize_object_model_3d (WindowHandle, [ObjectModel3DDeformed,ObjectModel3DSearchScene,ObjectModel3DReferencePointsDeformed], VisCamParam, PoseOut, [VisRefParName,'color_1','color_2','point_size_2'], [VisRefParValue,'white','green',10], Title, [], [], PoseOut)*

ObjectModel3DDeformed,ObjectModel3DSearchScene,ObjectModel3DReferencePointsDeformed显示效果

在这里插入图片描述

3.3.3，计算形变值，渲染形变效果，呈现实例形变模型

 * 3.8，获取相对于匹配模型的位姿get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'rigid_pose', 0, Pose)* 3.9 ，匹配模型刚体变化rigid_trans_object_model_3d (ObjectModel3DReference, Pose, ObjectModel3DRigidTrans)* * 3.10，计算标准模型与形变模型之间的距离，为标识出变形程度作准备。* ObjectModel3DReference：标准未发生形变的模型；ObjectModel3DDeformed：发生形变的模型* ObjectModel3DRigidTrans：已与 3D场景实例对齐的标准模型get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_x', DRX)get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_y', DRY)get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_z', DRZ)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_x', RTX)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_y', RTY)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_z', RTZ)Deformation := sqrt((DRX - RTX) * (DRX - RTX) + (DRY - RTY) * (DRY - RTY) + (DRZ - RTZ) * (DRZ - RTZ))* 3.11，计算出变形程度DeformationClipped := min2(Deformation,gen_tuple_const(|Deformation|,0.02))* Create a new 3D object model for the visualization of the deformations,* add one single point with a special deformation, and set calculated deformations* with negative sign to use the part from green to red defined in the lut 'color1'*创建一个新的3D对象模型的可视化变形，*添加一个特殊的变形单点，并设置计算变形*带负号表示使用表“color1”中定义的从绿色到红色的部分* 3.12，显示变形后的模型，按照变形大小进行颜色渲染* 返回模型的几何中心坐标get_object_model_3d_params (ObjectModel3DRigidTrans, 'center', Center)gen_object_model_3d_from_points ([DRX,Center[0]], [DRY,Center[1]], [DRZ,Center[2]], ObjectModel3DVisDeformations)
*     gen_object_model_3d_from_points ([DRX], [DRY], [DRZ], ObjectModel3DVisDeformations)get_object_model_3d_params (ObjectModel3DReference, 'triangles', Triangles)set_object_model_3d_attrib_mod (ObjectModel3DVisDeformations, 'triangles', [], Triangles)* '&deformation'表示设置的是变形属性，属于扩展属性类型。* 'points'指定将属性附加到点数据上（而非多边形或三角形等）*  [-DeformationClipped,0.02]：属性值数组，第一个元素应是变形量数据，第二个0.02表示变形缩放系数set_object_model_3d_attrib_mod (ObjectModel3DVisDeformations, '&deformation', 'points', [-DeformationClipped,0.02])Title := 'Step 3: Search for deformed model instances\n \n变形的参考对象模型\n（用颜色编码表示，红色代表较大变形）'visualize_object_model_3d (WindowHandle, ObjectModel3DVisDeformations, VisCamParam, PoseOut, ['intensity_0','lut_0'], ['&deformation','color1'], Title, [], [], PoseOut)* * Cleanupclear_object_model_3d (ObjectModel3DScene)clear_object_model_3d (ObjectModel3DSearchScene)clear_deformable_surface_matching_result (DeformableSurfaceMatchingResult)clear_object_model_3d (ObjectModel3DDeformed)clear_object_model_3d (ObjectModel3DReferencePointsDeformed)clear_object_model_3d (ObjectModel3DRigidTrans)clear_object_model_3d (ObjectModel3DVisDeformations)
endfor
clear_object_model_3d (ObjectModel3DReference)
clear_deformable_surface_model (DeformableSurfaceModel)

ObjectModel3DVisDeformations显示效果

可变形表面匹配模型，通过查找搜索点与3D场景实例匹配，生成3D场景实例的形变模型。

在这里插入图片描述

4，完整代码

* 参考案例库：find_deformable_surface_model.hdev* 可变形表面匹配模型应用，适用于弹性物体（如橡胶，布料）的形变检测与定位。
* 
* 描述：
* 变形表面匹配模型将标准的模型 形变为与 3D 场景下的实例相识的模型* This example shows how to use HALCON's deformable surface-based
* 3D matching to find potentially deformed objects in 3D scenes.
* In this example, a CAD model is used as reference model. Note that it
* would also be possible, to created the reference model from (multiple)
* views of the object (for more details, see the HDevelop example program
* reconstruct_3d_object_model_for_matching.hdev).
* To extend the supported range of deformations, deformed variants
* of the 3D object model are added to the deformable surface model.
* Finally, with the created deformed surface model, the object is searched
* for in a number of 3D scenes.
* 
* The example demonstrates how to use the operators
*   create_deformable_surface_model, and
*   find_deformable_surface_model,
*   add_deformable_surface_model_sample,
*   get_deformable_surface_model_param, and
*   get_deformable_surface_matching_result.
* 
* Initialization
dev_update_off ()
* 配置显示参数
VisCamParam := [0.01,0,6e-6,6e-6,400,300,800,600]
VisPose1 := [0.00,0.01,0.62,122.0,357.0,120.0,0]
VisPose2 := [0.08,-0.77,0.35,298.0,353.0,315.0,0]
VisRefParName := ['alpha_0','color_0']
VisRefParValue := [0.75,'blue']
dev_close_window ()
dev_open_window_fit_size (0, 0, VisCamParam[|VisCamParam| - 2], VisCamParam[|VisCamParam| - 1], -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
* 
Message := ['This example shows how to use HALCON\'s deformable surface-based 3D matching to find potentially deformed objects in 3D scenes.',' ','In this example, a CAD model is used as reference model.','To extend the supported range of deformations, deformed variants of the 3D object model are added to the deformable surface model.','Finally, with the created deformed surface model, the object is searched for in a number of 3D scenes.']
MessageWrapped := regexp_replace(Message + ' ',['(.{0,60})\\s','replace_all'],'$1\n')
disp_message (WindowHandle, MessageWrapped, 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* 
* Define headings for the individual steps
Steps := 'Step 1: Create model'
Steps[1] := 'Step 2: Extend model (optional)'
Steps[2] := 'Step 3: Search for (deformed) instances of the object in 3D scenes'
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0], 'window', 12, 12, 'white', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* 
* =============================part01,创建可变形表面匹配模型* 1.1,读取 用作参照物的 3D 模型
read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_model', 'm', 'convert_to_triangles', 'true', ObjectModel3DReference, Status)
* 判断参考对象是否有法线
get_object_model_3d_params (ObjectModel3DReference, 'has_point_normals', ParamValue)
* Define reference points that will be transformed into the search scene.
* Here, points where the silicone baking mold might be grasped
* with a vacuum cup are defined. Note that it is not necessary that
* the reference points lie on the surface of the object.* 1.2,定义用于可变形表面模型搜索所需的搜索点
ReferencePointX := [4.2,4.2,4.2,-4.2,-4.2,-4.2] * 0.01
ReferencePointY := [7.75,0,-7.75,7.75,0,-7.75] * 0.01
ReferencePointZ := [3.0,3.0,3.0,3.0,3.0,3.0] * 0.01
* 
* Visualize the 3D object model together with the reference points
Title := 'Step 1: Create model\n \nCreate deformable surface model from a 3D object model of a\nsilicone baking mold (blue) and some reference points (green)'* 1.3，由搜索点组成的模型
gen_object_model_3d_from_points (ReferencePointX, ReferencePointY, ReferencePointZ, ObjectModel3DReferencePoints)
visualize_object_model_3d (WindowHandle, [ObjectModel3DReference,ObjectModel3DReferencePoints], VisCamParam, VisPose1, [VisRefParName,'color_1','point_size_1'], [VisRefParValue,'green',10], Title, [], [], PoseOut)
clear_object_model_3d (ObjectModel3DReferencePoints)
* 
* Create deformable surface model from the 3D object model* 1.4，创建可变形表面匹配模型
* stiffness:模型的刚性程度，控制模型在匹配中可以变形的程度
* 0.85:值越接近1模型的刚性越好（变形越小）,值越接近0模型越灵活（允许更大的变形）
create_deformable_surface_model (ObjectModel3DReference, 0.03, 'stiffness', 0.85, DeformableSurfaceModel)
* 
* Optionally, add the reference points that will be transformed to the
* geometry of the deformed search object.* 1.5，给变形表面匹配模型添加搜索的参考点
* 可选项，添加搜索的参考点，用于定义匹配过程中的关键位置（如抓取点，特征点）
add_deformable_surface_model_reference_point (DeformableSurfaceModel, ReferencePointX, ReferencePointY, ReferencePointZ, ReferencePointIndex)
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0:1], 'window', 12, 12, ['dim gray','white'], 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()* ================================= part02，添加样本，扩大支持变形的支持范围（可选）
* 
* 可选项，将3D对象模型的变形实例添加到可变形表面模型
* 扩展变形的支持范围。
* 为此，使用上面创建的可变形表面模型在不同的场景中查找变形对象
* 从匹配结果中提取确定的变形，并随后添加到可变形表面模型中，以扩展支持范围
*的变形。
for Index := 1 to 3 by 1* * 2.1，读取3D对象实例read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_scene_' + Index$'02d', 'm', [], [], ObjectModel3DScene, Status)* * 2.2，消除背景，平滑，法线归一化，使匹配速度更快，鲁棒性更强。preprocess_object_model_3d (ObjectModel3DScene, 'true', 'true', 'true', ObjectModel3DSearchSceneForDef)* * * Visualize the preprocessed 3D scene that shows a deformed instance of the objectif (Index == 1)dev_clear_window ()Title := 'Step 2: Extend model\n \nProvide 3D scenes with deformed variants of the 3D object model'visualize_object_model_3d (WindowHandle, ObjectModel3DSearchSceneForDef, VisCamParam, VisPose2, [], [], Title, [], [], PoseOut)endif* * Find the position of the reference model in the 3D sceneTitle := 'Step 2: Extend model\n \nProcess scene ' + Index + ' of 3...'dev_clear_window ()disp_message (WindowHandle, Title, 'window', 12, 12, 'black', 'true')* 2.3，在3D场景中搜索可变形表面匹配的最佳匹配实例，适用于弹性物体（如橡胶，布料）的形变检测与定位find_deformable_surface_model (DeformableSurfaceModel, ObjectModel3DSearchSceneForDef, 0.03, 0, [], [], Score, DeformableSurfaceMatchingResult)* * Visualize matching resultsTitle := 'Step 2: Extend model\n \nDeformed reference object model (blue) aligned with deformed\nobject in 3D scene (white)'* 2.4,返回变形后的 3D 模型get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_model', 0, ObjectModel3DDeformed)visualize_object_model_3d (WindowHandle, [ObjectModel3DDeformed,ObjectModel3DSearchSceneForDef], VisCamParam, PoseOut, VisRefParName, VisRefParValue, Title, [], [], PoseOut)* visualize_object_model_3d (WindowHandle, ObjectModel3DDeformed, [], [], ['color_0'], ['blue'], [], [], [], PoseOut2)* Get the deformed variant of the sampled reference model and add it as training sampledev_clear_window ()* 2.5,获取变形样本(点云形式呈现)disp_message (WindowHandle, 'Step 2: Extend model\n \nAdd deformation from found instance to deformable surface model...', 'window', 12, 12, 'black', 'true')get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_sampled_model', 0, ObjectModel3DDeformedSampled)* 2.6，向可变形表面匹配模型添加样本数据add_deformable_surface_model_sample (DeformableSurfaceModel, ObjectModel3DDeformedSampled)* visualize_object_model_3d (WindowHandle,  ObjectModel3DDeformedSampled, [], [], [], [], [], [], [], PoseOut1)* Cleanupclear_object_model_3d (ObjectModel3DScene)clear_object_model_3d (ObjectModel3DSearchSceneForDef)clear_deformable_surface_matching_result (DeformableSurfaceMatchingResult)clear_object_model_3d (ObjectModel3DDeformed)clear_object_model_3d (ObjectModel3DDeformedSampled)
endfor
* 
* Introduce next step
dev_clear_window ()
disp_message (WindowHandle, Steps[0:2], 'window', 12, 12, ['dim gray','dim gray','white'], 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* 
*=====================================  part03,不同的3D场景中匹配对象，按照差异大小进行颜色渲染* 在不同的3D场景中找到对象，即使它看起来变形了
for Index := 4 to 7 by 1* dev_clear_window ()disp_message (WindowHandle, 'Step 3: Search for deformed model instances\n \nLoad scene ' + (Index - 3) + ' of 4...', 'window', 12, 12, 'black', 'true')* * 3.1, 读取包含（变形）对象实例的3D场景read_object_model_3d ('silicone_baking_mold/silicone_baking_mold_3d_scene_' + Index$'02d', 'm', [], [], ObjectModel3DScene, Status)**todo:如果对象模型缺少triangles与surface_attributes等关键信息会被降级为变量窗口？* The background of the 3D scene is not eliminated to show that the* (deformed) instance of the object can be found in the complete scene* 3.2,3D场景的背景没有被消除，以显示（变形）对象的实例可以在完整的场景中找到preprocess_object_model_3d (ObjectModel3DScene, 'false', 'true', 'true', ObjectModel3DSearchScene)* * Visualize the 3D scenedev_clear_window ()Title := 'Step 3: Search for deformed model instances\n \n3D scene in which the deformed object is searched for'* 3.3,显示增强后的对象实例的3D场景visualize_object_model_3d (WindowHandle, ObjectModel3DSearchScene, VisCamParam, VisPose2, [], [], Title, [], [], PoseOut)* * 3.4，进行基于表面的可变形匹配find_deformable_surface_model (DeformableSurfaceModel, ObjectModel3DSearchScene, 0.03, 0, [], [], Score, DeformableSurfaceMatchingResult)* * 3.5，获取变形的模型get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'deformed_model', 0, ObjectModel3DDeformed)* * 3.6，获取变形后6个搜索点的位置get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_x', 'all', ReferencePointXDeformed)get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_y', 'all', ReferencePointYDeformed)get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'reference_point_z', 'all', ReferencePointZDeformed)* * Visualize the matching resultgen_object_model_3d_from_points (ReferencePointXDeformed, ReferencePointYDeformed, ReferencePointZDeformed, ObjectModel3DReferencePointsDeformed)Title := 'Step 3: Search for deformed model instances\n \nMatched and deformed reference object model (blue)\nand deformed reference points (green)'* 3.7，显示匹配结果（变形后的3D模型+变形后的6个搜索点+3D场景中的实例）* ObjectModel3DDeformed：变形后的3D模型；ObjectModel3DSearchScene：3D场景中的实例;* ObjectModel3DReferencePointsDeformed:形变后的6个模型搜索点组成的模型visualize_object_model_3d (WindowHandle, [ObjectModel3DDeformed,ObjectModel3DSearchScene,ObjectModel3DReferencePointsDeformed], VisCamParam, PoseOut, [VisRefParName,'color_1','color_2','point_size_2'], [VisRefParValue,'white','green',10], Title, [], [], PoseOut)* * Visualize the deformations* 3.8，获取相对于匹配模型的位姿get_deformable_surface_matching_result (DeformableSurfaceMatchingResult, 'rigid_pose', 0, Pose)* 3.9 ，匹配模型刚体变化rigid_trans_object_model_3d (ObjectModel3DReference, Pose, ObjectModel3DRigidTrans)* * 3.10，计算标准模型与形变模型之间的距离，为标识出变形程度作准备。* ObjectModel3DReference：标准未发生形变的模型；ObjectModel3DDeformed：发生形变的模型* ObjectModel3DRigidTrans：已与 3D场景实例对齐的标准模型get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_x', DRX)get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_y', DRY)get_object_model_3d_params (ObjectModel3DDeformed, 'point_coord_z', DRZ)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_x', RTX)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_y', RTY)get_object_model_3d_params (ObjectModel3DRigidTrans, 'point_coord_z', RTZ)Deformation := sqrt((DRX - RTX) * (DRX - RTX) + (DRY - RTY) * (DRY - RTY) + (DRZ - RTZ) * (DRZ - RTZ))* 3.11，计算出变形程度DeformationClipped := min2(Deformation,gen_tuple_const(|Deformation|,0.02))* Create a new 3D object model for the visualization of the deformations,* add one single point with a special deformation, and set calculated deformations* with negative sign to use the part from green to red defined in the lut 'color1'*创建一个新的3D对象模型的可视化变形，*添加一个特殊的变形单点，并设置计算变形*带负号表示使用表“color1”中定义的从绿色到红色的部分* 3.12，显示变形后的模型，按照变形大小进行颜色渲染* 返回模型的几何中心坐标get_object_model_3d_params (ObjectModel3DRigidTrans, 'center', Center)gen_object_model_3d_from_points ([DRX,Center[0]], [DRY,Center[1]], [DRZ,Center[2]], ObjectModel3DVisDeformations)
*     gen_object_model_3d_from_points ([DRX], [DRY], [DRZ], ObjectModel3DVisDeformations)get_object_model_3d_params (ObjectModel3DReference, 'triangles', Triangles)set_object_model_3d_attrib_mod (ObjectModel3DVisDeformations, 'triangles', [], Triangles)* '&deformation'表示设置的是变形属性，属于扩展属性类型。* 'points'指定将属性附加到点数据上（而非多边形或三角形等）*  [-DeformationClipped,0.02]：属性值数组，第一个元素应是变形量数据，第二个0.02表示变形缩放系数set_object_model_3d_attrib_mod (ObjectModel3DVisDeformations, '&deformation', 'points', [-DeformationClipped,0.02])Title := 'Step 3: Search for deformed model instances\n \n变形的参考对象模型\n（用颜色编码表示，红色代表较大变形）'visualize_object_model_3d (WindowHandle, ObjectModel3DVisDeformations, VisCamParam, PoseOut, ['intensity_0','lut_0'], ['&deformation','color1'], Title, [], [], PoseOut)* * Cleanupclear_object_model_3d (ObjectModel3DScene)clear_object_model_3d (ObjectModel3DSearchScene)clear_deformable_surface_matching_result (DeformableSurfaceMatchingResult)clear_object_model_3d (ObjectModel3DDeformed)clear_object_model_3d (ObjectModel3DReferencePointsDeformed)clear_object_model_3d (ObjectModel3DRigidTrans)clear_object_model_3d (ObjectModel3DVisDeformations)
endfor
clear_object_model_3d (ObjectModel3DReference)
clear_deformable_surface_model (DeformableSurfaceModel)