Convert 2D Pose to 3D Pose

Function Description

Converts 2D poses generated from orthogonal projection into 3D poses in the robot coordinate system or camera coordinate system by combining depth map information. The input 2D pose contains (x, y) coordinates and rotation angle θ around the Z axis. It maps to the depth image pixel position through orthogonal projection relationship, combines mask to filter valid depth values within a specified neighborhood and takes the mean to obtain Z. Finally outputs 3D pose.

Use Cases

Industrial robot grasping: 2D vision locates target objects, combines depth map to obtain accurate 3D poses.
Precision measurement: Such as solder joint height (Z value) detection and plane position (X/Y) correction for circuit boards.

Inputs

2D Pose List (Required): 2D pose list to be converted, containing (x, y, θ).
2D Mask (Required): 2D mask corresponding to the depth map, used to filter valid depth values.
Depth Map (Required): Depth map after orthogonal projection, used to calculate the Z value of the 3D pose.

Outputs

3D Pose List: Converted 3D pose list.

Parameters

Kernel Size for Searching Adjacent Depth Values

On the depth map, taking the target pixel as the center, collects valid depth values within a neighborhood of specified size combined with the mask and takes the mean as Z. The larger the neighborhood, the smoother the Z value, but the more susceptible to background interference; the smaller the neighborhood, the more sensitive to noise. Unit is pixel. For example, setting to 10 means searching a 10×10 neighborhood.