cvtkit.geometry
Module including geometric routines.
depth_to_normal(view, depth)
view: view camera depth: depthmap
essential_from_features(src_image_file, tgt_image_file, K)
Computes the essential matrix between two images using image features.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_image_file
|
str
|
Input file for the source image. |
required |
tgt_image_file
|
str
|
Input file for the target image. |
required |
K
|
ndarray
|
Intrinsics matrix of the two cameras (assumed to be constant between views). |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
The essential matrix betweent the two image views. |
fundamental_from_features(src_image_file, tgt_image_file)
Computes the fundamental matrix between two images using image features.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_image_file
|
str
|
Input file for the source image. |
required |
tgt_image_file
|
str
|
Input file for the target image. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
The fundamental matrix betweent the two image views. |
geometric_consistency_error(src_depth, src_cam, tgt_depth, tgt_cam)
Computes the geometric consistency error between a source and target depth map.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_depth
|
NDArray[Any]
|
Depth map for the source view. |
required |
src_cam
|
NDArray[Any]
|
Camera parameters for the source depth map viewpoint. |
required |
tgt_depth
|
NDArray[Any]
|
Depth map for the target view. |
required |
tgt_cam
|
NDArray[Any]
|
Camera parameters for the target depth map viewpoint. |
required |
Returns:
| Type | Description |
|---|---|
NDArray[Any]
|
The binary consistency mask encoding depth consensus between source and target depth maps. |
homography(src_image_file, tgt_image_file)
Computes a homography transformation between two images using image features.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_image_file
|
str
|
Input file for the source image. |
required |
tgt_image_file
|
str
|
Input file for the target image. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
The homography matrix to warp the target image to the source image. |
homography_warp(features, intrinsics, extrinsics, hypotheses, group_channels, vwa_net=None, view_weights=None)
Performs homography warping to create a Plane Sweeping Volume (PSV). Parameters: cfg: Configuration dictionary containing configuration parameters. features: Feature maps to be warped into a PSV. intrinsics: Intrinsics matrices for all views. extrinsics: Extrinsics matrices for all views. hypotheses: Depth hypotheses to use for homography warping. group_channels: Feature channel sizes used in group-wise correlation (GWC). vwa_net: Network used for visibility weighting. vis_weights: Pre-computed visibility weights. virtual: If True, reference camera is not used in computing feature correlation (virtual reference camera)
Returns:
| Type | Description |
|---|---|
|
The Plane Sweeping Volume computed via feature matching cost. |
homography_warp_var(features, ref_in, src_in, ref_ex, src_ex, depth_hypos)
Performs homography warping to create a Plane Sweeping Volume (PSV). Parameters: features: Feature maps to be warped into a PSV. level: Current feature resolution level. ref_in: Reference view intrinsics matrix. src_in: Source view intrinsics matrices. ref_ex: Reference view extrinsics matrix. src_ex: Source view extrinsics matrices. depth_hypos: Depth hypotheses to use for homography warping.
Returns:
| Type | Description |
|---|---|
|
The Plane Sweeping Volume computed via feature matching cost. |
match_features(src_image, tgt_image, max_features=500)
Computer matching ORB features between a pair of images.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_image
|
ndarray
|
The source image to compute and match features. |
required |
tgt_image
|
ndarray
|
The target image to compute and match features. |
required |
max_features
|
int
|
The maximum number of features to retain. |
500
|
Returns:
| Name | Type | Description |
|---|---|---|
src_points |
ndarray
|
The set of matched point coordinates for the source image. |
tgt_points |
ndarray
|
The set of matched point coordinates for the target image. |
plane_coords(K, P, depth_hypos, H, W)
Batched PyTorch version
project_depth_map(depth, KP_inv)
Projects a depth map into a list of 3D points
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
depth
|
Tensor
|
Input depth map to project. |
required |
cam
|
Camera parameters for input depth map. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
A float Tensor of 3D points corresponding to the projected depth values. |
project_renderer(renderer, K, P, width, height)
Projects the scene in an Open3D Offscreen Renderer to the 2D image plane.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
renderer
|
OffscreenRenderer
|
Geometric scene to be projected. |
required |
K
|
ndarray
|
Camera intrinsic parameters. |
required |
P
|
ndarray
|
Camera extrinsic parameters. |
required |
width
|
float
|
Desired image width. |
required |
height
|
float
|
Desired image height. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
The rendered image for the scene at the specified camera viewpoint. |
psv(cfg, images, intrinsics, extrinsics, depth_hypos)
Performs homography warping to create a Plane Sweeping Volume (PSV). Parameters: cfg: Configuration dictionary containing configuration parameters. images: image maps to be warped into a PSV. intrinsics: intrinsics matrices. extrinsics: extrinsics matrices. depth_hypos: Depth hypotheses to use for homography warping.
Returns:
| Type | Description |
|---|---|
|
The Plane Sweeping Volume computed via feature matching cost. |
rigid_transform(points, transform)
Apply's a rigid transform to a collection of 3D points.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
NDArray[Any]
|
Array of 3D points of size [N x 3] |
required |
transform
|
NDArray[Any]
|
rigid transform to be pre-multiplied of size [4x4] |
required |
Returns:
| Type | Description |
|---|---|
NDArray[Any]
|
The transformed 3D points. |
sample_volume(volume, z_vals, coords, H, W, near_depth, far_depth, inv_depth)
Parameters:
Returns:
soft_hypothesis(data, target_hypo, focal_length, min_hypo, max_hypo, M, delta_in=1)
Parameters:
Returns:
soft_hypothesis_numpy(data, target_hypo, focal_length, min_hypo, max_hypo, M=1, delta_in=1)
Parameters:
Returns:
uniform_hypothesis(cfg, device, batch_size, depth_min, depth_max, img_height, img_width, planes, inv_depth=False, bin_format=False)
Parameters:
Returns:
visibility(depths, K, Ps, vis_th, levels=4)
Parameters:
Returns:
visibility_mask(src_depth, src_cam, depth_files, cam_files, src_ind=-1, pixel_th=0.1)
Computes a visibility mask between a provided source depth map and list of target depth maps.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
src_depth
|
ndarray
|
Depth map for the source view. |
required |
src_cam
|
ndarray
|
Camera parameters for the source depth map viewpoint. |
required |
depth_files
|
List[str]
|
List of target depth maps. |
required |
cam_files
|
List[str]
|
List of corresponding target camera parameters for each targte depth map viewpoint. |
required |
src_ind
|
int
|
Index into 'depth_files' corresponding to the source depth map (if included in the list). |
-1
|
pixel_th
|
float
|
Pixel re-projection threshold to determine matching depth estimates. |
0.1
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The visibility mask for the source view. |
visibility_numpy(depths, K, Ps, vis_th=None)
Parameters:
Returns:
z_planes_from_disp(Z, b, f, delta)
Computes the near and far Z planes corresponding to 'delta' disparity steps between two cameras.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
Z
|
Tensor
|
Z buffer storing D depth plane hypotheses [B x C x D x H x W]. (shape resembles a typical PSV). |
required |
b
|
Tensor
|
The baseline between cameras [B]. |
required |
f
|
Tensor
|
The focal length of camera [B]. |
required |
delta
|
float
|
The disparity delta for the near and far planes. |
required |
Returns:
| Type | Description |
|---|---|
Tuple[Tensor, Tensor]
|
The tuple of near and far Z planes corresponding to 'delta' disparity steps. |