cosense3d.modules.utils package

Submodules

cosense3d.modules.utils.box_coder module

class cosense3d.modules.utils.box_coder.BoxPredCoder(with_velo=False)

Bases: object

decode(centers, reg)

Parameters:

• centers – Tensor (N, 3) or (B, N, 2+).

• reg – dict, box - (N, 6) or (B, N, 6) dir - (N, 8) or (B, N, 8) scr - (N, 4) or (B, N, 4) vel - (N, 2) or (B, N, 2), optional pred - (N, 5) or (B, N, 5), optional

Returns:

decoded bboxes.

encode(centers, gt_boxes, meter_per_pixel, gt_preds)

Parameters:

• centers – (N, 3)

• gt_boxes – (N, 8) [batch_idx, x, y, z, l, w, h, r]

• meter_per_pixel – tuple with 2 elements

• gt_preds – (N, 8) [batch_idx, x, y, z, l, w, h, r], gt boxes to be predicted

Returns:

encoded bbox targets.

class cosense3d.modules.utils.box_coder.CenterBoxCoder(with_velo=False, with_pred=False, reg_radius=1.6, z_offset=1.0)

Bases: object

decode(centers, reg)

Parameters:

• centers – Tensor (N, 3) or (B, N, 2+).

• reg – dict, box - (N, 6) or (B, N, 6) dir - (N, 8) or (B, N, 8) scr - (N, 4) or (B, N, 4) vel - (N, 2) or (B, N, 2), optional pred - (N, 5) or (B, N, 5), optional

Returns:

decoded bboxes.

encode(centers, gt_boxes, meter_per_pixel, gt_preds=None)

Parameters:

• centers – (N, 3)

• gt_boxes – (N, 8) [batch_idx, x, y, z, l, w, h, r]

• meter_per_pixel – tuple with 2 elements

• gt_preds –

Returns:

class cosense3d.modules.utils.box_coder.ResidualBoxCoder(mode: str = 'simple_dist')

Bases: object

decode(anchors, boxes_enc, dir_scores=None)

decode_direction(ra, vt, dir_scores=None)

encode(anchors, boxes)

encode_direction(ra, rg)

cosense3d.modules.utils.box_coder.build_box_coder(type, **kwargs)

cosense3d.modules.utils.common module

cosense3d.modules.utils.common.bias_init_with_prob(prior_prob: float) → float

initialize conv/fc bias value according to a given probability value.

cosense3d.modules.utils.common.cat_coor_with_idx(tensor_list)

cosense3d.modules.utils.common.cat_name_str(module_name)

Parameters:

module_name – str, format in xxx_yyy_zzz

Returns:

class_name: str, format in XxxYyyZzz

cosense3d.modules.utils.common.clip_sigmoid(x: Tensor, eps: float = 0.0001) → Tensor

Sigmoid function for input feature.

Parameters:

• x – Input feature map with the shape of [B, N, H, W].

• eps – Lower bound of the range to be clamped to. Defaults to 1e-4.

Returns:

Feature map after sigmoid.

cosense3d.modules.utils.common.draw_sample_prob(centers, reg, samples, res, distr_r, det_r, batch_size, var0)

cosense3d.modules.utils.common.fuse_batch_indices(coords, num_cav)

Fusing voxels of CAVs from the same frame :param stensor: ME sparse tensor :param num_cav: list of number of CAVs for each frame :return: fused coordinates and features of stensor

cosense3d.modules.utils.common.get_conv2d_layers(conv_name, in_channels, out_channels, n_layers, kernel_size, stride, padding, relu_last=True, sequential=True, **kwargs)

Build convolutional layers. kernel_size, stride and padding should be a list with the lengths that match n_layers

cosense3d.modules.utils.common.get_norm_layer(channels, norm)

cosense3d.modules.utils.common.get_voxel_centers(voxel_coords, downsample_times, voxel_size, point_cloud_range)

Get centers of spconv voxels.

Parameters:

• voxel_coords – (N, 3)

• downsample_times –

• voxel_size –

• point_cloud_range –

Returns:

cosense3d.modules.utils.common.instantiate(module_name, cls_name=None, module_cfg=None, **kwargs)

cosense3d.modules.utils.common.inverse_sigmoid(x, eps=1e-05)

Inverse function of sigmoid.

Parameters:

• x – (Tensor) The tensor to do the inverse.

• eps – (float) EPS avoid numerical overflow. Defaults 1e-5.

Returns:

Tensor: The x has passed the inverse function of sigmoid, has same shape with input.

cosense3d.modules.utils.common.limit_period(val, offset=0.5, period=6.283185306)

cosense3d.modules.utils.common.linear_last(in_channels, mid_channels, out_channels, bias=False, norm='BN')

cosense3d.modules.utils.common.linear_layers(in_out, activations=None, norm='BN')

cosense3d.modules.utils.common.meshgrid(xmin, xmax, ymin=None, ymax=None, dim=2, n_steps=None, step=None)

cosense3d.modules.utils.common.meshgrid_cross(xmins, xmaxs, n_steps=None, steps=None)

cosense3d.modules.utils.common.pad_l(tensor, value=0)

cosense3d.modules.utils.common.pad_r(tensor, value=0)

cosense3d.modules.utils.common.topk_gather(feat, topk_indexes)

cosense3d.modules.utils.common.weighted_mahalanobis_dists(reg_evi, reg_var, dists, var0)

cosense3d.modules.utils.common.xavier_init(module: Module, gain: float = 1, bias: float = 0, distribution: str = 'normal') → None

cosense3d.modules.utils.conv module

class cosense3d.modules.utils.conv.ConvModule(in_channels: int, out_channels: int, kernel_size: int | Tuple[int, int], stride: int | Tuple[int, int] = 1, padding: int | Tuple[int, int] = 0, dilation: int | Tuple[int, int] = 1, groups: int = 1, bias: bool | str = 'auto', conv_cfg: Dict | None = None, norm_cfg: Dict | None = None, act_cfg: Dict | None = {'type': 'ReLU'}, inplace: bool = True, with_spectral_norm: bool = False, padding_mode: str = 'zeros', order: tuple = ('conv', 'norm', 'act'))

Bases: Module

A conv block that bundles conv/norm/activation layers.

This block simplifies the usage of convolution layers, which are commonly used with a norm layer (e.g., BatchNorm) and activation layer (e.g., ReLU). It is based upon three build methods: build_conv_layer(), build_norm_layer() and build_activation_layer().

Besides, we add some additional features in this module. 1. Automatically set bias of the conv layer. 2. Spectral norm is supported. 3. More padding modes are supported. Before PyTorch 1.5, nn.Conv2d only supports zero and circular padding, and we add “reflect” padding mode.

Args:

in_channels (int): Number of channels in the input feature map.

Same as that in nn._ConvNd.

out_channels (int): Number of channels produced by the convolution.

Same as that in nn._ConvNd.

kernel_size (int | tuple[int]): Size of the convolving kernel.

Same as that in nn._ConvNd.

stride (int | tuple[int]): Stride of the convolution.

Same as that in nn._ConvNd.

padding (int | tuple[int]): Zero-padding added to both sides of

the input. Same as that in nn._ConvNd.

dilation (int | tuple[int]): Spacing between kernel elements.

Same as that in nn._ConvNd.

groups (int): Number of blocked connections from input channels to

output channels. Same as that in nn._ConvNd.

bias (bool | str): If specified as auto, it will be decided by the

norm_cfg. Bias will be set as True if norm_cfg is None, otherwise False. Default: “auto”.

conv_cfg (dict): Config dict for convolution layer. Default: None,

which means using conv2d.

norm_cfg (dict): Config dict for normalization layer. Default: None. act_cfg (dict): Config dict for activation layer.

Default: dict(type=’ReLU’).

inplace (bool): Whether to use inplace mode for activation.

Default: True.

with_spectral_norm (bool): Whether use spectral norm in conv module.

Default: False.

padding_mode (str): If the padding_mode has not been supported by

current Conv2d in PyTorch, we will use our own padding layer instead. Currently, we support [‘zeros’, ‘circular’] with official implementation and [‘reflect’] with our own implementation. Default: ‘zeros’.

order (tuple[str]): The order of conv/norm/activation layers. It is a

sequence of “conv”, “norm” and “act”. Common examples are (“conv”, “norm”, “act”) and (“act”, “conv”, “norm”). Default: (‘conv’, ‘norm’, ‘act’).

forward(x: Tensor, activate: bool = True, norm: bool = True) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()

property norm

training: bool

cosense3d.modules.utils.conv.build_conv_layer(cfg: Dict | None, *args, **kwargs) → Module

Build convolution layer. Modified from openmmlab.

Args:

cfg (None or dict): The conv layer config, which should contain:

• type (str): Layer type.

• layer args: Args needed to instantiate an conv layer.

args (argument list): Arguments passed to the __init__

method of the corresponding conv layer.

kwargs (keyword arguments): Keyword arguments passed to the __init__

method of the corresponding conv layer.

Returns:

nn.Module: Created conv layer.

cosense3d.modules.utils.conv.build_padding_layer(cfg: Dict, *args, **kwargs) → Module

Build padding layer.

Args:

cfg (dict): The padding layer config, which should contain:

• type (str): Layer type.

• layer args: Args needed to instantiate a padding layer.

Returns:

nn.Module: Created padding layer.

cosense3d.modules.utils.edl_utils module

cosense3d.modules.utils.edl_utils.logit_to_edl(logits)

Parameters

logits: Tensor, (…, C),

Returns

cosense3d.modules.utils.gaussian_utils module

cosense3d.modules.utils.gaussian_utils.center_to_img_coor(center_in, lidar_range, pixel_sz)

cosense3d.modules.utils.gaussian_utils.cornernet_gaussian_radius(height, width, min_overlap=0.5)

cosense3d.modules.utils.gaussian_utils.draw_gaussian_map(boxes, lidar_range, pixel_sz, batch_size, radius=None, sigma=1, min_radius=2)

cosense3d.modules.utils.gaussian_utils.gaussian_2d(shape: List[int], sigma: float = 1.0) → ndarray

Generate gaussian map.

Parameters:

• shape – Shape of the map.

• sigma – Sigma to generate gaussian map. Defaults to 1.

Returns:

Generated gaussian map.

cosense3d.modules.utils.gaussian_utils.gaussian_radius(box_dims, pixel_sz, overlap, min_radius=2)

cosense3d.modules.utils.gaussian_utils.mahalanobis_dists_2d(sigmas, dists)

Compute the squared mahalanobis distances.

Parameters:

• sigmas – (N, 2), standard deviation of Gaussian distribution

• dists – (N, 2), distances to gaussian center

Returns:

(N), squared mahalanobis

cosense3d.modules.utils.gaussian_utils.weighted_mahalanobis_dists(vars, dists, weights=None)

Compute the squared mahalanobis distances.

Parameters:

• vars – (N, 2), variances of Gaussian distribution.

• dists – (N, 2), distances to gaussian center at each axis.

• weights – weights to be applied to the output probability.

Returns:

(N), squared mahalanobis

cosense3d.modules.utils.init module

cosense3d.modules.utils.init.bias_init_with_prob(prior_prob: float) → float

initialize conv/fc bias value according to a given probability value.

cosense3d.modules.utils.init.constant_init(module: Module, val: float, bias: float = 0) → None

cosense3d.modules.utils.init.kaiming_init(module: Module, a: float = 0, mode: str = 'fan_out', nonlinearity: str = 'relu', bias: float = 0, distribution: str = 'normal') → None

cosense3d.modules.utils.init.normal_init(module: Module, mean: float = 0, std: float = 1, bias: float = 0) → None

cosense3d.modules.utils.init.trunc_normal_init(module: Module, mean: float = 0, std: float = 1, a: float = -2, b: float = 2, bias: float = 0) → None

cosense3d.modules.utils.init.uniform_init(module: Module, a: float = 0, b: float = 1, bias: float = 0) → None

cosense3d.modules.utils.init.xavier_init(module: Module, gain: float = 1, bias: float = 0, distribution: str = 'normal') → None

cosense3d.modules.utils.me_utils module

cosense3d.modules.utils.me_utils.bev_sparse_to_dense(self, preds)

cosense3d.modules.utils.me_utils.devoxelize_with_centroids(out: SparseTensor, x: TensorField, h_embs)

cosense3d.modules.utils.me_utils.downsample_embeddings(embeddings, inverse_map, size, mode='avg')

cosense3d.modules.utils.me_utils.downsample_points(points, tensor_map, field_map, size)

cosense3d.modules.utils.me_utils.get_conv_block(nc, k=3, d=3, tr=False, bn_momentum=0.1, distributed=False)

create sparse convolution block :param nc: number of channels in each layer in [in_layer, mid_layer, out_layer] :param k: kernel size :param tr: transposed convolution :return: conv block

cosense3d.modules.utils.me_utils.get_kernel_map_and_out_key(stensor, stensor_out=None, kernel_size=3, stride=1, dilation=1, kernel_type='cube', kernel_generator=None)

Generate kernel maps for the input stensor. The hybrid and custom kernel is not implemented in ME v0.5.x, this function uses a kernel mask to select the kernel maps for the customized kernel shapes. :param stensor: ME.SparseTensor, NxC :param kernel_type: ‘cube’(default) | ‘hybrid’ :return: masked kernel maps

cosense3d.modules.utils.me_utils.indices2metric(indices, voxel_size)

Voxel indices to voxel center in meter

cosense3d.modules.utils.me_utils.me_coor_to_grid_indices(lr, voxel_size, stride, coor)

cosense3d.modules.utils.me_utils.metric2indices(coor, voxel_size)

“Round towards floor

cosense3d.modules.utils.me_utils.mink_coor_limit(lidar_range, voxel_size, stride)

cosense3d.modules.utils.me_utils.minkconv_conv_block(in_dim, out_dim, kernel, stride, d=3, bn_momentum=0.1, activation='LeakyReLU', tr=False, expand_coordinates=False, norm_before=False, distributed=False)

cosense3d.modules.utils.me_utils.minkconv_layer(in_dim, out_dim, kernel, stride, d, tr=False)

cosense3d.modules.utils.me_utils.normalize_centroids(down_points, coordinates, tensor_stride)

cosense3d.modules.utils.me_utils.normalize_points(points, centroids, tensor_map)

cosense3d.modules.utils.me_utils.prepare_input_data(points_list, voxel_size, QMODE, floor_height, coor_dim=3, feat_dim=3)

cosense3d.modules.utils.me_utils.sparse_to_dense(stensor, voxel_size, det_r)

cosense3d.modules.utils.me_utils.stride_centroids(points, counts, rows, cols, size)

cosense3d.modules.utils.me_utils.update_me_essentials(self: object, data_info: dict, stride: int | None = None)

Update essential variables for ME-based models

Parameters:

• self – instance of a python class

• data_info –

  • det_r: float

  • lidar_range: [xmin, ymin, zmin, xmax, ymax, zmax]

  • voxel_size: [vx, vy, vz]

• stride –

Returns:

cosense3d.modules.utils.me_utils.voxelize_with_centroids(x: TensorField, enc_mlp, pc_range)

cosense3d.modules.utils.misc module

class cosense3d.modules.utils.misc.MLN(c_dim, f_dim=256)

Bases: Module

Args:

c_dim (int): dimension of latent code c f_dim (int): feature dimension

forward(x, c)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

reset_parameters()

training: bool

class cosense3d.modules.utils.misc.MLN2(c_dim, f_dim=256)

Bases: Module

Args:

c_dim (int): dimension of latent code c f_dim (int): feature dimension

forward(x, c)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

reset_parameters()

training: bool

class cosense3d.modules.utils.misc.SELayer_Linear(channels, act_layer=<class 'torch.nn.modules.activation.ReLU'>, gate_layer=<class 'torch.nn.modules.activation.Sigmoid'>, norm=False)

Bases: Module

forward(x, x_se)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

cosense3d.modules.utils.nbr_attn module

class cosense3d.modules.utils.nbr_attn.NeighborhoodAttention(emb_dim, n_nbr=16, num_pose_feat=64, **kwargs)

Bases: Module

Generate reference points and attend neighborhood features.

forward(memory, mem_coor, q_coor, B)

Args:

q: (S, D) kv: (L, D) q_coor: (S, 3), [idx, x, y] kv_coor: (L, 3)

Returns:

training: bool

cosense3d.modules.utils.norm module

cosense3d.modules.utils.norm.build_norm_layer(cfg: Dict, num_features: int, postfix: int | str = '') → Tuple[str, Module]

Build normalization layer. Modified from openmmlab.

Parameters:

• cfg – (dict) The norm layer config, which should contain: - type (str): Layer type. - layer args: Args needed to instantiate a norm layer. - requires_grad (bool, optional): Whether stop gradient updates.

• num_features – (int) Number of input channels.

• postfix – (int | str) The postfix to be appended into norm abbreviation to create named layer.

Returns:

tuple[str, nn.Module]: The first element is the layer name consisting of abbreviation and postfix, e.g., bn1, gn. The second element is the created norm layer.

cosense3d.modules.utils.positional_encoding module

cosense3d.modules.utils.positional_encoding.coor2ratio(coor, lidar_range)

cosense3d.modules.utils.positional_encoding.img_locations(img_size, feat_size=None, stride=None)

cosense3d.modules.utils.positional_encoding.nerf_positional_encoding(tensor: Tensor, num_encoding_functions: int = 6, include_input: bool = False, log_sampling: bool = True) → Tensor

Apply positional encoding to the input.

Parameters:

• tensor – Input tensor to be positionally encoded.

• num_encoding_functions – Number of encoding functions used to compute a positional encoding (default: 6).

• include_input – Whether or not to include the input in the positional encoding (default: True).

• log_sampling –

Returns:

Positional encoding of the input tensor.

cosense3d.modules.utils.positional_encoding.pos2posemb1d(pos, num_pos_feats=256, temperature=10000)

cosense3d.modules.utils.positional_encoding.pos2posemb2d(pos, num_pos_feats=128, temperature=10000)

cosense3d.modules.utils.positional_encoding.pos2posemb3d(pos, num_pos_feats=128, temperature=10000)

cosense3d.modules.utils.positional_encoding.ratio2coord(ratio, lidar_range)

Module contents

cosense3d.modules.utils.build_torch_module(cfg)