cosense3d.modules.fusion package

Submodules

cosense3d.modules.fusion.attn_fusion module

class cosense3d.modules.fusion.attn_fusion.DenseAttentionFusion(feature_dim, neck=None, **kwargs)

Bases: BaseModule

forward(ego_feats, coop_feats=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.attn_fusion.SparseAttentionFusion(stride, in_channels, **kwargs)

Bases: BaseModule

format_output(output)

forward(ego_feats, coop_feats=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fuse_feature_at_stride(ego_feat, coop_feat, stride, fuse_key)

training: bool

cosense3d.modules.fusion.box_fusion module

class cosense3d.modules.fusion.box_fusion.BoxFusion(lidar_range, **kwargs)

Bases: BaseModule

cluster_fusion(clusters, scores, labels, times, global_time)

Merge boxes in each cluster with scores as weights for merging

clustering(boxes, scores, labels, times, global_time)

forward(ego_preds, coop_preds, memory, global_times, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

merge_sync_boxes(c, s)

temporal_cluster_fusion(clusters, scores, labels, times, global_time)

Merge boxes in each cluster with scores as weights for merging

training: bool

cosense3d.modules.fusion.box_fusion.limit_period(val, offset=0.5, period=6.283185306)

cosense3d.modules.fusion.fax module

This class is about swap fusion applications

class cosense3d.modules.fusion.fax.Attention(dim, dim_head=32, dropout=0.0, agent_size=6, window_size=7)

Bases: Module

Unit Attention class. Todo: mask is not added yet.

Parameters

dim: int

Input feature dimension.

dim_head: int

The head dimension.

dropout: float

Dropout rate

agent_size: int

The agent can be different views, timestamps or vehicles.

forward(x, mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.fax.FeedForward(dim, hidden_dim, dropout=0.0)

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.fax.PreNormResidual(dim, fn)

Bases: Module

forward(x, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.fax.SwapFusionBlock(input_dim, mlp_dim, dim_head, window_size, agent_size, drop_out)

Bases: Module

Swap Fusion Block contains window attention and grid attention.

forward(x, mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.fax.SwapFusionBlockMask(input_dim, mlp_dim, dim_head, window_size, agent_size, drop_out)

Bases: Module

Swap Fusion Block contains window attention and grid attention with mask enabled for multi-vehicle cooperation.

forward(x, mask)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.fax.SwapFusionEncoder(input_dim=128, mlp_dim=256, agent_size=5, window_size=8, dim_head=32, drop_out=0.1, depth=3, mask=False, decoder=None, **kwargs)

Bases: BaseModule

Data rearrange -> swap block -> mlp_head

forward(ego_feat, coop_cpm, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

cosense3d.modules.fusion.keypoints module

class cosense3d.modules.fusion.keypoints.KeypointsFusion(lidar_range, train_from_epoch=0, **kwargs)

Bases: BaseModule

cluster_fusion(clusters, scores)

Merge boxes in each cluster with scores as weights for merging

clustering(boxes, scores)

forward(ego_feats, coop_feats, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

cosense3d.modules.fusion.keypoints.limit_period(val, offset=0.5, period=6.283185306)

cosense3d.modules.fusion.maxout_fusion module

class cosense3d.modules.fusion.maxout_fusion.BEVMaxoutFusion(**kwargs)

Bases: BaseModule

forward(ego_feats, coop_feats, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.maxout_fusion.SparseBEVMaxoutFusion(pc_range, resolution, **kwargs)

Bases: BaseModule

format_output(output)

forward(ego_feats, coop_feats, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

cosense3d.modules.fusion.naive_fusion module

class cosense3d.modules.fusion.naive_fusion.NaiveFusion(stride, **kwargs)

Bases: BaseModule

format_output(output)

forward(ego_feats, coop_feats=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fuse_feature_at_stride(ego_feat, coop_feat, stride, fuse_key)

training: bool

cosense3d.modules.fusion.spatial_query_fusion module

class cosense3d.modules.fusion.spatial_query_fusion.SpatialQueryAlignFusionRL(in_channels, pc_range, resolution, num_pose_feat=64, **kwargs)

Bases: BaseModule

align_coordinates(ego_bctr, ego_rl, ego_rl_pred, ego_pose, cpfeat)

format_output(output, **kwargs)

forward(det_local, roadline, roadline_preds, ego_queries, ego_pose_corrected, ego_poses, ego_poses_aug, cpms, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.spatial_query_fusion.SpatialQueryFusion(in_channels, pc_range, resolution, **kwargs)

Bases: BaseModule

format_output(output)

forward(ego_feats, coop_feats, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

cosense3d.modules.fusion.temporal_fusion module

class cosense3d.modules.fusion.temporal_fusion.LocalNaiveFusion(in_channels, feature_stride, lidar_range, pos_dim=3, topk_ref_pts=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, **kwargs)

Bases: BaseModule

This is a naive replacement of LocalTemporalFusion by only selecting the topk points for later spatial fusion

forward(local_roi, global_roi, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gather_topk(rois, bev_feats, stride, topk)

training: bool

class cosense3d.modules.fusion.temporal_fusion.LocalTemporalFusion(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=128, topk_ref_pts=1024, topk_feat=512, num_propagated=256, memory_len=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, norm_fusion=False, **kwargs)

Bases: BaseModule

Modified from TemporalFusion to standardize input and output keys

embed_pos(pos, dim=None)

forward(local_roi, global_roi, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gather_topk(rois, bev_feats, stride, topk)

init_weights()

temporal_alignment(query_pos, tgt, ref_pts, ref_feat, mem_dict, ref_time=None)

training: bool

class cosense3d.modules.fusion.temporal_fusion.LocalTemporalFusionV1(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=128, topk_ref_pts=1024, topk_feat=512, num_propagated=256, memory_len=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, norm_fusion=False, **kwargs)

Bases: LocalTemporalFusion

forward(rois, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.temporal_fusion.LocalTemporalFusionV2(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=128, topk_ref_pts=1024, topk_feat=512, num_propagated=256, memory_len=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, norm_fusion=False, **kwargs)

Bases: LocalTemporalFusion

forward(local_roi, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class cosense3d.modules.fusion.temporal_fusion.LocalTemporalFusionV3(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=128, topk_ref_pts=1024, topk_feat=512, num_propagated=256, memory_len=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, norm_fusion=False, **kwargs)

Bases: BaseModule

TemporalFusion with feature flow

embed_pos(pos, dim=None)

forward(local_roi, global_roi, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gather_topk(rois, bev_feats, stride, topk)

init_weights()

temporal_alignment(query_pos, tgt, ref_pts, ref_feat, mem_dict, ref_time=None)

training: bool

class cosense3d.modules.fusion.temporal_fusion.TemporalFusion(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=128, topk_ref_pts=1024, topk_feat=512, num_propagated=256, memory_len=1024, ref_pts_stride=2, transformer_itrs=1, global_ref_time=0, **kwargs)

Bases: BaseModule

embed_pos(pos, dim=None)

forward(rois, bev_feat, mem_dict, time_scale=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gather_topk(rois, bev_feats, stride, topk)

init_weights()

temporal_alignment(query_pos, tgt, ref_pts, ref_feat, mem_dict, ref_time=None)

training: bool

class cosense3d.modules.fusion.temporal_fusion.TemporalLidarFusion(in_channels, transformer, feature_stride, lidar_range, pos_dim=3, num_pose_feat=64, topk=2048, num_propagated=256, memory_len=1024, num_query=644, **kwargs)

Bases: BaseModule

embed_pos(pos, dim=None)

forward(rois, bev_feat, mem_dict, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gather_topk(rois, bev_feats)

init_weights()

temporal_alignment(query_pos, tgt, ref_pts, mem_dict)

training: bool

Module contents