limap.point2d.superglue package

Submodules

limap.point2d.superglue.superglue module

class limap.point2d.superglue.superglue.AttentionalGNN(feature_dim: int, layer_names: List[str])

Bases: Module

forward(desc0: Tensor, desc1: Tensor) → Tuple[Tensor, Tensor]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class limap.point2d.superglue.superglue.AttentionalPropagation(feature_dim: int, num_heads: int)

Bases: Module

forward(x: Tensor, source: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class limap.point2d.superglue.superglue.KeypointEncoder(feature_dim: int, layers: List[int])

Bases: Module

Joint encoding of visual appearance and location using MLPs

forward(kpts, scores)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
limap.point2d.superglue.superglue.MLP(channels: List[int], do_bn: bool = True) → Module

Multi-layer perceptron

class limap.point2d.superglue.superglue.MultiHeadedAttention(num_heads: int, d_model: int)

Bases: Module

Multi-head attention to increase model expressivitiy

forward(query: Tensor, key: Tensor, value: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class limap.point2d.superglue.superglue.SuperGlue(config)

Bases: Module

SuperGlue feature matching middle-end

Given two sets of keypoints and locations, we determine the correspondences by:

  1. Keypoint Encoding (normalization + visual feature and location fusion)

  2. Graph Neural Network with multiple self and cross-attention layers

  3. Final projection layer

  4. Optimal Transport Layer (a differentiable Hungarian matching algorithm)

  5. Thresholding matrix based on mutual exclusivity and a match_threshold

The correspondence ids use -1 to indicate non-matching points.

Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning Feature Matching with Graph Neural Networks. In CVPR, 2020. https://arxiv.org/abs/1911.11763

default_config = {'GNN_layers': ['self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross'], 'descriptor_dim': 256, 'keypoint_encoder': [32, 64, 128, 256], 'match_threshold': 0.2, 'sinkhorn_iterations': 100, 'weight_path': None, 'weights': 'outdoor'}
download_model(path)
forward(data)

Run SuperGlue on a pair of keypoints and descriptors

log_optimal_transport(scores: Tensor, alpha: Tensor, iters: int) → Tensor

Perform Differentiable Optimal Transport in Log-space for stability

log_sinkhorn_iterations(Z: Tensor, log_mu: Tensor, log_nu: Tensor, iters: int) → Tensor

Perform Sinkhorn Normalization in Log-space for stability

training: bool
limap.point2d.superglue.superglue.arange_like(x, dim: int)
limap.point2d.superglue.superglue.attention(query: Tensor, key: Tensor, value: Tensor) → Tuple[Tensor, Tensor]
limap.point2d.superglue.superglue.normalize_keypoints(kpts, image_shape)

Normalize keypoints locations based on image image_shape

Module contents