limap.line2d.LineTR package

Submodules

limap.line2d.LineTR.extractor module

class limap.line2d.LineTR.extractor.LineTRExtractor(options=BaseDetectorOptions(set_gray=True, max_num_2d_segs=3000, do_merge_lines=False, visualize=False, weight_path=None), device=None)

Bases: BaseDetector

compute_descinfo(img, segs)

A desc_info is composed of the following tuple / np arrays: - the line descriptors with shape [num_lines, 256] - mat_klines2sublines: matrix assigning sublines to lines

extract(camview, segs)

Virtual method (for extractor) - extract the features for the detected segments

Parameters:

• view (limap.base.CameraView) – The limap.base.CameraView instance corresponding to the image

• segs – np.array of shape (N, 5), line detections. Each row corresponds to x1, y1, x2, y2 and score. Computed from the detect method.

Returns:

The extracted feature

get_descinfo_fname(descinfo_folder, img_id)

Virtual method (for extractor) - Get the target filename of the extracted feature

Parameters:

• descinfo_folder (str) – The output folder

• img_id (int) – The image id

Returns:

target filename

Return type:

str

get_module_name()

Virtual method (need to be implemented) - return the name of the module

read_descinfo(descinfo_folder, img_id)

Virtual method (for extractor) - Read in the extracted feature. Dual function for save_descinfo.

Parameters:

• descinfo_folder (str) – The output folder

• img_id (int) – The image id

Returns:

The extracted feature

save_descinfo(descinfo_folder, img_id, descinfo)

Virtual method (for extractor) - Save the extracted feature to the target folder

Parameters:

• descinfo_folder (str) – The output folder

• img_id (int) – The image id

• descinfo – The features extracted from the function extract

limap.line2d.LineTR.line_attention module

class limap.line2d.LineTR.line_attention.FeedForward(d_in, d_hid, dropout=0.1)

Bases: Module

Feed Forward layer

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_attention.MultiHeadAttention(n_heads: int, d_feature: int, dropout=0.1)

Bases: Module

Multi-Headed Attention

forward(q, k, v, mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_attention.ScaledDotProduct(scale, attn_dropout=0.1)

Bases: Module

Scaled Dot-Product Attention

forward(q, k, v, mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

limap.line2d.LineTR.line_process module

limap.line2d.LineTR.line_process.change_cv2_T_np(klines_cv)

limap.line2d.LineTR.line_process.filter_by_length(lines, min_length, max_sublines)

limap.line2d.LineTR.line_process.get_angles(lines)

limap.line2d.LineTR.line_process.get_dist_matrix(desc0, desc1)

limap.line2d.LineTR.line_process.get_line_dist(line)

limap.line2d.LineTR.line_process.line_tokenizer(klines, token_distance, max_tokens, pred_superpoint, image_shape)

limap.line2d.LineTR.line_process.point_on_line(line, dist_px)

limap.line2d.LineTR.line_process.preprocess(klines_cv, image_shape, pred_superpoint, mask=None, conf={})

limap.line2d.LineTR.line_process.remove_borders(lines, border: int, height: int, width: int, valid_mask_given=None)

Removes keylines too close to the border

limap.line2d.LineTR.line_process.sample_descriptors(keypoints, descriptors, s: int = 8)

Interpolate descriptors at keypoint locations

limap.line2d.LineTR.line_transformer module

class limap.line2d.LineTR.line_transformer.AttentionalPropagation(feature_dim: int, num_heads: int)

Bases: Module

forward(x, source)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.KeylineEncoder(feature_dim, mlp_layers, n_heads, n_att_layers, d_inner, dropout=0.1)

Bases: Module

Line Descriptive Networks & Positional Embedding for Tokens and Lines

forward(klines, resp, angle, pnt, desc, score, mask, return_attns=False)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.LineDescriptiveEncoder(d_feature: int, n_heads: int, n_att_layers: int, d_inner: int, dropout=0.1)

Bases: Module

Line Descriptive Network using the transformer

forward(desc, slf_attn_mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.LinePositionalEncoder(feature_dim, layers)

Bases: Module

Joint encoding of visual appearance and location using MLPs

forward(klines, responses, angles)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.LineTransformer(config)

Bases: Module

Line-Transformer Networks including Line Descriptive Nets and Line Signature Nets

default_config = {'d_inner': 1024, 'descriptor_dim': 256, 'image_shape': [480, 640], 'keyline_encoder': [32, 64, 128, 256], 'max_keylines': -1, 'max_tokens': 21, 'min_length': 16, 'mode': 'test', 'n_heads': 4, 'n_line_descriptive_layers': 1, 'remove_borders': 8, 'token_distance': 8, 'weight_path': None}

default_ret()

Default return dictionary for the exception

download_model(path)

forward(data)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

preprocess(klines_cv, image_shape, pred_superpoint, valid_mask=None)

Pre-process for line tokenization

subline2keyline(distance_sublines, mat_klines2sublines0, mat_klines2sublines1)

Convert sublines’ distance matrix to keylines’ distance matrix

training: bool

limap.line2d.LineTR.line_transformer.MLP(channels: list, do_bn=True)

Multi-layer perceptron

class limap.line2d.LineTR.line_transformer.MultiHeadedAttention(num_heads: int, d_model: int)

Bases: Module

Multi-head attention to increase model expressivitiy

forward(query, key, value)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.SelfAttentionalLayer(feature_dim: int, layer_names: list)

Bases: Module

forward(kline_desc)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class limap.line2d.LineTR.line_transformer.WordPositionalEncoder(feature_dim, layers)

Bases: Module

Joint encoding of visual appearance and location using MLPs

forward(words_in_line, scores_in_line)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

limap.line2d.LineTR.line_transformer.attention(query, key, value)

limap.line2d.LineTR.line_transformer.normalize_keylines(klines, kplines, image_shape)

Normalize keylines locations based on image shape

limap.line2d.LineTR.linetr_pipeline module

limap.line2d.LineTR.matcher module

class limap.line2d.LineTR.matcher.LineTRMatcher(extractor, options=BaseMatcherOptions(topk=10, n_neighbors=20, n_jobs=1, weight_path=None), topk=0, device=None)

Bases: BaseMatcher

check_compatibility(extractor)

get_module_name()

match_pair(descinfo1, descinfo2)

match_segs_with_descinfo(descinfo1, descinfo2)

match_segs_with_descinfo_topk(descinfo1, descinfo2, topk=10)

limap.line2d.LineTR.nn_matcher module

limap.line2d.LineTR.nn_matcher.nn_matcher(desc0, desc1, nn_thresh=0.8, is_mutual_NN=True)

Nearest Neighbor Matching using two descriptors

limap.line2d.LineTR.nn_matcher.nn_matcher_distmat(dist_mat, nn_thresh, is_mutual_NN=True)

Nearest Neighbor Matching using a distance matrix

Module contents