It's easy to describe the relative position of two single points, but how can we measure the spatial relationship between more complicated objects? Here, we consider simple 2D boxes and some ways to capture their relative position.

This webpage is an interactive supplement for our more in-depth articles that describe how to compute these measures in more detail.

Triangular Fuzzy Number Descriptor

Consider a pair of 2D boxes as shown in the interactive figure below. How would you describe the relative position of the red box with respect to the blue box? Suppose that the marked dot within each box represents the center of mass, which may not necessarily be in the exact middle of the object. One way to capture the spatial relationship between the two boxes is to represent their positions on the X and Y axes as triangular fuzzy numbers (TFNs). The minimum and maximum values of each TFN are the extents of the box along each axis, and the peak value represents the centroid. Subtracting the blue TFN from the red TFN gives a new relative position TFN shown in green. We can visualize the relative TFNs from each axis as a box in 2D space. We call this the Triangular Fuzzy Number Relative Position Descriptor (TFN-RPD), and it captures the minimum, maximum, and average distance between two objects along both axes.

In the figure below, you can adjust the shapes and positions of the red and blue boxes, as well as the centroids, to observe the TFN-RPD changes of the red box with respect to the blue box.

Histogram of Forces Descriptor

Another way we can describe the relative position between two objects is with the histogram of forces (HoF).. A force histogram represents the strength of the proposition "Object A is in direction $$\theta$$ from Object B." In the figure below, we show the histogram of constant forces ($$F_0$$) and the histogram of gravitational forces ($$F_2$$) between the blue and red boxes. The $$F_0$$ histogram uses a constant term when evaluating pairs of points from each object, whereas the $$F_2$$ histogram is inversely weighted by the distance between objects. The result is that the $$F_2$$ histogram is more sensitive to regions where the two objects are close together. Unfortunately, the $$F_2$$ histogram is undefined when the objects overlap. To workaround this, we can use a hybrid histogram ($$F_{02}$$) that blends the $$F_0$$ and $$F_2$$ histograms such that $$F_{02}$$ is equivalent to $$F_2$$ at far distances and $$F_0$$ when the objects overlap. Together, these two histograms make up the Histogram of Forces Relative Position Descriptor (HoF-RPD).

In the figure below, you can move and resize the two boxes, and switch between a vectorized version of the HoF that is quick to compute and two rasterized versions that can utilize the hybrid force histogram approach. The histograms show the strength of the forces that support the red box being in direction $$\theta$$ from the blue box.

Comparing Relative Position Descriptors

We can use the TFN-RPD or the HoF-RPD to compare the spatial relationships between two pairs of objects. Consider the relative position of the dark red box with respect to the dark blue box in the figure below. How similar is it to that of the light blue and light red boxes? There are many ways to evaluate this, some of which we compute here. For complete details, see our full article.

Single-Axis TFN Methods

In these methods, we use the difference TFNs along the X and Y axes (shown in green). Assuming the position of the blue box is given by the TFNs $$A_x$$ and $$A_y$$, and the position of the red box is given by $$B_x$$ and $$B_y$$, the difference TFNs are defined as

$$D_x = B_x - A_x$$
$$D_y = B_y - A_y$$

The X and Y similarity measures can be combined with either the min or mean value.

$$\mathrm{TFN\text{--}SA\text{--}Min\text{--}\mu} = \min(S_\mu^X, S_\mu^Y)$$
$$\mathrm{TFN\text{--}SA\text{--}Mean\text{--}\mu} = \frac{1}{2}(S_\mu^X + S_\mu^Y)$$

• TFN-SA-Min-Max, TFN-SA-Mean-Max:
  Similarity is defined as the maximum of the intersection area between the two TFNs. \mu_{\mathrm{max}}(D, D') = \max_{x \in \mathbb{R}} \bigg\{ \min\big(D(x), D'(x) \big) \bigg\}
• TFN-SA-Min-IoU, TFN-SA-Mean-IoU:
  Similarity is defined as the intersection over union measure of the two TFNs. \mu_{\mathrm{IoU}}(D, D') = \frac{|D \cap D'|}{|D \cup D'|}
• TFN-SA-Min-PD, TFN-SA-Mean-PD:
  Similarity is defined as the percent difference between the endpoints and centers of the two TFNs, $$D = (d_1, d_2, d_3)$$ and $$D' = (d_1', d_2', d_3')$$. \mu_{\mathrm{PD}}(D, D') = \frac{1}{1 + PD(D, D')} PD(D, D') = \sum_{i=1}^3 \frac{|d_i - d_i'|} {d_3 - d_1 + d_3' -d_1'}

Bounding Box TFN Methods

In these methods, the 2D bounding boxes formed by the difference TFNs (shown in green) are used directly, so there is no need to combine the individual axis measures with the min or mean.

• TFN-BB-IoU:
  Similarity is defined as the intersection over union of the two TFN bounding boxes. \mu_{\mathrm{IoU}}(D, D') = \frac{|D \cap D'|}{|D \cup D'|}
• TFN-BB-GIoU:
  Similarity is defined as the generalized intersection over union measure of the two TFN bounding boxes. Let $$C$$ be the convex hull of $$D$$ and $$D'$$. Then, \mu_{\mathrm{GIoU}}(D, D') = \frac{1}{2} \bigg( \frac{|D \cap D'|}{|D \cup D'|} - \frac{|C \setminus (D \cup D')|}{|C|} + 1 \bigg)
• TFN-BB-PD:
  Similarity is defined as the percent difference between the corner points and centers of the two TFN bounding boxes. \mu_{\mathrm{PD}}(D, D') = \frac{1}{1 + \sum_{j \in \{x, y\}}PD(D_j, D_j')}

Histogram of Forces Methods

In these methods, the histograms of constant and gravitational (or hybrid) forces are compared using one of the three following similarity measures. The $$F_0$$ and $$F_2$$ histogram (or $$F_{02}$$) similarities between the object pairs $$AB$$ and $$A'B'$$ are averaged into a combined measure.

\mathrm{HoF\text{--}\mu} = \tfrac{1}{2}\mu\big(F_0^{AB}, F_0^{A'B'}\big) + \tfrac{1}{2}\mu\big(F_2^{AB}, F_2^{A'B'}\big)

• HOF-T:
  Similarity between histograms is defined as a Tversky index. \mu_T(h_1, h_2) = \frac{\sum_{\theta}\min(h_1(\theta), h_2(\theta))}{\sum_{\theta}\max(h_1(\theta), h_2(\theta))}
• HOF-P:
  Similarity between histograms is defined as a Pappis' measure. \mu_P(h_1, h_2) = 1 - \frac{\sum_{\theta}|h_1(\theta) - h_2(\theta)|}{\sum_{\theta}|h_1(\theta) + h_2(\theta)|}
• HOF-C:
  Similarity between histograms is defined as the normalized cross-correlation. \mu_C(h_1, h_2) = \frac{\sum_{\theta}h_1(\theta)h_2(\theta)}{\sqrt{\sum_{\theta}h_1^2(\theta)}\sqrt{\sum_{\theta}h_2^2(\theta)}}

In the figure below, you can adjust the shapes and positions of the boxes and their centroids. The TFN-RPDs and HoF-RPDs are shown for the relationships between the dark red and blue boxes and the light red and blue boxes. Several similarity measures are computed to evaluate how similar the dark pair of boxes is to the light pair.

Each of the measures evaluates the similarity of the spatial relationships differently. The TFN methods are simple to compute, however the HoF approach is more computationally demanding. Depending on the intended application, some measures may be more appropriate than others. Note how some measures are quick to drop to zero when the object pairs are not well aligned, whereas other measures maintain positive values regardless of the positions of the objects. These are by no means the only ways to compute spatial similarity between objects, but they demonstrate the variety of different approaches that one can consider.