Haversine KNN Candidate Generation

What it does

This skill provides a practical implementation and guidance for generating candidate pairs for entity matching using KNN with the haversine distance. It's designed for location-based entities (POIs, stores, addresses) where comparing all N^2 pairs is infeasible. The approach computes K nearest geographic neighbors per record (optionally within country/region partitions), returns neighbor ranks and geographic distances (km), and produces a candidate DataFrame ready for downstream scoring/classification.

When to use it

Use this skill when you need an efficient candidate generation step for deduplication or record linkage tasks involving geospatial data. It's ideal for datasets with latitude/longitude columns where you want to limit comparisons to nearby records, or when you want to combine geo-based recall with text-based matching to improve overall candidate quality. Partitioning by country is recommended for global datasets to avoid irrelevant cross-region matches.

What's included

• Scripts: Python example code showing conversion to radians, per-partition KNN using scikit-learn NearestNeighbors (haversine metric), and assembling candidate pairs with distance and neighbor rank.
• References: Linked Kaggle examples for real-world baselines.
• Instructions: Step-by-step workflow (convert lat/lon to radians, partition, fit KNN, extract neighbors, build candidate DataFrame) and configuration guidance (n_neighbors typical range, distance unit conversion).

Compatible agents

Best used with data-science and Python-capable agents (Copilot/Code assistants, Python-running agent environments) that can execute or adapt the provided example code for production pipelines.