FS-MWK++ Beats Deep Learning at Finding Meaningful Features Without Labels

Picking out which columns of a dataset actually matter is easy when you have labels. Strip the labels away and the problem gets ugly fast, especially when hundreds of noisy features drown out the handful that carry real structure. A new paper by Xudong Zhang and Renato Cordeiro de Amorim proposes a fresh take: instead of training a neural network to guess which features matter, run a weighted clustering algorithm many times under different distance geometries and keep the features whose importance stays high across all of them.

The authors introduce three connected algorithms and release a reference implementation on GitHub. On synthetic benchmarks, the approach hits near-perfect accuracy at separating informative features from injected noise, outperforming deep learning baselines and even a supervised method that gets to peek at the labels.

Why unsupervised feature selection is hard

Clustering algorithms like k-means treat every feature as equally important. That assumption falls apart in high-dimensional data, which frequently contains redundant or irrelevant columns that obscure meaningful patterns and degrade both efficiency and accuracy. Most feature selection methods sidestep the issue by using labels, but in genomics, anomaly detection, or remote sensing, labels are often unreliable or simply missing.

The starting point here is Minkowski Weighted k-means (MWK), a variant that assigns each feature a per-cluster weight based on how tightly its values cluster around the centroid. Features with lower within-cluster spread get higher weights. The catch is that the computed weights are sensitive to the Minkowski exponent and initial centroids, so a single run can produce misleading rankings.