The scarcity of annotated data, particularly for rare diseases, limits the variability of training data and the range of de- tectable lesions, presenting a significant challenge for super- vised anomaly detection in medical imaging. To solve this problem, we propose a novel unsupervised method for med- ical image anomaly detection: Attention-Augmented Differ- entiable top-k Feature Adaptation (ADFA). The method uti- lizes Wide-ResNet50-2 (WR50) network pre-trained on Ima- geNet to extract initial feature representations. To reduce the channel dimensionality while preserving relevant channel in- formation, we employ an attention-augmented patch descrip- tor on the extracted features. We then apply differentiable top- k feature adaptation to train the patch descriptor, mapping the extracted feature representations to a new vector space, en- abling effective detection of anomalies. Experiments show that ADFA outperforms state-of-the-art (SOTA) methods on multiple challenging medical image datasets, confirming its effectiveness in medical anomaly detection.