Abstract: Modern transformer-based image encoders have redefined the field of computer vision, consistently setting new performance benchmarks across various tasks. Nevertheless, the inherent long-tail distribution of species in botanical datasets and incomplete representation of rare specimens in herbarium collections poses substantial challenges in accurately classifying diverse plant species.In this work, we propose a three-stage approach: first, we leverage a pre-trained vision transformer to extract robust image features; second, we introduce parameter specialization to address class imbalance; and third, we employ a residual fusion mechanism for unified predictions. We evaluate our method in few-shot, medium-shot and many-shot settings on the Herbarium 2021 and Herbarium 2022 datasets, and report state-of-the-art perfoemce on both datasets. These findings highlight the potential of transformer-based image encoders with targeted refinements — to advance biodiversity monitoring and botanical research.