Opponent modeling is essential to exploit suboptimal opponents in strategic interactions. Most previous works focus on building explicit models to predict the opponents' styles or strategies, which require a large amount of data to train the model and lack adaptability to unknown opponents. In this work, we propose a novel Learning to Exploit (L2E) framework for implicit opponent modeling. L2E acquires the ability to exploit opponents through a few interactions with different opponents during training of a neural network and can quickly adapt to new opponents with unknown styles during testing. To automatically produce challenging and diverse opponents for training, we further present a novel opponent strategy generation algorithm. We evaluate L2E on two poker games and one grid soccer game, which are the commonly used benchmarks for opponent modeling. Comprehensive experimental results indicate that L2E rapidly adapts to diverse styles of unknown opponents.