In this paper, we work on analyzing old paper advertisement (Ad). An Ad usually contains various types of textual and non-textual objects, which may also be in different orientations. We attempt to detect such objects from an early Indian-print paper Ad database comprising 1500 Ad images. The past major object detectors did not perform well on this database. We propose a deep reinforcement learning-based orientation-aware object detector. Our system learns by itself where to look and what to look of an Ad image. Therefore, it can bypass the impeding zone due to degraded image quality. To find the looking spot, we come up with a foveal transformation. In reinforcement learning, we present a scheme for shaping an internal reward with a top-up. For oriented object detection, we also propose a generic loss function. Our system obtained encouraging results from the experiments performed on the Ad database.