A style-based generative adversarial network (Style- GAN2) yields remarkable results in image-to-latent embedding. This work proposes a Detached Dual-channel Domain Encoder as an effective and robust method to embed an image to a latent code, i.e., GAN inversion. It infers a latent code from two aspects: a) a detached dual-channel design to support faithful image reconstruction; and b) a local skip connection that allows conveying pieces of information with image details. We further introduce a hierarchical progressive training strategy that allows the proposed encoder to separately capture different semantic features. The qualitative and quantitative experimental results show that the well-trained encodercan embed an image into a latent code in StyleGAN2 latent space with less time than its peers while preserving facial identity and image details well.