In this work, we interpreted the high braking index of PSR J1640-4631 with a combination of the magnetodipole radiation and dipole magnetic field decay models. By introducing a mean rotation energy conversion coefficient (zeta) over bar, the ratio of the total high-energy photon energy to the total rotation energy loss in the whole life of the pulsar, and combining the pulsar's high-energy and timing observations with a reliable nuclear equation of state, we estimate the pulsar's initial spin period, P-0 similar to (17-44) ms, corresponding to the moment of inertia I similar to (0.8-2.1) x 10(45) g cm(2). Assuming that PSR J1640-4631 has experienced a long-term exponential decay of the dipole magnetic field, we calculate the true age t(age), the effective magnetic field decay timescale tau(D), and ;the initial surface dipole magnetic field at the pole B-p (0) of the pulsar to be 2900-3100 yr, 1.07(2) x 10(5) yr, and (1.84-4.20) x 10(13) G, respectively. The measured braking index of n = 3.15(3) for PSR J1640-4631 is attributed to its long-term dipole magnetic field decay and a low magnetic field decay rate, dB(p)/dt similar to -(1.66-3.85) x 10(8) G yr(-1). Our model can be applied to both the high braking index (n > 3) and low braking index (n < 3) pulsars, tested by the future polarization, timing, and high-energy observations of PSR J1640-4631.