This paper designs a visual surveillance framework for human fall detection. In order to solve the conventional issues in fall detection, such as unsatisfactory feature generalization, low recall rates, and large computational time, we design a model that incorporates the deep convolutional neural network and the aggregated heuristic visual features in detecting the occurrence of falls. Firstly, the convolutional neural network (Openpose model) is utilized to extract human skeleton in the image. Secondly, the hand-crafted spatial features, such as the angle of human shank inclination, are aggregated to determine the fall presence. It should be noticed that our fall detection method has been integrated to healthcare IoT video surveillance architecture which has multiple GPU groups to perform real-time monitoring and alarming for the elderly in need. The experimental results prove that our method is able to accurately distinguish fall and non-fall activities with a competitive false-alarm rate.