Knowledge reasoning has been widely researched in knowledge graphs (KGs), but there has been relatively less research on hyper-relational KGs, which also plays an important role in downstream tasks. Existing reasoning methods on hyper-relational KGs are based on representation learning. Though this approach is effective, it lacks interpretability and ignores the graph structure information. In this paper, we make the first attempt at symbolic reasoning on hyper-relational KGs. We introduce rule extraction methods based on both individual facts and paths, and propose a rule-based symbolic reasoning approach, HyperPath. This approach is simple and interpretable, it can serve as a baseline model for symbolic reasoning in hyper-relational KGs. We provide experimental results on almost all datasets, including five large-scale datasets and seven sub-datasets of them. Experiments show that the expressive power of the proposed model is similar to simple neural networks like convolutional networks, but not as advanced as more complex networks such as Transformer and graph convolutional networks, which is consistent with the performance of symbolic methods on KGs. Furthermore, we also analyze the impact of rule length and hyperparameters on the model's performance, which can provide insights for future research in hypergraph symbolic reasoning. The code is available at https://github.com/von1000/HyperPath.