Abstract: The traditional coal gangue sorting method has problems such as high cost, low efficiency, and insufficient safety. In recent years, with the rapid development of deep learning related technologies, intelligent gangue sorting based on object detection algorithms has become an important research direction for gangue sorting. To achieve efficient sorting of gangue and coal blocks, this paper proposes a detection method based on a combination of machine vision and deep learning. This method is based on the YOLOv5s model. Firstly, a convolutional attention module （CBAM） is added to the backbone to improve the network’s feature extraction ability; secondly, a weighted bidirectional feature pyramid structure （BiFPN） is adopted in the neck region to enhance the multi-scale feature fusion of the network and avoid the occurrence of missed and false detections; in the prediction section, the EIoU function is used as the loss function of the improved model to further improve detection accuracy; finally, before training, expand the original dataset to further enhance the model’s generalization ability. The experimental results show that the average detection accuracy of the improved model is 95.3%, which is 2.1% higher than the original model and can effectively replace manual sorting.