We propose a learned inverted residual cascaded group small target detection transformer (LIS-DETR) model, a novel approach for small object detection in autonomous driving. This model has a unique backbone network, the basic inverted residual cascaded group mobile block, which enhances feature representation and reduces computational redundancy. A dedicated small detection layer is integrated to improve small object detection specifically. In addition, an adaptive learned positional encoding transformer layer is incorporated to strengthen global contextual relationships, and the designed inner-SIoU loss function further accelerates convergence speed. Experimental results show a 3.1% increase in mAP50 accuracy on VisDrone datasets and a 1.9% improvement on processed SODA10m datasets compared with baseline methods. These advances demonstrate the LIS-DETR model’s strong generalization ability and the significant potential to enhance the efficacy of autonomous driving systems.