Advances in aerospace engineering and aerodynamics have pioneered space exploration and helped support telecommunication infrastructure. But these same developments have also aided in the creation of weapons of devastating impact. This necessitates the development of ways for detecting and tracking rockets. While several methods, mostly based on Doppler radar exist, the need for active radio emissions limits the applicability of these systems. A passive system has several advantages over traditional techniques, however their potential is largely unexplored. This work seeks to tackle this research gap by exploring the potential of emerging computer vision townies applied to rocket detection and tracking. The advantages of such a system are the relatively low cost as well as passive nature making observation stations harder to detect and easier to deploy. This work explores the potential of pre-trained, lightweight YOLOv8 architectures for rocket detection in real-world situations. A publicly available dataset is utilized and a comparative analysis is carried out between nano and small models. Both models demonstrate favorable outcomes with an accuracy of 0.90 for rocket body detection and 0.93 for engine flame detection. Nevertheless, rocket detection into space is still difficult, with a precision of 0.64 for this class. This paper indicates areas for additional refinement and demonstrates the potential of computer vision technology in passive rocket detection.