I've been building this project as a way to learn computer vision.

I've been documenting the build progress on my YouTube channel: https://www.youtube.com/channel/UCcT3iGeJJfpGU1QehvG48tg/videosw

Currently the project sort of works as an initial version, but I may come back to it at some point and try to improve it. If you want to see the final testing watch this video, and then below I'll put more details about the build process

Machine Vision: The OpenCV Autonomous Nerf Turret

The Autonomous OpenCV Turret is absolute peak robotics engineering. It crosses over from basic Arduino sensor logic into the realm of Python-based A.I. image processing via a PC or Raspberry Pi, using the Arduino purely as a slave actuator.

I started with creating a prototype to test the face-tracking without the nerf gun, which would both mean it would be lighter and easier to quickly to together, and also that I wouldn't be constantly shot by nerf bullets whilst trying to get the software working.

The testing setup comprises of two servos, one for pan and one for tilt, with a webcam mounted on top, as shown below:

The Python Face detection

The Arduino cannot process video. Your PC webcam must handle the image.

1. OpenCV (Python): The Python script grabs the 1080p webcam feed. It uses the Haar Cascade Classifier library, explicitly trained to look for human faces.
2. The Bounding Box: The instant someone walks into the frame, Python draws a green tracking box around their face. It calculates the exact X/Y center coordinates of that box (e.g., X: 450, Y: 300).
3. Serial Comm: The Python script shoots an immediate command via the USB cable to the Arduino: X450Y300\n.

The webcam is plugged into a computer running a python script with OpenCV. That detects a face, and sends the coordinates of where it is within the image to an Arduino, which moves the pan and tilt servos so that the centre of the face (green dot) is within the centre of the image (white square). I used a guide by Shubham Santosh (https://create.arduinocontent.cc/projecthub/shubhamsantosh99/face-tracker-using-opencv-and-arduino-55412e) for this part, so thanks to him for pretty much half the project!

Translating Coordinates to Servos

The Arduino is waiting.

1. The Parsing: It reads the string, extracting 450 and 300.
2. The Mapping: It maps the massive screen resolution (1920x1080) into the physical 180-degree limits of the Pan and Tilt MG996R Servos.
3. The Fire Sequence: If(Abs(targetX - currentX) < 10) (Meaning the barrel is perfectly locked on the center of the face), the Arduino clicks a heavy-duty relay.
   • The relay dumps 12V into the heavily modified, motorized Nerf blaster's firing mechanism, launching a barrage of foam darts!

I could then design and print a larger and more sturdy two axis mount for the nerf gun, and a firing mechanism which is triggered when the face is within the centre of the screen (shooting me in the face, apologies for my lack of eye protection!)

Critical Machinery

• A PC running Python 3 and OpenCV.
• A Motorized Nerf Blaster (e.g., Stryfe or Nemesis).
• Massive Pan/Tilt Turret mounting rig.
• Arduino Mega: To parse the rapidly changing serial strings and drive the servos simultaneously.
• Relay Modules / MOSFETs: For the firing solenoids.
• (Note: Always wear eye protection when testing autonomous launching equipment).

And after that, all that was left to do was test it!

It works, although currently motion is a bit jerky, and it might be worth adding in a feature to scan the room if no face is detected.

To see the test (and me get shot in the face) check out the testing video: https://www.youtube.com/watch?v=N5vzpi4yF7A&ab_channel=AdamBeedle