Alternative Approaches

The vision-based approach using pretrained CNNs proved unreliable due to insufficient training data and high visual variability across video sources. Here are two alternative approaches that would be more robust and accurate for production use.

---

1. Fiducial Markers for Pose Estimation

Attach small printed markers (like AprilTags or ArUco markers) to both the pipette and tube, then use well-established computer vision libraries to detect their 3D pose.

How It Works

1. Print markers — Attach an AprilTag or ArUco marker to the pipette body and another to the tube holder
2. Detect markers — Use OpenCV's cv2.aruco module or the apriltag library to detect markers in each frame
3. Estimate pose — With known marker sizes and camera intrinsics, compute the 3D rotation and translation of each marker
4. Calculate angle — The relative angle between pipette and tube is simply the difference in their orientations

Why It's Better

Advantage	Explanation
Sub-degree accuracy	Marker detection provides precise 3D pose estimation
Lighting invariant	Binary patterns work under varying lighting conditions
No training required	Uses geometric algorithms, not learned features
Real-time capable	Detection runs at 100+ FPS on standard hardware
Well-documented	Extensive libraries and tutorials available

2. Accelerometers / IMUs

Attach small inertial measurement units (IMUs) to both the pipette and tube to directly measure their orientations.

How It Works

1. Mount IMUs — Attach a 6-axis or 9-axis IMU to each component (pipette and tube)
2. Calculate relative angle — Subtract the tube's orientation from the pipette's orientation

Why It's Better

Advantage	Explanation
Direct measurement	Measures actual physical orientation, not visual features
No occlusion issues	Works even when camera view is blocked
High precision	Modern IMUs achieve < 0.1° accuracy with sensor fusion
No camera required	Eliminates all vision-related failure modes
Works in any lighting	Including low-light (i.e., uv-sensitive) experiments

Recommendation

For a production system at Transfyr:

1. Short-term: Add AprilTag markers to the equipment and use OpenCV for detection. This requires minimal hardware changes and provides immediate accuracy improvements.

2. Long-term: Integrate IMUs into the pipette and tube fixtures. This provides the most reliable measurements and eliminates all vision-related failure modes.

Both approaches are battle-tested in robotics and industrial automation, with extensive documentation and community support.