Implementation, Development, and Evaluation of a Single-Camera Robotic Pick-and-Place System for Dynamic Object Detection, Tracking, and Pickup - Master's Thesis
This graduate-level thesis explored the development of a cost-effective, single-camera robotic pick-and-place system capable of detecting, tracking, and intercepting moving parts on a conveyor belt. Unlike many commercial systems that rely on multiple cameras, distance sensors, and complex calibration procedures, this project investigated whether similar tasks could be achieved with only an off-the-shelf robotic arm and a single end-mounted industrial camera. The research was motivated by the needs of small- and medium-sized manufacturers, who often cannot justify the expense and integration complexity of advanced multi-sensor systems. Over the course of one year, the project progressed from stationary object pickup to dynamic interception, developing and testing multiple predictive algorithms to calculate where and when a robot should intercept moving parts. The system demonstrated reliable performance and confirmed the feasibility of simplifying vision-based automation while maintaining practical accuracy. This work not only contributed technical insights into robotic vision and control but also provided significant experience in graduate-level research design, interdisciplinary collaboration, and the management of a professional research project from conception through validation.
Need & Idea
Many existing robotic pick-and-place systems achieve excellent precision, but they do so at high cost and with considerable setup requirements. They frequently rely on multiple sensors, such as LiDAR, infrared distance sensors, or external vision arrays, all of which require careful calibration and specialized software. This complexity makes such systems impractical for smaller manufacturers who need automation solutions that are accessible, adaptable, and affordable.The central question of this research was whether a single-camera approach could achieve reliable pickup without sacrificing too much performance. By minimizing hardware requirements and simplifying setup, the project sought to demonstrate that robotics could be made more approachable and scalable for companies with limited resources. In essence, the project explored whether automation could be made simpler without losing its effectiveness, opening opportunities for broader industrial adoption.
System Development and Setup
The experimental platform combined an Epson VT6 six-axis robotic arm, a parallel gripper, and a five-megapixel industrial camera mounted directly on the gripper using a custom-designed bracket. The mounting ensured that the camera’s perspective remained aligned with the gripper, reducing calibration requirements. The camera was integrated into Epson’s RC+ environment using OnRobot-compatible software, which allowed image processing and part recognition to run natively within the control system.
The setup process required careful coordination between hardware and software. First, the camera feed was calibrated to identify part orientation and position within a bounding box. Then, the offset between the gripper centerline and the camera centerline was measured and incorporated into the control program. Once this offset was defined, the robot could convert image coordinates into motion commands with sufficient precision to attempt pick-and-place operations. This stage of the project established the foundation for testing both static and dynamic pickup scenarios, ensuring that vision recognition and robotic actuation worked together as a cohesive system.
Static Pickup
Before tackling moving targets, the robot was tested on stationary parts. In this setup, the camera identified the object’s location and orientation, and the robot accounted for the fixed offset between the gripper and the camera to position itself accurately above the part. The robot then executed a vertical descent to the pickup height and closed its gripper to capture the part. This phase confirmed that the vision-to-motion pipeline was effective. By starting with static parts, the team validated that the robot could consistently identify objects, translate image data into motion commands, and perform successful pickups. While relatively straightforward, this stage was critical because it established that the system’s foundation was sound before introducing the added complexity of motion prediction on a conveyor.
Dynamic Pickup
Once static pickup was proven, the focus shifted to intercepting parts in motion. Because the conveyor introduced continuous movement, the robot needed a way to predict where and when to meet the part. Three different algorithms were developed, each with distinct advantages and limitations. The first method was a direct kinematic approach, which calculated when to descend based only on vertical travel time. This approach was simple and easy to run, but because it ignored diagonal travel, it often resulted in noticeable error, particularly at longer conveyor distances. The second method was the hypotenuse approach, which accounted for the full diagonal travel of the gripper as it descended toward the part. By considering both vertical and horizontal components of the motion, this method was more accurate than the direct kinematic approach. However, it required more complex calculations and placed heavier demands on the system’s computing and synchronization. The third method was an ambush strategy, where the robot pre-positioned itself at a point along the conveyor path, descended to the pickup height, and simply waited for the part to arrive. This eliminated some timing complexities but amplified the effects of even small errors in velocity estimation or trajectory prediction. While more reliable in some respects, it was also more sensitive to inaccuracies in motion tracking.
Testing & Results
Testing involved both the comparison of the three pickup methods and the evaluation of how key system variables affected performance. Conveyor speed was varied to assess how well each algorithm could adapt to faster-moving parts. Scanning height was adjusted to study how the field of view and perspective influenced detection accuracy. The number of velocity estimation iterations was also modified to determine the balance between prediction accuracy and system responsiveness. Finally, robot motion parameters such as acceleration and maximum speed were tested to explore their effect on interception success. The results showed clear trends. Faster conveyor speeds reduced accuracy across all methods, with the ambush method being particularly sensitive. Higher scanning heights decreased precision by reducing the resolution of image-based detection. More velocity iterations generally improved predictions, though excessive iterations delayed response times and reduced effectiveness. Among the three methods, the hypotenuse approach consistently proved the most balanced in terms of accuracy and reliability, while the direct kinematic method remained the simplest to implement, and the ambush method offered strengths in specific conditions but suffered when errors accumulated.
Conclusions and Takeaways
The research demonstrated that a single-camera robotic system is not only possible but also practical in certain contexts. While it cannot match the sub-millimeter precision of multi-sensor industrial platforms, it provides a compelling option for manufacturers that prioritize affordability and ease of integration. By eliminating the need for additional sensors and complex calibration, this approach lowers both cost and implementation barriers, making automation more accessible to smaller companies. More broadly, the project underscored the importance of balancing complexity with usability in industrial automation. A simpler system that performs reliably at moderate accuracy may be more valuable to many companies than an advanced but cost-prohibitive alternative. The work provides insight into where streamlined robotics can be most impactful and highlights opportunities for developing future low-cost automation solutions.
Completing this thesis provided extensive technical and professional growth. From a technical perspective, it deepened knowledge of industrial robotics, vision-based object recognition, motion control, and the integration of hardware and software into a unified system. It involved learning to work with Epson’s RC+ environment, programming vision routines, and refining algorithms to handle dynamic conditions. On the professional side, the project required managing a team of four undergraduate students from multiple disciplines, delegating tasks effectively, and guiding the project’s progress on a tight one-year timeline. It reinforced skills in project planning, data analysis, and experimental design, as well as the ability to adapt research goals in response to practical challenges. Perhaps most importantly, it offered insight into the realities of industrial automation. Innovation is not always about achieving the highest possible accuracy with the most advanced tools; it is often about finding ways to make systems simpler, cheaper, and easier to deploy. This project proved that significant impact can be achieved by focusing on accessibility and cost-effectiveness. By bridging academic research with practical industrial needs, the work demonstrated both technical capability and an applied understanding of the manufacturing environment. Full thesis paper download below.