|
Flexible operation of a robotic agent in an uncalibrated environment
requires the ability to recover unknown or partially known parameters of
the workspace through sensing. Of the sensors available to a robotic
agent, visual sensors provide information that is richer and more complete
than other sensors. However, the integration of vision sensors with a
robotic system raises a number of issues that have not been addressed by
traditional robotics and computer vision research. |
|
|
Professor Papanikolopoulos' previous research introduced a framework
called controlled active vision for efficient integration of the
vision sensor in the feedback loop. This framework emphasized eye-in-hand
robotic systems (the vision sensor is mounted on or close to the
manipulator's end-effector) and was applied to the problem of robotic
visual tracking and servoing with very promising results. Full 3-D
robotic visual tracking was achieved at rates of 30 Hz with targets moving
at maximum speeds of 80 cm/sec. Most importantly, the tracking was
successful even under the assumption of poor calibration of the
eye-in-hand system. Algorithms that incorporated the use of multiple
windows and numerically stable confidence measures were combined with
stochastic controllers in order to provide a satisfactory solution to the
tracking problem. The special relations between the characteristics of
computer vision and control algorithms were highlighted through a series
of experimental results. |
|
|
One of the most important contributions of this work was the introduction
of adaptive control techniques in order to compensate for inaccurate
modeling of the environment, such as depth estimation. One example was in
the case of inaccurate knowledge of the features' depth where the
displacements were fed to adaptive SISO controllers that estimated the
desired robot motion. The human operator was integrated with the
autonomous servoing modules through a sophisticated system architecture. |
|
|
We have also developed robust techniques for the derivation
of depth from a large number of feature points on a target's surface and
for the accurate and high-speed tracking of moving targets. These
techniques are used in a system that operates with little or no a priori
knowledge of the object-related parameters present in the environment. The
system is designed under the Controlled Active Vision framework and
robustly determines parameters such as velocity for tracking of moving
objects and depth maps of objects with unknown depths and surface
structure. Such determination of intrinsic environmental parameters is
essential for performing higher level tasks such as inspection,
exploration, tracking, grasping, and collision-free motion planning. For
both applications, the Minnesota Robotic Visual Tracker (a single visual
sensor mounted on the end-effector of a robotic manipulator combined with
a real-time vision system) is used to automatically select feature points
on surfaces, to derive an estimate of the environmental parameter in
question, and to supply a control vector based upon these estimates to
guide the manipulator. |
|
|
Five years ago, we implemented a flexible system that performs autonomous grasping of a moving target in a partially calibrated environment. The object of
interest is not required to appear in a specific location, orientation, or
depth, nor is it required to remain motionless during the grasp. The
proposed system was derived using the Controlled Active Vision framework
and provided the flexibility to robustly interact with the environment in
the presence of uncertainty. The proposed approach was experimentally
verified using the MRVT system to automatically select targets of interest
and to guide the manipulator in the grasping of a target. More recently,
the grasping system was expanded and improved with the use of pressure
snakes. |
|
|
We have also worked on a model-based approach for visual
tracking and servoing in robotics. Deformable active models are proposed
as an effective way for tracking a rigid or semi-rigid (possibly partially
occluded) object in movement within the manipulator's workspace.
Deformable models imitate, in real-time, the dynamic behavior of elastic
structures. These computer-generated models are designed to capture the
silhouette of rigid or semi-rigid objects with well-defined boundaries, in
terms of image gradient. They consist of several hundred control points
that are processed in parallel. By means of an eye-in-hand robot arm
configuration, the desired motion of the end-effector is computed with the
objective of keeping the target's position and shape invariant with
respect to the camera frame. Optimal estimation and control techniques
(LQG regulator) are used in order to deal with noisy measurements
provided by the vision sensor. Experimental results show that the
deformable models achieve robust tracking even when the target is
partially occluded. These techniques have been applied to assembly tasks
that involve a combination of computer vision and force robot control.
|