In the simulation 1 - moving target in a straight line in Fig. 6,

it is found out that the error converges to zero. When target follows a

circle (simulation 2), the system can track targets, but the error is still

great so it needs to improve. When the function cos or sin is changes

from positive to negative and in the opposite at ¼, ¾, 0, ½ circular of

the circle, the tracking target error is fluctuations, but it still tracks the

target. Different from chapter 1, it lost tracking at ½ circular. In

simulation 3, 4, when the target moves following a rectangle and

random in the space, the system still captures. When the moving target

simultaneous changed orientation and velocity, the initial error of the

system increases, but it still tracks target rapidly because external

effects as well as the effects of uncertain parameters in the system

model are well compensated by the neural network controller based

on-line learning algorithm. The impact of noise and uncertain

parameters is reduced. Otherwise, image and robot Jacobian matrices

(Jimg and Jrobot) have not a singularity. This condition helps the system

move follow in the complex way

27 trang |

Chia sẻ: honganh20 | Ngày: 21/02/2022 | Lượt xem: 240 | Lượt tải: 0
Bạn đang xem trước 20 trang tài liệu **Tóm tắt Luận án A research into the control of mobile robot manipulator to track target based on visual information**, để xem tài liệu hoàn chỉnh bạn click vào nút DOWNLOAD ở trên

, Thai Nguyen,
Dec 12-14, pp. 217 – 227.
[5] Le Van Chung, Pham Thuong Cat (2014), “Robust visual tracking
control of pan tilt – stereo camera system g i
pp.167-173.
[6] hu g h h g t i u h 013), “Phương
pháp điều khiển hệ servo thị giác stereo sử dụng bệ Pan-Tilt -
- g - 382.
[7] hu g “Phát triển hệ pan/tilt – nhiều camera bám mục tiêu
di động”, Thai Nguyen University Journal of Science and
Technology, 116(02), tr. 41-46.
1
INTRODUCTION
In recent years, there has been a great deal of research on
robotics control using visual information. But the achieved results still
reveal some limitations. For example, using a camera on a mobile
robot only allows full tracking of the target's information when
knowing the target's moving plane. As with the use of two cameras on
the pan tilt system, but not considering the deterioration of the
Jacobian matrix affecting the grip capacity of the system. Besides, the
mathematical model of a robot is often difficult to achieve the
required accuracy because there are many unspecified parameters in
the system such as measurement of parameters or coefficient of
friction, inertia mo e t etc, usually changes during operation. In
addition, it is difficult to optimize the parameters in the robot
controllers to achieve the desired accuracy.
With the above reasons, the author has chosen the topic:
“A research into the control of mobile robot manipulator to track
target based on visual information to develop some control
algorithms that use image information with many uncertain
parameters.
The objective of the thesis
The main object of the research is to focus on pan-tilt robots
and mobile robots with wheels.
Scope of research
Researching methods of controlling pan-tilt tracking moving
target using image information from 2 cameras with many uncertain
parameters.
Developing algorithms for controlling integrated system
include mobile robot, pan-tilt and two cameras tracking moving
targets.
New findings of the doctoral dissertation
1- The application of artificial neural networks in compensate for
uncertainties in a pan-tilt system model with two cameras. Based on
that, the kinematic and dynamic controllers are constructed for the
pan-tilt two cameras system to track moving targets with uncertain
parameters.
2- The thesis has developed a dynamic model for the integrated
system that including mobile robot, pan-tilt and stereo cameras. Also,
2
the thesis has built two control methods, sliding mode controller and
quadratic perfofmance optimal controller for the above integrated
system.
The layout of the thesis
Chapter 1: Overview.
Chapter 2: Developing controller for pan-tilt stereo camera system
to track the moving target.
Chapter 3: Some improvements in controlling servo system to
track the target.
Chapter 4: Developing control method for mobile robot.
CHAPTER 1
OVERVIEW
1.1 Problem?
In order to control a robot system using two cameras to work
better, the problem is:
The first: Developing methods to control pan-tilt systems using
image information to track moving targets when there are uncertain
parameters.
The second: Build a Jacobian matrix image is a square matrix for
the system to track the moving target for the better system.
The third: Developing some methods to control the integrated
system combining a mobile robot with a pan-tilt robot that carries two
cameras to track the moving target and be able to move closer to the
target in the space.
1.2. Overview of controlling robots using visual information
When using a pan-tilt robot with two cameras, another
problem poses the degradation of the Jacobi matrix when taking
inverse pseudocode. When using a camera, the Jacobian matrix of the
pan-tilt system - a camera is square and invertible. But when using 2
cameras, the Jacobian matrix of the system will be (3x6), the Jacobian
matrix of the system will be (3x2). So in transformations we have to
take the inverse pseudo cause that is the cause of singularities.
3
Hình 1.2 Some methods of controlling robots
1.3 The research issues of the thesis
- Building kinematic/dynamic model of pan-tilt robot and
developing classical control method combining with Neural Network
to get better results including:
- Research and improve to build a square image Jacobian matrix
- Developing advanced control methods for robots when having
uncertain parameters.
- Building dynamic models for the integrated system that
including mobile robots, pan-tilt with 2 cameras and controllers for
the above system.
- Using Lyapunov stability method and Barbalat's lemma
proves stability and Matlab to verify results.
Chapter 2
DEVELOPING CONTROLLER FOR PAN TILT STEREO
CAMERA TO TRACK MOVING TARGET
In this chapter, the kinematic control algorithm combined with
the neural network is built to control the rotation angle of the pan - tilt
robot so that the target image is always maintained at the desired
Optimal
paramet
ers
Optimal
output
Control robots using visual
information
Classical method
Kine-
matic
Dynam
-ic
Modern method
Sliding
mode
Combined with
neural network
Optimal control,
adaptable
Modern
methods
with
neuron
network
Classical
methods
with
neuron
network
4
position on the model of the image frames. The content of the chapter
consists of 4 main parts: building a kinematic model of the pan-tilt
stereo camera system, designing control algorithm, verifying and
comparing with the controller not associated with Neural network and
conclusions.
2.1. Kinematic model of stereo visual servoing system with
uncertain parameters
2.1.1 Determination of Image Jacobian matrix:
Fig 2. 1 Pan-tilt PTU-D48E-Series & camera coordinates
The velocity of t rget’s i ge o c er s:
( )imagm = J m v (2.9)
- is the image Jacobi matrix:
- v is the velocity vector of the camera system.
2 2
( ) 2
0
2
2 2
( ) ( )
0
2 2
2 2
( ) 2
0
2
imag
U U U U U U V f U U UR L L L R L L L L R VL
B fB f f
U U V U U f V V U U U UR L L L R L L L R L R
B fB f f f
U U U U U U V f U U UR L R L R R L R L R VL
B fB f f
J (m)
(2.10)
2.1.2 Kinematic equations of Pan-Tilt platform:
Denote robotJ the Jacobian of the Pan-Tilt platform, we have:
(2.11)
imagJ (m)
( )robotx = J q q
5
Fig 2. 2 Camera system model
2.1.3 Formulation of stereo visual servoing problem with uncertain
parameters:
We calculate the image feature error: dε m m
The kinematic control problem of stereo visual servoing is to find
control law ( )q = K ε to control the system track the moving target so
the tracking error ε converges to zero
imagm = J (m)u ; robotv = x = J (q)q ; q = K(ε)
2.2. Control law design
In this chapter, a neural control method is proposed for Pan -
Tilt - stereo camera system to track a moving object when there are
many uncertainties in the parameters of both camera and Pan-Tilt
platform.
ˆ ,
d d 1ε =m-m = Jq+f -m -Kε+f +u (2.21)
The structure of the chosen artificial neural network is of RBF
type as shown in Fig.6. It has three layers: the input layer includes the
three components of error ε , the output layer includes 3 linear
neurons and hidden layer contains neurons with Gaussian output
function:
; j = 1,2,3 (2.22)
2
2
exp
j j
j
j
c
6
Fig 2. 6 Structure of proposed visual tracking system with many
uncertain parameters.
Control Method 1:
The stereo camera system described by the model in Esq.
(2.9), (2.11), with uncertain parameters controlled by the neural
network defined by Eqs (2.22), (2.23) will track moving targets with
the error , ε ε 0 if the speed of the Pan-Tilt joints is determined by
the Eqs. (2.25), (2.26), (2.27) and learning rules (2.28):
ˆ ˆ ˆ+ , + + ,
d 1 d 1q J [(m - Kε)+u ] J (m -Kε)+J u 0 1= u +u (2.25)
0
ˆ +
du = J m -Kε (2.26)
1
ˆ
,
1
+ ,
1 1
ε
u = ( )Wσ -
ε
u = J u
(2.27)
T
W εσ (2.28)
where K is a positive definite symmetric matrix. TK = K > 0 , the
coefficients 1, 0 .
Proof:
Chose a positive definite candidate Lyapunov function as
follows:
3
1
1
2
i i
i
V T Tε ε w w (2.29)
0V Tε Kε ε (2.37)
2.3 SIMULATION
Simulation 1: Fixed target
Camera center at the initial time: m(0) = [-40, 30, 0] (pixel);
Image coordinates of the target stand still at mt= [-20, 0, 20] (pixel);
7
Fig 2. 7 Image feature error when
using neural control
,
1u 0
Fig 2. 8 Image feature error when
no neural control is used
,
1u = 0
Simulation 2: Moving target in a straight line
Moving target from point A(0m,1.8m,0m) to B(-0.3m, 1.8m, -
0.5m) on the plane ZCOCXC
Fig 2. 9 Tracking error coordinates when the target moves along a straight
line.
Fig 2. 10 Image feature error coordinates when the target moves
along a straight line.
8
Fig 2. 11 Tracking error
coordinates when the target
moves along a straight line and
no neural control is used.
Fig 2. 12 Image feature error
coordinates when the target
moves along a straight line and
no neural control is used.
Simulation 3: Moving target in an arc
The target follows the circular arc with center coordinates at the origin
O(0, 0), radius r = 1 on the plane ZCOCXC
Fig 2. 13 Tracking error coordinates when the target moves along an
arc.
Fig 2. 14 Image feature error coordinates when the target moves
along an arc..
Simulation 4: Moving target in arcs with acceleration.
9
The target follows the circular arc similar to the arc of
simulation 3. However, in the first 1/6 arc, the velocity increases
steadily with acceleration 1cm/s
2
, after 3 seconds it will move with the
constant speed 3cm/s. In 1/6 end of the arc, its moving speed reduces
with the deceleration -1cm/s
2
Fig 2. 15 Tracking error coordinates when the target moves along an
arc with changing
Fig 2. 16 Image feature error coordinates when the target moves
along an arc with changing
2.4. Conclusion of chapter 2
This chapter proposes a new control method for the stereo
visual servoing system using neural networks with on-line learning
rules to compensate for the impacts of uncertain parameters such as
inertial torque, Jacobian matrix, friction in the joints, noise effects,
etc he ro osed ethod gu r tees the st bility of the over ll
system and eliminates tracking error. Control algorithm is highly
adaptable and is able to resist the noise impact on the system. The
global asymptotic stability of the whole system is proved by the
10
Lyapunov stability theory. The simulation results with the
uncertainties up to 20% in the case of fixed target, moving target in a
straight line or circular arc, show that the tracking error converges to
zero. These results are consistent with the theoretical.
.
CHAPTER 3
SOME IMPROVEMENTS IN CONTROLLING
SERVO SYSTEM TO TRACK TARGET
As in chapter 2, the author has noticed that in order to control
the pan- tilt system with two cameras to track target work well, the
control problem has some issues to solve as follows:
- Firstly, it is necessary to build a square image Jacobian matrix
so that performing the inverse and avoid singular points leading to
losing grip.
- The second is building dynamic controllers in combination
with neural networks to compensate for the effects of uncertainty
parameters inside the model as well as external noise.
- The third is to optimize some parameters in the neural network
to get better outputs.
In Part 1 of this chapter, the author built a 3D model for two
camera system to obtain the full Jacobian matrix. In part 2, the author
built the dynamic controller using neural network with optimized
parameters, the stability of the system is demonstrated by Lyapunov
method and Barballat lemma. Part 3 is the simulation results. Finally,
some conclusions.
3.1 3D visual model for eye-in-hand stereo camera system
3.1.2 3D virtual stereo camera model systems
A 3D visual space is built according to the following steps:
First step, from geometrical relations between the target and that
feature images we calculated the coordinates of the target point
Tc zyxx in the camera coordinate frame cccc ZYXO
Second step, a reference coordinate frame with the origin
located at the same position as is defined. In order to
transform OC to OV, the rotation matrix
(Fig 3.2) is used. The
vvvv ZYXO
cccc ZYXO
v
CR
11
Fig 3.2 3D visual stereo camera model
projection of Tc zyxx in is defined in
as Tvvvv yxzx
Third step, the reference coordinate frame vvvv ZYXO is used to
define two virtual camera's frame 1111 vvvv ZYXO , 2222 vvvv ZYXO
associated with stereo cameras. Their location on Xv and Zv axes are
far away from Ov the distances λ
Last step, the virtual camera model is combined with 3D visual
camera model to construct a 3D virtual Cartesian space having feature
point vector denoted as: Tvvvs xzz 121x . vimgJ is the visual
Jacobian matrix: vvimgvs f xJx (3.9)
vv
v
vv
v
v
v
v
vimg
zz
y
zz
x
x
z
x
1
0
)(
0
1
)(
0
)(
1
2
2
2
J (3.10)
3.1.3 Avoid singularity
cccc ZYXO vvvv ZYXO
12
The singularity of vimgJ that can be avoided by choosing the
r eter λ such s λ > x xv, zv).
3.1.4 Stereo visual servoing problem with uncertain parameters
0)
ˆ( xJqJJx C
v
vimgvs Rf
. (3.19)
3.2. Dynamics of robot manipulator with uncertainties.
The dynamics of a serial n-link rigid robot with friction, and
uncertainty can be written as follows:
τdqgqqqCqqM )()(),()( t (3.20)
3.3. Robust neural control of stereo camera system with pan tilt
robot.
3.3.1 Construction of robust controller
The control law is chosen as follows:
0 1( )D P NR τ A K ε K ε b τ τ τ , (3.30)
0 ( )D P τ A K ε K ε b , (3.31)
3.3.2 Layer construction of RBF neural network
The structure of choosing artificial neural network is a Radial
Basis Function (RBFNN) network. It has 3 layers.
Input layer is vecto Tsss 321s
Hidden layer computation The hidden layer consists of neurons with
output function calculated by Gaussian form.
Output layer The output values of the network are approximate
function f1
Control Method 2: The image error dynamics (3.32b) of the
uncertain pan-tilt – Stereo camera tracking system (3.19), (3.20)
will be asymptotically stable with the error if the control
torque is chosen by following (3.36), (3.37) and online learning
rules (3.38):
(3.36)
(3.37)
, (3.38)
0ε
( )D P NR τ A K ε K ε b τ
1NR
ε
τ A Wσ
ε
T W sσ
13
Fig 3.6 Structure of proposed visual tracking system
Proof: We choose the candidate Lyapunov function as follows:
3
1
2
1
2
1
),(
i
i
T
i
TV wwssWs . (3.39)
Taking the derivative of V along time, yields:
0 sGss TV (3.49)
2 T TV
s s
s G β Wσ β Gs
s s
(3.50)
We found that V is bounded because βs, are bounded,
s
s
is the
unit vector, G is the positive-definite constant matrix and 0, .
Thus V is uniformly continuous. According to Barbalat's lemma, we
have 0s when 0t and it forces 0ε and the system is
asymptotically stable
4. Simulation results
Simulation 1: Moving target in a straight line from point A
(0m,3m,0m) to B (-0.5m, 3m, -0.3m)
14
Fig 3.7. Tracking error coordinates when moving target in a straight
line
Simulation 2: Moving target in a circle O (0, 0, 0), r = 1
Fig 3.9. Tracking error coordinates when moving target follows a
circle.
15
a) b) c)
Fig 3.11.a) Tracking error coordinates in X, Z axes. b) Torques of pan
and tilt joints. c) Joints angle q
Simulation 3: Moving target in a rectangle with changing velocity.
Moving target follows the rectangle as figure 8 from point (-1m, 3m,
0m) to (-1m, 3m, 1m) in 10 seconds. In the next 10 seconds, target
moves from point (-1m, 3m, 1m) to (1m, 3m, 1m) and the same to
return to the starting point.
Fig 3.12 Tracking error coordinates when moving target is in a rectangle
with changing velocity.
16
a) b) c)
Fig 3.13 a) Tracking error coordinates in X, Z axes. b) Torques of pan
and tilt joints. c) Joints angle q
Simulation 4: Moving target in space with random velocity and
direction. The start of target at point (3, 3, 0.5) in O0 coordinates; yt =
0.1t. The movement of the target following x, z axes is plane-parallel
motion: 0 ≤ t <5s: v = /s) ω = r d/s) 5 ≤ t <10s: v = 0.5 +
0.15sin((t- )π/ ); ω = 0.15sin((t-5)π/10); 10 ≤ t <15s: v = 0.75. ω =
0; 15 ≤ t <20s: v = 0.75+0.15sin((t-15)π/10); ω= -0.15sin((t-15) π/10);
20 ≤ t <25s: v = 0.75. ω = -0.15; 25 ≤ t <30s: v = 0.75. ω =-0.15 -
0.15sin((t-25)π/10); t >30s: v = 0.5, ω = -0.3. λ = 20.
Fig 3.14 Tracking error coordinates when moving target is in
space with random velocity and direction.
17
a) b) c)
Fig 3.15 a) Tracking error coordinates in X, Z axes. b) Torques of
pan and tilt joints. c) Joints angle q
In the simulation 1 - moving target in a straight line in Fig. 6,
it is found out that the error converges to zero. When target follows a
circle (simulation 2), the system can track targets, but the error is still
great so it needs to improve. When the function cos or sin is changes
from positive to negative and in the opposite at ¼, ¾, 0, ½ circular of
the circle, the tracking target error is fluctuations, but it still tracks the
target. Different from chapter 1, it lost tracking at ½ circular. In
simulation 3, 4, when the target moves following a rectangle and
random in the space, the system still captures. When the moving target
simultaneous changed orientation and velocity, the initial error of the
system increases, but it still tracks target rapidly because external
effects as well as the effects of uncertain parameters in the system
model are well compensated by the neural network controller based
on-line learning algorithm. The impact of noise and uncertain
parameters is reduced. Otherwise, image and robot Jacobian matrices
(Jimg and Jrobot) have not a singularity. This condition helps the system
move follow in the complex way.
Fig 3.17. Tracking error coordinates without part of neural network in
the controller when target moves following a straight line and a circle
3.5. Conclusion
18
The problem of visual tracking control of pan-tilt-stereo camera
system is dialled with when there are uncertainties in both the
Jacobian matrix and the dynamics of the system. A 3D virtual stereo
camera model having a full-rank of image Jacobian is constructed to
solve local minima and image space singularity problems of the
classical image Jacobian.
Furthermore, the authors propose a robust visual control scheme with
online learning RBF neural network to compensate the effects of
uncertainties. Due to online learning algorithms of RBF neural
network continuously update is done only with some multiplication
and integral. Therefore the calculations should not be too large, in
accordance with real-time system. The asymptotic stability of the
overall system is proved by Lyapunov stability method. Simulations
show that the proposed control scheme is relatively effective even in
the case of the target moving in a circle or rectangle meanwhile the
dynamics and the Jacobian of the pan-tilt robot are not known exactly.
In the other hand, all parameters in the controller can be optimized to
have better results. In this paper, I only optimized two parameters.
Other parameters such as ƞ, δ, G, H also can be optimized. I will
consider all effect of the optimization parameters in the next step of
my research.
Chapter 4
DEVELOPING CONTROL METHOD FOR MOBILE ROBOT
In this chapter, the system is adjusted so that the target image is
always maintained at the center of the camera coordinate system. The
wheels are put in the right forces to move and approach the target with
time and energy consumption is optimal. The chapter content consists
of 4 main parts: building the dynamic model, dynamic of the mobile
robot – pan tilt - stereo camera system, designing a controller to
control the integrated system to approach the target. In the shortest
period of time, simulating verification and conclusions. In this
chapter, the author presents the optimal control algorithm and sliding
mode control algorithm combined with the direct moment to control
the whole system.
19
Fig 4.1 Structure of mobile robot- pan/tilt-stereo camera
4.1. Construct the kinematic model and kinematic controller for
mobile robot - pan tilt - stereo camera to track a moving
target
4.1.1 Determination of Image Jacobian matrix
The images Jacobian is defined follows the equal (2.10).
4.1.2 Kinematic controller for pan-tilt robot.
The desired velocity of pan - tilt joints is:
as
1 as1 1
as
2 2
( )
l t
l td
pl t
d
m
J Km m
m
(4.13)
The desired coordinates, directions of the mobile robot is calculated
by:
0
1
0
1
0 0
cos
sin
atan 2( , )
md t b
md t b
md b t t
x x k
y y k
y x
(4.16)
4.1.3 Determination of the speed of the mobile robot wheels to reach
to the target.
The desired angular velocity of the two wheels of the mobile robot is:
1
/ 4 / 4
/ 2 / 2
rd m
ld m
vd d
d k d k
(4.21)
20
4.2. CONTROLLER FOR DYNAMIC SYSTEM
4.2.1 Dynamic model for mobile robot – pan/tilt system
The dynamics of integrated mobile robot – pan - tilt robot
with no friction or effect of noise can be written as follows:
( ) ( , ) ( )s s sM q v C q v v g q = τ (4.22)
The tracking error e1 is:
1 d se v v (4.25)
4.2.2 Optimal Controller Design
We choose control law as follows:
*
1τ h He u (4.30)
* 1 1
2
( , )
( )
T
T F tt
s
u R B R e
s
(4.43)
Control method 3: Optimal control signal u* (4.43), control law
(4.30) will make the dynamic system in the equation (4.22) tracking
the target with desired velocity and the objective function (4.33)
minimum. If the parameters satisfy the condition (4.39, 4.40, 4.41).
Fig 4.5 Structure of system
4.2.3 Design Adaptive CTC controller for dynamic system.
The adaptive control law is designed as:
1 1ˆˆ , , ( )T T
s
a Γ D q q v M q B Ps (4.49)
Where Γ is r × r) di go l ositive - definite constant matrix, and
P is a (2n ×2n) symmetric positive - definite constant matrix satisfying
T
A P PA Q , (4.50)
Q is a symmetric positive - definite constant matrix.
21
Fig 4.6 Structure of system
4.3. SIMULATION
4.3.1 Simulation with optimal controller
Mục tiêu di chuyển trong không gian theo quỹ đ o h s u:
xu t phát t i vị trí (x, y, z) = (2, 4, 0) so với gốc tọ độ s u đó tọ độ
tọ độ z = 0.1t. Còn tọ độ x y thì th y đổi theo vận tốc thẳng và góc
h s u: t = 0-5s: v=0.5m/s. ω = 0 rad/s. t = 5-10s: v = 0.5 +
0.15*sin((t-)*pi/10); ω=0.15*sin((t-5)*pi/10); t = 10-15s: v=0.75m/s.
ω = 0 rad/s. t = 15-20s: v = 0.75 + 0.15*sin((t-15)*pi/10); ω =-
0.15*sin((t-15)*pi/10). t = 20-25s: v=0.75m/s. ω = -0.15 rad/s. t = 25-
30s: v=0.75m/s. ω =-0.15 - 0.15*sin((t-25)*pi/10); t >30s v=0.5m/s. ω
= -0.3 rad/s;
Fig 4.8 Tracking target of stereo camera.
Fig 4.9 Tracking target of mobile robot
22
Fig 4.10 Error e between desired and measured angular velocity
4.3.2 Simulation with Adaptive CTC controller
The target moves following an arc O(5.5m, 0.5m), r = 4m. in
20s.
Fig 4.11 Tracking target of mobile robot
Fig 4.12 Error e between desired and measured angular velocity
4.4. Conclusion chapter 4
In this chapter the kinematic and dynamic models of the integrated
system are constructed. In addition, the authors propose a quadratic
performance of optimal control for the system. The pseudo-inverse
matrix of Jp is reversible at -π/ circle of the target movement, but it
still causes of shock at about 35 seconds (Fig 4.9). However, we will
research about inverse properties of it in others paper. The research
results are presented in document [3] & [4] of the list of
23
published works.
5. CONCLUSION
5.1. The main research content of the thesis
The thesis has implemented the following main contents:
- Analyze control methods for pan-tilt robot to develop new control
methods. The thesis has proposed 1 kinetic control method and 01
dynamic controller using artificial neural network for pan-tilt system
using 2 cameras to track a moving target. The control methods are
well proven by Lyapunov stability the

Các file đính kèm theo tài liệu này:

- tom_tat_luan_an_a_research_into_the_control_of_mobile_robot.pdf