State Space Constrained Iterative Learning Control for 3DOF Robotic Manipulator

In this paper, the trajectory tracking problem of a nonlinear robotic system with 3DOFs under the control signal obtained through nonlinearly constrained state spaceIterative Learning Control (ILC) methods is considered.The focus of this paper is the analysis of different control system parameters on the convergence rate of two constrained state space ILCalgorithms:Bounded Error Algorithm (BEAILC) and Constrained Output algorithm (COILC), as well as the comparison between these two algorithmsthrough simulations. The obtained results have shown that COILC algorithm converges faster than BEAILC algorithm when compared with the same learning and feedback parameters, due to lower trajectory restrictions. Also, it has been shown that an increase in feedback gains can decrease the number of iteration terminations during the learning process, thus allowing for more of the trajectory error information to be learned from during the single iteration. Moreover, simulations have shown that the decrease in learning parameter values will increase the number of iterations required to obtain the desired tracking accuracy.


INTRODUCTION
In recent years, there has been increased interest in the use of various advanced control techniques for engineering and scientific applications. Recently, iterative learning control (ILC) has attracted researchers and scientists' attention as one of the promising fields in intelligent control, [1,2]. In 1984, Arimoto et al. [3] proposed ILC as one of theadvanced learning control strategies for the control of robot manipulators. Namely, some mechatronic systems, such as production machines or industrial robots, often perform the same task repeatedly.
Iterative learning control is an intelligent control method which, through repetition in every iteration, using the information from previous trials, improves the control signal for the next trial in order to decrease (eliminate) the tracking error of a repetitivetask.
This method can be compared to a human skill practice. Rehearsing the movement and observing the error over time the error decreases as the number of repetitions increases.
In industrial applications robotic manipulators are programmed to execute a certain task that is repeated numerous times during its life cycle, making the ILC viable control method to consider, [4][5][6].
The key purpose of using ILC is to achieve high performance after a few iterations. One advantage of ILC is that there is no requirement for the dynamic model of the controlled system. A typical ILC in the time domain presents a simple off-line feedforward learning control. In terms of how to use the tracking error signal of the previous iteration to form the control signal of the current iteration, the structures of ILC appeared as D-type, P-type, PD-type, and PID-type, [2].
However, there are several critical requirements that limit the applications of ILC, especially to complex nonlinear (robotic) systems. Despite the widespread use of ILC in robot motion control, few attempts have been made to create an integrated design, [7,8].
In reality industrial robots have space boundaries, velocity limits and other saturations which make them constrained state space systems. Thus, all operations performed by these manipulators are contained within the constrained state space, setting particular trajectory planning and tracking requirements near the state space boundaries [8].
Another problem of the standard ILC procedure is the transient error growth. It is possible that in the first several iterations the tracking error grows significantly before it starts to converge to zero [9].
In this paper, constrained state space Iterative Learning Control methods for a nonlinear robotic manipulator with 3 DOFs are suggested. Especially, two constrained state space ILC algorithms are applied: Bounded Error Algorithm (BEAILC) and Constrained Output algorithm (COILC), as well as the comparison between these two algorithms through simulations are done.

CONSTRAINED STATE SPACE ILC
The robotic system can be represented with the following equation (obtained from Lagrange's Equations of Motion of second kind) [10]]: As a consequence, violation of constraints can cause the failure of the ILC trial (as it will be interrupted prematurely) and could potentially cause damage to the robot or its surroundings. Such violation can occur if the robot is following the desired trajectory that is near to the state space limits (Figure 1a). Research has shown that despite the proven convergence of the standard nonlinear ILC method, during the first few iterations the error grows (sometimes significantly) before it starts to converge to zero potentially violating the operative space boundaries. If the output trajectory is constrained in such a way that generalized coordinates boundaries aren't violated, then it's possible to apply the ILC method to the manipulator. As solutions to this transient error growth problem, two of the following algorithms are proposed: • Bounded Error Algorithm (BEA) • Constrained Output Algorithm (CO) The idea behind both algorithms is that the tracking process is terminated if the generalized coordinates preset boundaries are violated. Both algorithms break the equal trial time duration postulate during the premature termination of the tracking process, but their convergence is proven [9].

Figure 1b. Block diagram of BEAILC and COILC algorithms
The sufficient condition for both algorithms' convergence, taking into account the corresponding feedback and feedforward update laws (Figure 1b), is: For a high convergence rate, the learning operator should be as close as possible to the inertia matrix. As the real inertia matrix is unknown, it's preferable that the learning operator is chosen as estimated inertia matrix , as it is proven in the previous research [12].

Bounded Error Algorithm
Bounded Error Algorithm solves the previously mentioned problem by tracking the error norm during the iteration. Iteration is terminated momentarily as the error norm reaches a predetermined limit.A control signal is corrected before the next iteration only with information collected before the interruption of the previous iteration.
In combination with the feedforward control signal generated with BEA ILC algorithm, the following feedback term will be used: where ( ) A q is the estimated inertia matrix, and K v and K p are feedback gains. Planned desired trajectory inside the generalize coordinates constrains satisfies the inequality: where is the accuracy of the ILC method. Accordingly, δ can be selected as: , then the restriction can be applied to the output trajectory at every iteration by applying BEA algorithm, represented through the following steps [13]: 1. Set the initial iteration number k = 0 and begin theiterative procedure 2. Starting from the initial position (0) , then the tracking process is stopped and T k is set to the stop time of iteration . 3. At the end of the current iteration the learning controller updates the input control signals for the next iteration u k+1 according to the following learning update law: where L p and L v are learning gains. 4. If the overall output error is less than or equal to an acceptable tracking accuracy and T k equals T, thenthe learning procedure terminates successfully andthe optimal feedforward control signal is u k . Otherwise, set k = k + 1 and go to step 2. As a result of applying the BEA algorithm, the output trajectory in each iteration lays inside the hypercylinder with a radius of epsilon around the desired trajectory, thus always staying inside of state space constrains ( Figure 2). This algorithm can speed up the rate of convergence when compared to the standard ILC, where the error accumulation increases the number of iterations required to eliminate said accumulation.
The downside of BEA algorithm is its over restriction in the areas where the trajectory is well far from its generalized coordinates boundaries, leading to higher number of iterations due to more frequent iteration interruptions.

Constrained Output Algorithm
CO algorithm directly limits the output trajectory maximum value. In the contrary of BEA which forces entire trajectory to stay inside of the hypercylinder with a radius of an epsilon, which is defined as the closest safe distance from the generalized coordinates boundaries, CO limits only the maximum output value allowing deviations from the desired trajectory in areas where it's safe to do so. CO algorithm can be described through the following steps [9], taking into account the feedback term (5): 1. Set the initial iteration number k = 0 and begin theiterative procedure 2. Starting from the initial position q k (0) = q d (0) the system is tracking the desired trajectory under the control u(q,t) = u k (t) + u fb (t) while , 1 , 2, . , .
then the tracking process is stopped and T k is set to the stop time of iteration k. 3. At the end of the current iteration the learning controller updates the input control signals for the next iteration u k+1 according to the following learning update law: 4. If the overall output error is less than or equal to an acceptable tracking accuracy and T k equals T, then the learning procedure terminatessuccessfully and the optimal feedforward control signal is u k . Otherwise, set k = k + 1 and go to step 2.
This algorithm should speed up the process of finding the desired control signal, due to its reduced number of iteration terminations caused by lower restrictions compared to BEA algorithm.

SIMULATION RESULTS
Trajectory tracking simulations of the 3DOF robotic system ( Figure.3) using BEAILC and COILC algorithms are conducted in MATLAB and Simulink environments. Simulink modelled robot differential equations are solved using the Runge-Kutta method (ODE4), where the simulation step was 0.00001.
Mass for each segment is chosen as: 0.15 kg, 0.5 kg, 0.35 kg; and the length of each segment is chosen correspondingly: 0.15 m, 0.5 m, 0.35 m.
The desired trajectories defined in the space of generalized coordinates for joints are: Initial conditions for generalized coordinates and their derivatives are: The learning operator is chosen as the estimated inertia matrix for both algorithms, which is obtained by varying segments' mass and length so that the sufficient condition for convergence is met: The rest of the control system parameters were manually set and chosen through trial and error.
Generalized coordinates boundaries and control system parameters for both algorithms were set so that the simulation results are comparable.
Hypercylinder radius for BEAILC algorithm is set as: Generalized coordinates limits for COILC algorithm are set as: The desired tracking accuracy that has to be obtained by both algorithms i max e µ < is: Simulation results will be shown through the individual joint trajectory tracking with the parameters obtained after the learning process, then the maximum tracking error through iterations will be shown, iteration duration time, as well as the trajectories during the learning process for each joint.

The first set of parameters -BEA and CO
As suggested in [ [13] and [14], parameters are chosen as following diagonal matrices: 1 20 * , 60 * 1 00 * , 20 * Applying the BEA algorithm with (16) parameters, the desired accuracy was obtained after 21 iterations, with maximum tracking errors:  Figures 6, 7 and 8 it can be seen that the iterations were interrupted by the violation of generalized coordinatesconstraints, while on Figure9 duration of iterations can be seen individually. As the iteration duration time increases, the control system successfully decreases the tracking error and incrementally obtains more learning information as the iteration interruptions occur later in the tracking process. Due to high restrictions enforced by hypercylinder, interruptions occur regularly, thus decelerating the learning process and increasing the number of iterations. In Figure 4 final trajectory tracking for the third joint can be seen.
In Figures 7 and 8 it can be seen that maximum tracking errors are limited to the value of ε = 0.3, which means that the algorithm is working properly. Besides that, it can be seen which of the joints caused the iteration interruption as well as that monotonic maximum error convergence can be interrupted as the iteration is interrupted, depending on the position of the joint in the moment of interruption. The trajectory tracking through iterations for the third joint is shown in Figure 10.
Applying the CO algorithm with (16) parameters, the desired accuracy was obtained after 13 iterations, with maximum tracking errors: In comparison with BEAILC with same controller parameters and previously mentioned comparable boundaries, it can be seen that CO algorithm requires less iterations to achieve desired tracking accuracy. Due to less restricting constrains, interruptions were less frequent, which resulted in faster convergence. On Figure 5 it can be seen that the final trajectory for the joint 3 almost overlaps the desired trajectory.
In Figure 11 enlarged constrain violation is shown for the third joint where q 3 = 1.1. Despite that the absolute value of maximum error was higher in other areas of trajectory, it didn't interrupt the iteration, proving that the algorithm is implemented correctly.

The second and third set of parameters -BEAILC
Control system parameters can be chosen in a way that the individual joints can be targeted with different gain values. Second parameter set:  Applying the BEA algorithm with (17) parameters, the desired accuracy was obtained after 25 iterations, with maximum tracking errors: In comparison with previous parameter set where the third joint caused most of the iteration terminations ( Figure 8), with this parameter set it can be seen that now the second joint caused most of the trial terminations (Figures 12, 13 and 14).
Applying the BEA algorithm with (18) parameters, the desired accuracy was obtained after 22 iterations, with maximum tracking errors: Comparing the iteration duration time of the first set and this current set 3 of parameters ( Figure.15), number of iteration increased due to lower learning gains. On the other hand, due to higher feedback gains the number of interruptions has decreased. The interruptions with this set of parameters were caused primarly by the third joint.

The second and third set of parameters -COILC
With (17) and (18) sets of parameters the COILC simulation results are following: Applying the CO algorithm with (17) parameters, the desired accuracy was obtained after 11 iterations, with maximum tracking errors: As in previous comparison, it can be seen that CO algorithm takes less iterations to obtain the desired tracking accuracy when compared to the BEA algorithm.
Applying the CO algorithm with (18) parameters, the desired accuracy was obtained after 17 iterations, with maximum tracking errors: Like in the case of BEAILC, convergence is slower due to lower learning gains compared to the first parameter set, but thanks to the lower restrictions and higher feedback gains with the third parameter set iterations weren't terminated (Figure18), which means that the output trajectory was inside the CO boundaries, Figure  19. Maximum error norms for joint 3 for both sets can be seen on Figures 16 and 17. More simulation results can be found in [15].

BEAILC convergence rate influence of radius
For convergence rate influence of radius demonstration the following parameter will be used: Values for ε are: From Figures 20a, 20b and 20c shown below, it can be seen that with radius value increase, BEA algorithm requires less iterations to obtain the desired tracking accuracy (convergence is faster). This effect is more noticeable for BEA than CO, due to higher trajectory restriction.

CONCLUSION
From previously shown simulation results, it can be observed that both algorithms successfully managed to decrease the tracking error under the desired accuracy, with the learning matrix chosen as the estimated inertia matrix so that the convergence condition is met.
It is confirmed that BEAILC algorithm takes more iterations to obtain desired tracking accuracy compared to COILC algorithm with the same parameter set, due to more limiting constrains. The hypercylinder around the desired trajectory enforced by BEA over-constrains the trajectory in the areas where it's physically safe for error to increase slightly in favour of obtaining more learning information from the current trial. On the other hand, COILC algorithm with its maximum and minimum limiting values, allows higher error values as long as the trajectory is inside its constrains, thus obtaining more information from the current iteration resulting in faster convergence. Loosening the hypercylinder radius ε dec-reases the number of iterations required to obtain the desired tracking accuracy (speeds up the convergence).