Discrete-time inverse optimal control for a reaction wheel pendulum: a passivity-based control approach

1. Introduction

Industrial processes are usually made up of nonlinear dynamic systems that must be controlled to properly carry out a specific process [1]. For this reason, the study of diverse linear and nonlinear control strategies in the prototype test systems is a very important step to validate their performance [2, 3]. Since this helps to understand the phenomena and the physical behavior of the nonlinear dynamic system, simplifications in practical applications are feasible [4]. A classical nonlinear dynamic system used to assess the capacity and robustness of the controllers is the pendulum, which has different variants, such as the reaction wheel pendulum, Pendubot, Acrobot, the pendulum on a cart with linear displacement, pendulum models with two and three bars and the Furuta pendulum with a rotating base [3, 5, 6, 7], among others. All of these prototype test systems can emulate the challenges of nonlinear dynamics such as mobile robots, walking robots, aircraft, rockets, and electrical machines (motors/generators), among others.

The reaction wheel pendulum (RWP) is analyzed in this paper, which was introduced by Spong in [6]. The RWP is varied in the classic inverted pendulum. It contains a bar that can spin without restrictions around the bracket point (pivot) at one of its ends, as illustrated in Fig. 1.

In general, the RWP has two main challenges. The first one is local, which maintains stability at the equilibrium point [9]. The second challenge consists of raising the pendulum from its rest position to the vertical position, this challenge is well-known as swing-up [10]. The swingup challenge is a solved problem where the energy- based strategies can take the pendulum closer to the equilibrium position without any problem [8, 11, 12].

Several linear and nonlinear controllers have been proposed to tackle the local problem. In the case of linear controllers, it is needed to use linearization methods (e.g., Taylor’s series or trigonometric approximations) that properly works nearly at the operating point [5, 13]. However, the performance and stability of these techniques are risky when the RWP moves away from the linearization point [14]. In the case of nonlinear controllers several approaches have been proposed, such as fuzzy logic [15], artificial neural networks [10], exact or partial feedback linearization [16], passivity-based control [14], Lyapunov-based control approach [8], and function sliding control [17], among others. However, some of these approaches may present some issues, such as not ensuring closed-loop stability, oscillations at the operation point, or performing an online optimization process [13]. Other approaches can be difficult to implement in practice and require many adjustment parameters. Although some of them guarantee the closed- loop system’s stability, their control law may not be optimal. Unlike these previous works, we propose a nonlinear optimal discrete control method based on the discrete representation of the system, which ensures the asymptotic stability of the RWP in closed-loop. In addition, this controller has the advantage that does not require to solve the associated Hamilton-Jacobi-Bellman equation, which typically appears in optimal control [2].

The main contributions of this research are summarized below:

• The application of the inverse optimal control to regulate state variables in a classical and well-known nonlinear system, i.e., the RWP mechanism, with the possibility of ensuring asymptotic stability, passivity and optimal design based on the minimization of the Lagrangian function of the system. This control works by making negative the feedback of the passive output that is defined as a combination of the quadratic Lyapunov function and the desired control input.

• The description step by step of the control design by presenting the demonstration of the passivity. stability and optimally properties, which will help readers in the comprehension of the inverse optimal control design for regulating state variables in nonlinear systems in a tutorial form.

• The comparison of the inverse control approach with classical and strong nonlinear techniques such as passivity-based and Lyapunov-based, which can demonstrate the superior performance of the proposed approach in terms of numerical convergence, i.e., time of stabilization.

It is important to mention that after a complete review of the significant literature about inverse optimal control applications, we do not find evidence of the implementation of this control strategy to regulate the position and velocity in the reaction wheel pendulum, which was identified as a gap in the current literature that this research intends to complete.

The remainder of this document is organized as follows: Section 2 presents the continuous and discrete modeling of the RWP system by offering its simplified version with two state variables as recommended in [8]. Section 3 describes in detail the inverse optimal control theory applied to discrete nonlinear systems by presenting three lemmas related to passivity, stability, and optimality, which guarantees the best control design. Section 4 offers the numerical validation of the proposed inverse optimal control design and its comparison with nonlinear approaches based on passivity and Lyapunov designs, demonstrating the superior performance of the proposed controller regarding convergence times, i.e., time of stabilization. Section 5 presents the main conclusions derived from this research as well as possible future works using inverse optimal control design for trajectory tracking.

2. Dynamical modeling of the reaction wheel pendulum

The RWP is a classical dynamical system used to validate linear and nonlinear control strategies since it contains strong non-linearities with trigonometric functions that make it comparable with second-order synchronous machine models, transportation systems, or bridge crane models, among others. In Fig. 1 it is presented a representation of the RWP system with its main physical variables

2.1. Continuous formulation

The RWP has a motor coupled to the opposite end of the pivot, acting on an inertia wheel in which the oscillations of the wheel are controlled due to the reaction torque τ. The pendulum angle 𝜑 (from the vertical axis) and the angle α between the pendulum and wheel are measured with sensors located at each of the axes of rotation. Defining θ = 𝜑 + α, the dynamical model of the RWP system can be written as follows:

where 𝑎, 𝑏 and c are constants related to the physical parameters of the system, 𝜑 represents the angular position of the pendulum measured from the vertical axis, and θ is the relative angle of the reaction wheel measured from the same vertical reference. For the implementation of the control design, since the behavior of the variable θ is defined as the double integral of the control input, this is not needed to be included in the state variable representation as recommended in [6]. Note that the angular position of the wheel can be completely determined by the integral of the control input as follows:

which implies that if the control input well-defined, then the speed position of the wheel will tend to zero when the angular position of the pendulum reach the equilibrium point.

To transform the set of equations (1) to a state-space representation, the following state variables will be used: x₁ = 𝜑 and x₂ = 𝑥̇₁. After substituting these into (1), the following second-order dynamical model is obtained:

It is important to mention that the objective of control in the RWP model is to regulate all the state variables, i.e., take all of them to zero from any initial condition.

2.2. Discretization of the dynamical model

To obtain a time discretization of the dynamical of the reaction wheel pendulum, it is used the backward differences in order to determine the next step (X_k+1 ) as function of the current information X_k of the dynamical system, i.e.,

where 𝑇_𝑠 is the discretization time. Now, if we apply the discretization defined in (3) on (2), then, we reach the following discrete model for the reaction wheel pendulum.

It is worthy to mention that the discrete model presented by (4) is not unique, since it is dependent on the approximation used in the time derivative function of the state variables; nevertheless, the backward approach is one of the most employed in literature due to its simplicity [18].

Note that in a compact representation the discrete model (4) can be rewritten as follows:

where 𝑓(𝑥𝑘) ∈ ℝ^𝑛 is a vector of nonlinear functions of the state variables and 𝑓(𝑥𝑘) ∈ ℝ^𝑛𝑥𝑚 is known as the input matrix. Note that in the case of the RWP model n =2 and m = 1. Note that these functions take the following form:

To develop a controller based on the discrete representation in the following section, it is considered the compact structure defined in (5) with nonlinear functions (vectors and matrices) presented in (6).

3. Inverse optimal control design

In this section three main aspects of the inverse optimal control design are explored for nonlinear discrete systems. For doing so, let us define the general structure of the system under analysis as follows.

Definition 1. A nonlinear dynamical system in the discrete domain with the form,

fulfills passivity properties, is globally asymptotically stable and there is a control law with the form 𝑢_𝑘 = − 𝑦_𝑘, such that a functional cost function is minimized, i.e., uk is an optimal control law. Note that in (7), y_k is the output of the system and h(x_k) and j(x_k) take the followings structures:

being 𝑄 a symmetry positive definite matrix, i.e., 𝑄 = 𝑄T> 0.

To demonstrate each one of the properties presented in Definition 1, let us consider a candidate Lyapunov function with a quadratic form as follow.

which is positive definite for all x_k ≠ 0 and zero only for x_k= 0. In addition, let us define a general form for the control input u_k as follow

where w_k is the new input and β (x_k) can be defined as presented bellow

being I an identity matrix with appropriate dimensions.

Definition 2. The dynamical system (7) exhibits passivity properties if there is a matrix Q such that the following inequality is held.

3.1. Passivit

To demonstrate passivity properties in the the dynamical discrete system consider Lemma 1 as presented below.

Lemma 1. The dynamical system in (7) is a feedback passive system for the output 𝑦̃_𝑘 The control input is defined as (10), where 𝑦̃_𝑘 takes the following form

where

Proof. To proof the feedback passivity properties of the dynamical system (7), consider the variation of the Lyapunov function for the current and the future states as follows

Note that using (7) and (10) in (15), we have

From (16), we can observe that:

Now, if we consider Definition 2 and expressions in (17) to be replaced in (16), then, we have

which confirms that the discrete system is passive from the output 𝑦̃_𝑘 to the new input w_k and the proof about passivity is completed.

3.2. Stability

To demonstrate passivity properties the stability properties in the sense of Lyapunov for closed-loop operation, let us consider the following Lemma.

Lemma 2. The system (7) is asymptotically stable in the sense of Lyapunov with the control input ((10)) if wk is defined as

Proof. To proof stability in the sense of Lyapunov, we can transform the dynamical system (7) with the control input (10) as an equivalent system with the following structure

Now, if we consider the difference between the current and the next step of the Lyapunov function defined in (15), we have

From (21), we can note that:

in addition, from Lemma 2, we know that 𝑤_𝑘= −𝑦̃_𝑘 which implies that in conjunction with (22), the expression (21) takes the following form

which allows to conclude that the system (7) is globally asymptotically stable in 𝑥𝑘= 0 since the candidate Lyapunov function is radially unbounded. This completes the proof

3.3. Optimality

Lemma 3. The inverse control law (10) is considered optimal since it stabilizes the dynamical system as presented in Subsection 3.2, and it minimizes the following functional cost

where 𝐿(𝑥_𝑘, β(𝑥_𝑘)) is the LaGrangian function of the system that can be written as

being 𝑙(𝑥_𝑘) defined as

Note that the optimal solution for the functional cost is ℱ^⋆ = 𝒱(𝑥₀), being it x0 the initial condition for the dynamical system (7).

Proof. To demonstrate the control law 𝛽(𝑥_𝑘), that is an optimal function, let us consider the Hamiltonian of the system as

which has the global minimum as

To minimize this Hamiltonian function, we can rewrite (26) considering (25) as follows

The solution of the minimization function (28) considering (26) and the variation of the candidate Lyapunov function (15) as presented in [2], it is taken the following form

in addition, if we consider (7b) and (8), then, we can simplify (30) as presented below

It is important to mention that the solution of (30) for 𝛽(𝑥_𝑘) takes the following structure

which confirms control function initially defined in (11) as an optimal control law since it minimizes the functional cost (24).

In order to determine the optimal value for the LaGrangian function (24), let us consider that the interval of analysis [0, N], being N a natural number with the following result

In the case of the optimal control law β(𝑥_𝑘),this is optimal if it makes zero the Hamiltonian function ℋ(𝑥_𝑘, β(𝑥_𝑘)) demonstrated in [2]; in addition, we know based on the stability properties of the inverse optimal control that when N → ∞ the Lyapunov function 𝒱(𝑥_𝑁) → 0for any initial condition x0, which implies that 𝒱(𝑥₀).

3.4. General commentaries

In the application of the studied inverse optimal control it is worthy to mention that:

• To stabilize a nonlinear discrete dynamical system with the form defined in (7) it is used the optimal control law (𝑢_𝑘=β(𝑥_𝑘)) guaranteeing passivity, stability and optimallity properties.

• The application of the inverse optimal control design is subject to the fact that the dynamical system be zero detectable, which can be expressed mathematically as presented in Definition 3.

Definition 3. A system (7) is locally zero-sate observable (locally zero-state detectable) if there is a neighborhood 𝒵 of 𝑥_𝑘 = 0 ∈ ℝ^𝑛 such that for all 𝑥₀ ∈ 𝒵

where 𝜙(𝑘, 𝑥_0,0) = 𝑓̃(𝑥_𝑘 ) is the trajectory of the unforced dynamics 𝑥_𝑘+1 = 𝑓(𝑥_𝑘) with initial condition x₀. If Z=ℝ^n, the system is zero-state observable (respectively zero-state detectable).

4. Numerical validation

In this section the numerical validation of the inverse optimal control design is presented for a reaction wheel pendulum defined in (6) with constants as reported in [8]. In these simulations, we consider the following cases: i) the evaluation of the controller for multiple control gains, i.e., values in the Q matrix; and ii) the comparison of the inverse control design with a Lyapunov-based control design reported in [8] and the passivity-based control design reported in [14]. It is important to mention that as recommended in [5], the magnitude of the control function, i.e., |𝑢_𝑘| can be at most 10.

Note that the resulting control law 𝑢_𝑘 = β(𝑥_𝑘) for the reaction wheel pendulum presented in (6) by using the definition (11), takes the following structure

4.1. Simulation for different values of Q

In this simulation case, we select the components of the the matrix Q that appear in the control law (33) as follows 5 × 10⁶ ≤ q₂₁ ≤ 40 × 10⁶ by fixing q₂₂ as 12 × 10⁵. Figure 2 presents the physical performance of the reaction wheel pendulum regarding the state variables x₁ and x₂ (angular position and speed of the pendulum bar) and the control input u when different values of the gains in the matrix Q are evaluated.

From Fig. 2 the following facts can be extracted:

• The value of the gain q₂₂ in the control input (33) determines the time required to regulate the angular position of the pendulum; nevertheless, the lowest time to stabilize the system is about 400 samples, i.e., 400 ms, as can be seen in Fig. 2(a).

• Values for the gain q₂₂ lower than 30 × 10⁶ produce responses on the angular position of the RWP system such as a first-order dynamical system (see Fig. 2(a)), while values greater than these produce responses similar to second-order dynamical systems. Note that the previous numerical performance was reached when q₂₂ has been fixed as 12 × 10⁵.

• The behavior of the angular speed in Fig. 2(b) is governed by the control input presented in 2(c) since the control input is saturated to its bounds, and the speed rapidly decreases. At the same time, u_k is negative and quickly increases when u_k becomes positive. In addition, the convergence of the angular speed to the origin (variable regulation) takes at least 500 ms in the best scenario, i.e., the best combination of gains q₂₁ and q₂₂.

• The saturation of the control input presented in 2(c) is implemented as recommended in [9] to avoid unreachable solutions in real RWP systems since this control represents the torque applied to the reaction wheel by a direct-current motor, which can be understood as the current observed by the motor, that is small in these applications [6].

4.2. Comparison with nonlinear controllers

Here, the proposed inverse optimal control is compared with a nonlinear controller based on a direct Lyapunov control proposed in [8], the structure of this control law is presented below

being k₁ and k₂ defined as 3500 and 135, respectively. In addition, the proposed inverse optimal control is also compared with a nonlinear passivity-based controller proposed in [14], which has the following control law

being 𝑗₁ = −1, α = 3500 and 𝑟₂ =135, which are selected to make it comparative with the Lyapunov-based design.

It is important to mention that all the three controllers defined in (33), (34) and (35) are based on Lyapunov stability theory, which implies that all of them have global asymptotic stability properties for the closed-loop operation. In addition, it is possible to observe that all of them have a very similar control law, which is composed of linear feedback of the states x_1k and x_2k and the nonlinear effect of the sinusoidal function weighted by a constant [19].

In Fig. 3 it is presented the comparison between the proposed inverse optimal controller and the Lyapunov- based design and the passivity-based approach reported in [8] and [14], respectively.

From results in Fig. 3 we can observe that the Lyapunovbased and the passivity-based approaches have the same numerical performances since the angular position are overlapping for both controllers. In addition, these controllers take about 470 ms to establish around the reference. In comparison, the proposed inverse optimal control approach reaches the reference signal in about 370 ms, which demonstrates its superiority in performance. It is worthy to mention that the comparative approaches present an overpass to the reference signal. This implies that some oscillations in the vertical position are experienced. At the same time, the proposed method does not present this behavior, which confirms its efficiency in contrast to powerful and well-known nonlinear approaches.

5. Conclusions and future works

A nonlinear discrete control method based on the inverse optimal design was presented in this paper to solve the problem of variable regulation in nonlinear physical systems by using a reaction wheel pendulum as an example of application. The studied control design has three main advantages, such as passivity, asymptotic stability in the sense of Lyapunov, and optimality. This implies that the stable behavior of all the state variables is ensured during closed-loop operation.

Regarding nonlinear control approaches reported in specialized literature for regulating state variables in the RWP system, the inverse optimal control method demonstrated superior numerical performance in comparison to Lyapunovbased and passivity-based control reports, since the proposed controller stabilized the system in about 360 ms. Conversely, the comparative approaches make it in 470 ms, i.e., 110 samples before. In addition, the proposed approach presents a behavior similar to a first-order dynamical system without overpasses when control gains q₂₁ and q₂₂ are correctly selected, while the passivity-based and the Lyapunov based work as second-order systems by presenting small oscillations around the reference signals.

As future works, it will be possible to have the following researches:

• To apply the inverse optimal control to tracking trajectory problems such as voltage regulation in power electronic converters or motion control in robots.

• The application of the inverse optimal control design to reduce sub-synchronous oscillation in single- and multimachine power systems.

• Applied optimization methods to find the best control gains, i.e., components of the Q matrix to minimize quality indicators such as mean square error or integral square error, among others.

References

Additional information

How to cite: O. D. Montoya, W. Gil-González, F. M. Serra, “Discrete-time inverse optimal control for a reaction wheel pendulum: a passivity-based control approach,” Rev. UIS Ing., vol. 19, no. 4, pp. 123-132, 2020, doi: https://doi.org/10.18273/revuin.v19n4-2020011

Revista UIS Ingenierías
ISSN: 1657-4583
Vol. 19
Num. 4
Año. 2020

Discrete-time inverse optimal control for a reaction wheel pendulum: a passivity-based control approach

Oscar DaniloWalterFederico MontoyaGil-GonzálezMartin Serra
Universidad Distrital Francisco José de CaldasUniversidad Tecnológica de BolívarUniversidad Nacional de San Luis,ColombiaColombiaArgentina