---

One of the challenges encountered in the application of classical reinforcement learning methods to real-control problems is the curse of dimensiality. In order to overcome this difficulty, hybrid algorithms that combine reinforcement learning with various function approximators have attracted many research interests. In this paper, a novel Neural Reinforcement Learning (NRL) scheme which is based on Sarsa learning and Radial Basis Function (RBF) network is proposed. The RBF network is used to approximate the Action Value Function (AVF) on-line. The inputs of RBF network are state-action pairs of system and its outputs are corresponding approximated AVF. As the necessary condition for the convergence of NSL to the optimal task performance, the existence of stationary points for NSL which coincide with the fixed points of Approximate Action Value Iteration (AAVI) are proved. The validity of the proposed algorithm is tested through simulation examples: mountain car control task, and acrobot problem. Overall results demonstrate that our algorithm can effectively improve convergence speed and the efficiency of experience exploitation.

---

In this paper, a two point guidance law for homing interceptors using finite time second order sliding mode control and based on parallel navigation is proposed. In the proposed guidance law, sliding surface is selected as the line of sight rate and the target maneuvers are considered as an uncertainty which only needs the upper bound of these maneuvers. Furthermore, the proposed algorithm can guarantee the finite time convergence of the LOS rate to zero or a small neighborhood of zero. Therefore, the performance and stability of guidance loop against maneuvering targets are increased.

---

Applying supervised learning in robot navigation encounters serious challenges such as inconsistence and noisy data, difficulty to gathering training data, and high error in training data. Reinforcement Learning (RL) capabilities such as lack of need to training data, training using only a scalar evaluation of efficiency and high degree of exploration have encourage researcher to use it in robot navigation problem. However, RL algorithms are time consuming also have high failure rate in the training phase. Here, a novel idea for utilizing advantages of both above supervised and reinforcement learning algorithms is proposed. A zero order Takagi-Sugeno (T-S) fuzzy controller with some candidate actions for each rule is considered as robot controller. The aim of training is to find appropriate action for each rule. This structure is compatible with Fuzzy Sarsa Learning (FSL) which is used as a continuous RL algorithm. In the first step, the robot is moved in the environment by a supervisor and the training data is gathered. As a hard tuning, the training data is used for initializing the value of each candidate action in the fuzzy rules. Afterwards, FSL fine-tunes the parameters of conclusion parts of the fuzzy controller online. The simulation results in KiKS simulator show that the proposed approach significantly improves the learning time, the number of failures, and the quality of the robot motion.

---

In this paper, we present a novel continuous reinforcement learning approach. The proposed approach, called "Fuzzy Least Squares Policy Iteration (FLSPI)", is obtained from combination of "Least Squares Policy Iteration (LSPI)" and a zero order Takagi Sugeno fuzzy system. We define state-action basis function based on fuzzy system so that LSPI conditions are satisfied. It is proven that there is an error bound for difference of the exact state-action value function and approximated state-action value function obtained by FLSPI. Simulation results show that learning speed and operation quality for FLSPI are higher than two previous critic-only fuzzy reinforcement learning approaches i.e. fuzzy Q-learning and fuzzy Sarsa learning. Another advantage of this approach is needlessness to learning rate determination.

---

Visual servoing system is a system to control a robot by visual feedback so that robot drives from any arbitrary start position to the target positions. Various ways, including control by using model of the robot, designing controller directly, and using Jacobian matrix have been studied. Since there is not access to model of robot and obtaining a model of robot would be difficult and time consuming, in many cases, the control law is obtained using Jacobian matrix. In this paper, inverse of Jacobian matrix is approximated using artificial neural networks. The approximated neural models are used in control law directly. For each degree of freedom of the robot manipulator, a two-layer feedforward neural network is considered. The distance between end-effector and target along the x-axis and y-axis, and the shoulder joint coordinates along the x-axis and y-axis are the inputs of each of the networks and the outputs are the fraction of the related robot joint changes to the image features changes (the elements of the inverse of Jacobian matrix). The proposed method has been implemented on a real robot manipulator. The experimental results show that the proposed control system can move the end-effector to different target positions in workspace with good accuracy.

---

In this paper, a nonlinear and robust guidance system against target maneuvers has been designed. For this purpose, first a new high order sliding mode algorithm is proposed. The designed guidance law with this algorithm generates a smooth acceleration command that guarantees collision with target. In this algorithm, unlike previous high order sliding mode theories, the stability of close loop system in the presence of uncertainty is guaranteed, therefore the observer is not required for estimation of target maneuvers in the proposed guidance law. For designing two point guidance law using this algorithm, a sliding variable has been introduced using relative lateral velocity. Designed guidance law generates acceleration commands that guarantee convergence of sliding variable. Simulation results show the better performance of proposed guidance law in comparison with other guidance laws.

---

This paper proposes a framework of Supervised Deep Learning (SDL) for wheeled robot navigation in soft terrains with a focus on wall following and obstacle avoidance tasks. Here, it is supposed the robot is only equipped with a vision system (Kinect camera). The main challenge while using depth images is high dimensionality of images and extracting proper features of them with a purpose of reducing input dimensionality of controller. To do this, the deep learning is utilized in this paper and the appropriate features which are the representation of depth images are acquired. Four architectures are created using this features and the history of steering commands. These architectures are compared in WEBOT simulator. The experiments show that the proposed architecture with four groups of features including: the represented features of depth data, previous represented features, the position of trajectory in color image, and the history of previous steering commands can control the robot in soft terrain with a variety of obstacles as well.  

---

Vision-based robot control is a method to motion control of a robot using information extracted from visual sensors. In traditional approaches, a model of robot and camera are needed. Obtaining these models are time consuming and sometimes impossible. Recently, intelligent methods are used to cope the above challenges. In this paper, a hybrid fuzzy controller is proposed to control a robot manipulator. Visual inputs of the controller are provided by Kinect and outputs are the rotation of joints motors. The hybrid controller contains two controllers. The first controller in based on fuzzy inverse model which approximates real inverse model of robot using gathered data. In order to increase accuracy, a fuzzy expert controller is designed and it is used when the end-effector is in the predefined near-goal area. Since determining exact value of the fuzzy expert controller parameters is impossible, in addition to make system adaptive with small changes in the environment, actor-critic architecture is used. This architecture is a well known continuous reinforcement learning methods. The proposed method is applied to control a real robot manipulator (ARM_6AX18). Experimental results show that using the proposed method in practice, the end-effector reaches from any random start position to the goal position with a good accuracy in robot workspace.

---

---

The design of integrated guidance and control system for flying objects is one of the research fields in the aerospace that is considered by researchers in recent years. Due to the nonlinearity of the kinematic and dynamic equations of the homing interceptors in the terminal phase and also existence of uncertainties such as target maneuvers, external disturbances and variations in aerodynamic coefficients, siding mode control theory is a suitable method for designing integrated guidance and control system. One of the most problem of sliding mode is the presence of high frequency oscillations in the control signal that is called chattering, which makes it impossible to implement this controller. A method for smoothing the control signal in a sliding mode is to use a disturbance observer. In this paper, the control signal is completely smooth, with having the disturbances estimation and using a different scheme compared with standard sliding mode. Also, the finite time convergence is guaranteed in the presence of uncertainty. The proposed guidance and control system are evaluated in computer simulation.

---

Designing the estimator that can determine the attitude with a single sensor is vital due to the limited weight and volume in the nano-satellite, the problems caused by the limited lifetime of the mechanical gyroscope in the long term and the eclipse phenomenon. To compensate for data deficiency, a two-nested filter has been utilized in this paper. To this end, the attitude in the second filter is estimated using the sensor data and the magnetic field derivative estimation from the first filter by the extended Kalman filter. Two stochastic algorithms named as multiplicative extended Kalman filter and square-root unscented quaternion estimator are compared with the proposed symmetry-preserving nonlinear observer in order to obtain an appropriate accuracy for determining the attitude of the nano-satellite, which has only a three-axis magnetometer. The proposed method is based on invariant observers under the action of the Lie group. The moving frame approach has been used so that the observer's parameters can be adjusted through the invariant error dynamic equations. Simulation results confirm an acceptable accuracy in all three algorithms for both time and frequency response analyses. However, the root mean square error of the attitude error with a nonlinear observer is much less than the stochastic algorithm in case of a larger initial estimation error. Furthermore, this approach guarantees convergence by the Lyapunov stability proof owing to setting the parameters with periodic differential Riccati equations.

---

---

---

---

---

---