 |
 |
|
Example of robot manipulator equipped with tactile sensor |
Estimation of object pose using partifle filter |
Visual information, such as depth sensors and stereo cameras, is the main sensor information for robot manipulation of objects. However, for flexible, transparent, and translucent objects, it is difficult to grasp the position, posture, and shape of the object by visual information alone. To solve this problem, we propose an approach to estimate the position, posture, and shape of an object based on contact information with the object using tactile sensors attached to the robot's fingers. The object position and posture are represented by a probability distribution, and the distribution is estimated by relying on the tactile sensor information. We will develop a method for manipulating objects that cannot be accurately identified visually, including the acquisition of "probing motions" that are effective for identifying the position and posture of objects.
|
|
|
Example of robot manipulator equipped with tactile sensor |
Estimation of object pose using partifle filter |
Reference:
In production, it is necessary for people to teach the robot's movements. The most common method of teaching the robot's trajectory is to use a teaching pendant, but for tasks that involve contact and constraint with the environment (such as the door-opening motion shown on the left in the figure below), it takes a lot of effort and time to teach the robot the appropriate motion. We aim to improve the efficiency of this process by allowing the robot to perform trial and error using contact information. In the right side of the figure below, the robot starts trial-and-error from a roughly given path, and when it comes into contact with the environment, it uses the contact force information to modify the path sequentially. In addition, we have devised a method to adjust the meta-parameters of such path generation.
 |
 |
|
Example of contrained task by mobile manipulator |
Succesive modification of path based on contact information by a manipulator |
Reference:
Human adaptability includes the ability to adaptively recover function by reusing previously acquired neural circuits when partial dysfunction occurs in the body or brain. There is a need to understand this adaptive process. The purpose of this study is to develop a motor learning model that can reuse partially acquired physical relationships in the controller by introducing a mechanism called "mapping transformation estimation" into the motor learning model that estimates dependencies between different senses. As a specific task, we set up motor learning for both left and right arms, and have the robot acquire controllers for the left and right arms separately. By discovering the symmetry of the left and right arms through the mechanism of "mapping transformation estimation," the partial dynamics of the right arm can be used for motion generation when a part of the control circuit of the left arm malfunctions.
 |
 |
|
Estimation of partial physical relationship in motor learning |
Reuse of physical relationship between two controllers for left and right arms |
Reference:
Conventional motor learning methods are based on fixing state variables and control inputs, and identifying their input-output relationships in a black-box manner. In contrast, in this research, we consider a system that can perform redundant and multimodal sensing, and consider a method for motor control that includes the process of identifying dependencies between different sensors and estimating "which sensor information can be used to achieve the desired control. As a concrete example, we consider a musculoskeletal arm-arm system with artificial muscles. We estimate the dependency between various sensor information using mutual information, trace the dependency, and check the controllability of the other variable by controlling one of them. A control law is automatically generated by accumulating this process. It was confirmed that a common methodology can be used to achieve different objectives, such as tracking control and force control, and that a motor learning method that can adapt to partial failures and changes can be constructed.
 |
 |
|
Generation of dependency network of multimodal sensor information (movie) |
Process of automatic generation of controller (movie) |
 |
 |
| Motion learning of musculoskeletal arm in reaching task | Motion learning of musculoskeletal arm in force controlled task |
Reference:
Navigation of mobile robots in crowded environments such as train stations and stadiums requires consideration of interaction with people, consideration of the flow of people, and avoidance of not only collision avoidance but also psychological pressure on people. In this study, we developed, implemented, and verified a navigation method for driving in a crowded environment without interfering with the flow of people based on the information collected in the simulation. In this study, we developed, implemented, and verified a navigation method for driving in a crowded environment without interfering with the flow of people, based on the information collected in the simulation. A common problem in research on navigation in such a dense crowded environment is that it has not been compared in a common environment. In this study, we propose a test method to compare and evaluate the navigation performance of robots under the same conditions while controlling the density of human flow, and show the video of the experiment.(Example of navigation settings）
 |
 |
| 高密度雑踏環境下での移動ロボットのナビゲーション | 再現可能な高密度人流環境でのナビゲーション実験方法 |
Reference:
In order to realize a robot that can collaborate with humans in the same environment, it must be able to cope with anomalies (failures) and task variations. In particular, it is desirable for the robot to be able to communicate with its collaborators in language, to recognize failures more accurately, and to reflect their requests in its behavior. In this study, we develop a robot system that can respond to a person's request to correct its behavior and point out abnormalities (failures) in its work using natural language. Using motion data collected from real-world data and natural language data in which humans describe the situation, we develop a method to identify the execution state and motion status of a task and modify the motion. By doing so, we aim to realize robot intelligence that is more compatible with humans in a data-driven manner that reduces the design burden on designers (experts).
 |
 |
| Recognition of task states based on sentence and motion information | Example of task execution scenario by robot manipulator with a gripper |
Reference:
In this study, we propose a flexible environment recognition method based on feature extraction from image information based on information obtained by the robot's own motion. For the navigation problem of an autonomous mobile robot equipped with a camera and a laser range finder (LRF), we propose a method for recognizing obstacles on the road, which are difficult to distinguish by 3D shape recognition based on distance measurement, based on the correlation between image features and driving experience. We propose a method for recognizing obstacles on the road, which are difficult to identify by distance measurement-based 3D shape recognition, based on the extraction of correlations between image features and running experience. This method enables us to realize a robot that can move intelligently while identifying and judging whether it is easy to run and how to run.
 |
  |
| Uneven rough terrain in outdoor environment | Example of mobile robot and visual information processing |
Reference:
We propose a method of navigation in an unknown (unmapped) uneven terrain environment by making intelligent decisions based only on the information of the target point and nearby observations. the system determines whether a road is a runway or not based on the information from the laser rangefinder (LRF), and generates a route based on the results of the determination. When the vehicle reaches a fork, it decides which path to take based on its past experience of optimal control, and proposes a smarter path selection method than simply choosing the one that is closer to the target point. The proposed control method is verified using a simulator and an actual vehicle, which are constructed based on an actual rough terrain environment.
 |
  |
| UGV navigation in rough terrain outdoor environment | Recognition of environment and navigation in simulation |
Reference:
視覚入力の中から，「関係のあるもの」と「関係のないもの」を識別する過程を自律的に獲得できるロボットの学習法を提案する．ロボット身体や背景・物体に関する視覚的特徴や空間中の位置関係などに関する事前知識を用いずに，特徴間の位置変化や遮蔽による消失などの情報だけをもとに離散的な状態識別を行う．抽出を行った身体及び物体の特徴クラスタから，関節角度に対する条件付き確率を用いて位置予測を行うことで，リーチングや回避などの行動生成につなげられる．
 |
  |
| 確率的な依存関係を用いた状態識別の例 | 視覚特徴間の依存関係を表すネットワークの生成例 |
Reference:
This research proposes a manipulation acquisition framework based on off-line trials with failure/success. Two feature spaces are constructed using nonlinear mapping, feature space of force sensor information and feature space of configuration space. Thay are used to modify holding-up motion online. First a robot tries to hold up an object and verifies its force sensor information. The robot predicts whether tha task can be achieved using the sensor information. When it is predicted that the robot will fail to hold up, it modifies the configuration using the feature space information. The proposed framework was evaluated by simulations with a humanoid robot.
 Idea of modification of holding-up configurationusing force sensor information |
 Evaluation of holding-up motion by simulation with a humanoid robot |
Publication:
To improve the flexibility of robotic learning, it is important to realize an ability to generate a hierarchical structure. This paper proposes a learning framework which can dynamically change the planning space depending on the structure of tasks. Synchronous motion information is utilized to generate ’modes’ and hierarchical structure of the controller is constructed based on the modes. This enables efficient planning and control in low-dimensional planning space, though the dimension of the total state space is in general very high. Three types of object manipulation tasks are tested as applications, where an object is found and used as a tool (or as a part of the body) to extend the ability of the robot. The proposed framework is expected to be a basic learning model to account for body schema acquisition including tool affordances.
 Idea of motion generation based on separation of path planning of object and body |
 Total scheme of motion generation with multiple mode transitions which consist of modes with body motion and with body and object motions |
 Pushing motion generation considering multiple mode transitions |
 Object manipulation with an L-shaped tool, obtained by the proposed motion generation framework |
Reference:
This research proposes the learning of whole arm manipulation with a two-link manipulator. Our proposal combines a controller obtained by reinforcement learning (actor-critic) and a learning classifier realized by a Support Vector Machine (SVM). The classifier learns the boundary between slip and stick modes in torque space. Using the result of classification, the robot learns to move the object toward desired position while keeping the desired contact modes. Control input (torque) is first specified by the actor. The SVM classifier judges whether torque can maintain the desired slip or stick mode and, if not, it modifies the torque so that the desired mode is maintained. It was verified in the simulation that our proposed learning realized accelerating of the object and decelerating it while keeping the desired mode, i.e., avoiding undesired slipping of the object.
 Examples of object manipulation with contact mode changes: Holding up, pushing and rotating manipulations |
 Learning of object rotation task using SVM and model predictive control |
Publication:
It is important for robots that act in human-centered environments to build image processing in a bottom-up manner. This paper proposes a method to autonomously acquire image feature extraction that is suitable for motion generation while moving in unknown environment. The proposed method extracts low level features without specifying image processing for robot body and obstacles. The position of body is acquired in image by clustering of SIFT features with motion information and state transition model is generated. Based on a learning model of adaptive addition of state transition model, collision relevant features are detected. Features that emerge when the robot can not move are acquired as collision relevant features. The proposed framework is evaluated with real images of the manipulator and an obstacle in obstacle avoidance.
 Extraction of visual features relevant to collision and motion generation using the extracted features |
 Extraction of body and object based on random exploration and object manipulation using the extracted information |
Publication:
Sensors of robots that act in unstructured environment sometimes do not provide complete observation, due to occlusion or limitation of sensing range. This paper presents a motion generation method for robot with multiple sensors with limited sensing ranges. The proposed method introduce extension of the action-observation mapping to outside of the sensing range of a sensor, based on the diffusion-based learning of Jacobian matrices between control input and observation variable. Multiple observation spaces can be integrated by finding correspondence between the virtual observation spaces. When a target observation variable is given to the robot, it can generate a motion from an observation space toward the target with another observation space using the extended observation space. The proposed framework is verified by two robot tasks, reaching motion toward the floor with a manipulator and navigation of mobile robot around the wall. In both cases, observation space by camera with limited view was extended and appropriate motion trajectories were obtained.
 |
| Figure(left): Motion generation from sensor(observable) space (i) to sensor
space (j) Figure(mid): Application (1); Observation of end-effector by camera and measurement of distance to floor with proximity sensor Figure(right): Application (2); Navigation of mobile robot toward a wall with camera and proximity sensor |
Publication:
Robots that interact with humans in household environments are required to achieve multiple simultaneous tasks such as carrying objects, collision avoidance and conversation with human, in real time. This paper presents a design framework of multiple human-interacting tasks to meet the requirement by considering stochastic behavior of humans. The proposed designing method first introduces petri-net for parallel multiple tasks. The petri-net formulation is converted to Markov decision processes and processed in optimal control framework. Multiple task arbitration is resolved by optimization with approximated value functions. Two tasks of safety confirmation and conversation tasks are mutually interacting and expressed by petri-net. Tasks that normally tend to be designed by integrating many if-then rules can be dealt with in a systematic manner in the proposed framework, that is, in a state estimation and optimization framework. The proposed arbitration method was verified by simulations and experiments using RI-MAN, which was developed to do interactive tasks with humans.
 |
 |
| Expression of parallel tasks by petri net | Experiment with RI-MAN in human-interactive environment |
Publication:
This research discusses the design of decentralized capturing behavior by multiple mobile robots. The design is based on a gradient descent method with local information. The task of capturing a target is divided into two problems, enclosing behavior and grasping behavior. We give analysis on convergence of the local control policy in enclosing problem. In grasping behavior, we consider the force-closure condition in decentralized form for designing a local objective function. The proposed local control policies were evaluated in simulations, where the flexibility of the system was verified caused by the decentralized nature of the system.
Enclosing behavior of a circular moving object by 6 robots |
Grasping behavior of an ellipsoidal object by 4 robots |
Grasping behavior by 6 robots |
Experiment with mobile robots |
Reference:
This paper presents a feature extraction method for robotic motion learning that optimizes image resolution to the task, thereby minimizing computation time. It utilizes mean-shift algorithms and principal component analysis for feature extraction, reinforcement learning for motion learning, and trial and error for finding the appropriate resolution. When applied to a manipulator pushing an object, the resolution adjustment method reduces the task time from one minute to 21 seconds.
Proposed architecture for feature extraction from image inputs |
Experimental setup with camera and manipulator for object pushing task |
 Flow of feature extraction using image information |
Pocessed images at each time step with different resolutions |
Publications:
This paper proposes a reinforcement learning method for dynamic control problems with holonomic constraints. The learning method is applicable to problems where the actual motion of the system is restricted to lower-dimensional submanifolds, so long as certain conditions are satisfied. Such dynamic control problems occur in robotic manipulation, which usually includes some holonomic constraints between the object and the robot or the environment. By introducing nonlinear mapping to one-dimensional space and approximating the boundary of a discontinuous reward function, the proposed method results in effective learning. The method is evaluated in a one degree of freedom object rotating task with contact force considerations. The effectiveness of the proposed learning method was verified by comparison to ordinal Q-learning and Dyna without the proposed mapping method.
Constrained motion of manipulation by robot hand |
Submanifold generated by constrained motion in configuration space |
Object rotation task with cosnraint to keep contact between hand and object |
An example of reward profile |
Publications:
Adaptive resolution of function approximator is known to be important when we apply reinforcement learning to unknown problems. We propose to apply successive division and integration scheme of function approximation to Temporal Difference learning based on local curvature. TD learning in continuous state space is based on non-constant value function approximation, which requires the simplicity of function approximator representation. We define bases and local complexity of function approximator in the similar way to the autonomous decentralized function approximation, but they are much simpler. The simplicity of approximator element bring us much less computation and easier analysis. The proposed function approximator is proved to be effective through function approximation problem and a reinforcement learning common problem, pendulum swing-up task and acrobot stabilizing task.
Comparison of learning performance among RBF network, fixed approximation and proposed adaptive resolution approximation |
Performance of control obtained by adaptive resolution function approximation |
 |
|
Publication:
 |
 |
 |
 |
 Adaptive resolution in 1D function approximation problem |
 Adaptive resolution in 2D function approximation problem |
 Pendulum swing-up task |
 Comparison of learning performance between adaptive method(red) and fixed structure(blue) |
In this paper, we present a real-time decision making method for a quadruped robot whose sensor and locomotion have large errors, considering the observational cost and the optimality. We make a State-Action Map by off-line planning considering the uncertainty of the robot's location with Dynamic Programming (DP). Using this map, the robot can immediately decide optimal action which minimizes the time to reach a target state at any states. The number of observation is also minimized. We compress this map for implementation with Vector Quantization (VQ). The total loss of optimality through compression is minimized by using the differences of the values between the optimal action and the others. In the simulation, the performance of some soccer behaviors were improved in comparison with current methods. The proposed method is implemented on the real robot and the low computation under the restriction of the memory was verified in the experiment.
Expression of state transition with uncertainty in state space including uncertainty parameter |
 An example of observation strategy with real quadruped robot |
Publication: