The robot HERB is a mobile manipulator developed at CMU [1]. It was built using a Segway robot base, a custom aluminum frame, two Barrett WAM 7-DOF manipulators, and a custom 2-DOF pan-tilt neck mechanism with a universal mounting plate to attach various perception sensors on the head. The project described here was to develop an upgrade plan for this robot that would be called HERB version 3.0.

The existing robot HERB v2.0 uses a Segway RMP-200 mobile base. When HERB was originally developed in 2008, this was one of the only indoor mobile robot platforms that was available off-the-shelf. It had already been used successfully for other research robots performing tasks like opening doors [2] and finding objects around the office [3]. The differential-drive configuration means the robot can rotate in-place and the pneumatic wheels support a high payload. Originally the self-balancing Segway provided a novel platform for a mobile manipulator, which was an interesting research topic at the time. Later, caster wheels were added to HERB and the gyroscope was disabled. Unfortunately a limitation is that the base can’t move laterally (sideways)) in front of a table surface.

The neck is a custom 2-DOF pan-tilt neck mechanism. The actuators are the same motors and controllers as used in the Barrett WAM arms. The control software for the 14 joints in the robot’s arms was easily extended to support two extra joints. The neck is force controlled for safe operation around humans. However this sub-system is wearing out and has developed some issues.

The robot’s head is a universal bolt plate for attaching various different sensors, depending on the research application (see Fig. 4 below). Currently HERB has dual Microsoft Kinect v2 sensors (Fig. 1). One depth sensor is used to detect faces and body posture, using the Microsoft SDK. The second identical sensor is used for new perception algorithms developed on Linux and ROS.

An analysis of the robot’s functionality is shown in Table 1 below. Also, as part of the upgrade, some other sub-systems may also be affected. Currently localization of the mobile base is performed using a Hagisonic StarGazer system that detects fiducial markers on the ceiling. The Segway base has its own battery module. A second battery is used to power the on-board computers and a 48 VDC bus for the arm and neck motors.

HERB is surprisingly heavy! The new mobile base must support a payload of 130 kg (286 lbs). The robot’s mass was estimated from the CAD model using the density of aluminum, combined with the mass of the Barrett WAM arms and the heaviest choice of sensor payload (CRL MultiSense).

The robot is generally operated indoors on an industrial carpet floor. The majority of research projects have the robot located in front of a table or kitchen counter, so the base needs to drive sideways (laterally). The base should also be narrower than 32 inches so that it can maneuver through an ADA-compliant doorway (Fig. 2), like what is typically found in the home or office.

Various off-the-shelf mobile robot bases were compared, see Table 2 below. The most suitable options were two products manufactured by Neobotix. The team had a predilection for choosing a Mecanum base because it would be easier to write code for a holonomic base and ultimately the Neobotix MPO-500 was selected.

The total mass of the head (sensors + bracket) was estimated to be 4.5 kg (10 lbs). Various PTU (Pan Tilt Units) from Directed Perception and Schunk were compared (see Table 3 below).

Unfortunately there were no robot “neck” modules available off-the-shelf that met all the requirements for a collaborative home robot. We considered building our own pan-tilt mechanism but this was not our expertise.

The Directed Perception (FLIR) PTU D48 has an aesthetically pleasing appearance, however the unit does not always report back the current joint angles after changing position. This is not acceptable for a robot. Therefore we selected the Schunk PW-70. This meets the minimum requirement to support the sensor payload and not come crashing down when power is disconnected.

Head:

The HERB robot has used various “heads” (sensors) over the years, including a camera, spinning Hokuyo LiDAR, Kinect, Xtion, RealSense and an image projector (Fig. 4). This is typical for a research robot, where project requirements change and 3D sensor technology continues to improve. Ideally the head would also incorporate a microphone array but this was out of scope for the proposed upgrade.

Previously the LiDAR was used to scan objects and create 3D mesh models. This capability has been replaced by the Kinect sensor. However a LiDAR sensor would enable markerless, laser-based localization using features detected in the PointCloud (replacing the StarGazer system). It could also be used for collision detection with a wide FOV (Field of View), to detect humans standing next to the robot, or behind it (Fig. 5-6).

Various perception sensors and combinations thereof were compared (see Table 4 below). The only sensor package available off-the-shelf that met the requirements was the CRL MultiSense SL. This would be combined with an interchangeable RGB-D sensor like the Microsoft Kinect v2 or Intel RealSense. The stereo camera data (from the MultiSense) could be processed by one computer and the depth sensor data by a second computer.

A new frame for the robot’s body was fabricated from aluminum, which supports the two Barrett WAM arms. The arms were transplanted from the old robot (Fig. 9). The frame was bolted to the mobile base, which is a Neobotix MPO-500 with mecanum wheels. The holonomic base makes it easy to drive the robot around manually using a joystick (teleoperate) however the wheels do struggle under the weight of the robot.

The Schunk PW-70 pan-tilt works well for the “neck”, with stiff and rigid motion control. The “head” is an aluminum bracket that integrates a CRL MultiSense SL, Microsoft Kinect v2 and Intel RealSense sensors. This combination provides flexibility for different projects.

The final upgraded robot HERB v3.0 is shown in Fig. 11. Future work is to develop improved software for localization and base motion planning.