NaturalPoint Tracking Toolkits (Point Cloud and Rigid Body) Users Manual   6. Using the Rigid Body Software The Rigid Body toolkit is a robust, real-time 3D optical tracking solution. Markers can be attached to multiple objects in known patterns (rigid bodies) allowing them to be tracked in full 6DOF (position and orientation). Tracking data can be accessed in real time through network streaming support or the easy to use software API. The information provided below will help you get the best results from your OptiTrack cameras and Rigid Body software. 6.1 Getting Started Before using the Rigid Body software make sure to have all of the software and hardware properly installed and licensed as described in Chapter 3. Once the installation is complete you can begin using the Rigid Body software by following these instructions :   Connect 3 or more cameras to the USB port of the computer and arrange them to set up a capture volume as described in Section 3.5 ("Camera Placement").   Calibrate the cameras using the Point Cloud calibration tool as described in Section 4.2 ("Camera Calibration Tool") and save the calibration to a file.   Start the Rigid Body software by clicking on the "OptiTrack Rigid Body Tool" icon on your desktop.   Load the calibration into the Rigid Body software to begin capturing 3D point cloud data from the cameras.   Rigid bodies for tracking can be defined using the 3D point cloud data, or by loading existing rigid body definitions from a file.   3D Viewport Control The 3D viewport in the software is used to display cameras, 3D point cloud data and tracked rigid bodies. It is also used for selecting markers when defining rigid bodies. The mouse is used for marker selection and 3D viewport manipulation.   The Left mouse button is used for marker selection.   The Right mouse button rotates the 3D view when the button is held down and mouse is moved.   The Scroll Wheel on the mouse zooms the 3D view in and out.   The Middle button on the mouse pans/translates the 3D view when the button is held down and mouse is moved. 6.2 Working with Rigid Bodies The OptiTrack Rigid Body software creates a cloud of 3D points representing reflective markers which are visible to cameras within the capture volume. Rigid Bodies are clusters of reflective markers in a unique configuration which allow them to be identified and tracked in the cloud of 3D points. It is possible to track multiple rigid bodies at a time in full 6DOF (position and orientation).   The following section introduces best practices for defining rigid bodies and explains how to track them using the Rigid Body software.   6.2.1 Making Rigid Bodies To get the best tracking performance there are several factors listed below to consider when making the physical rigid bodies. The configuration(s) will also be influenced by the tracking volume and camera placement, the size and shape of the physical object which the markers are mounted to, and the other objects which may be present within the tracking volume.   Marker Types Spherical reflective markers will usually yield the most stable and accurate 3D tracking data. Hemispheric and flat markers are less desirable because their shape as it appears to the camera may change when it rotates, this can introduce errors when calculating their center of mass and position.   When using small rigid bodies, try to use small markers to reduce the chances of one marker occluding (blocking from view of a camera) other markers.   When tracking at larger distances from the camera, using larger markers can help improve the resolution of the tracking. Marker sizes typically range from 10 to 25 millimeters and larger.   NaturalPoint offers a range of high quality pre-made reflective markers for sale, for more details see Section 9 ("Cameras and Accessories").     Marker Quantity Three markers is the minimum required to define a rigid body. A larger number of markers can be used to increase the precision and reduce likelihood of the rigid body flipping. Using more than three markers also provides redundancy, then tracking can occur even when some of the assigned markers are not visible (the minimum visible for tracking is three).     Rigid Body Sizing The physical arrangement of the markers should not be too small and should not have markers too close together. Markers closer than 6 millimeters can result in the following issues :   Markers are more likely to overlap or occlude one another when seen from a camera, resulting in incorrect 3D position data or markers failing to be tracked.   If the point cloud tracking residual value is too high, or the calibration is poor, then rays for adjacent markers could be grouped together incorrectly resulting in misidentified marker locations and/or falsely reported markers.   Marker Arrangement For 3-marker rigid bodies, using asymmetrical marker arrangements improves the tracking reliability. Symmetrical triangles make it harder for the software to identify the correct orientation of a rigid body and increases the likelihood of arriving at an incorrect solution . This can often manifest as the rigid body flipping on its axis between frames.   When multiple rigid bodies will be tracked at the same time, avoid making rigid bodies which are too similar to each other. Making them individually unique with different marker arrangements and sizes will reduce the likelihood of misidentification and swapping.   6.2.2 Defining and Deleting Rigid Bodies Before you can begin tracking rigid bodies, you will need to create them in the software using point cloud data.   The first step is to load previously saved point cloud data or load a calibration and begin capturing new point cloud data. Once there is point cloud data to work from, the easiest way to redefine a rigid body is to select the markers in the viewport and press the "Add Rigid Body" button in the panel to the right. This will create a new rigid body in the shape of the selected markers. Selecting markers in the viewport will automatically pause the real-time capture or playback.     Multiple markers can be selected by holding the shift key down and drawing a rectangle over them using the left mouse button. Individual markers can be selected and de-selected by holding the ctrl key down while clicking on markers with the left mouse button. The color of the markers will change once they are selected.   A rigid body which uses all of the visible markers in the 3D viewport can be created by clicking the "Create From Visible Markers" button.   To preview the new rigid body, resume playback of the recorded or live data and the rigid body will now begin tracking. As rigid bodies are created, they will show up in a list in the panel to the right.   Rigid bodies can be removed by clicking on the name of a rigid body and pressing the "Delete Rigid Body" button.   6.2.3 Editing Rigid Bodies Once a rigid body has been created, it can be modified at a later time by selecting its name from the list of rigid bodies.     Properties : The name, ID, display color and tracking flexibility can be changed for a rigid body. The tracking flexibility controls how much the shape of the rigid body can distort before it will no longer track. Increasing the tolerance will allow the rigid body to track better if the physical shape is distorted, but can also result in the wrong markers being tracked for the rigid body or solving for the orientation incorrectly. The valid range of flexibility values are from 0 to 1.0 with a value of 1.0 being more flexible.   Movement Constraints : Constraints on how fast a rigid body can translate (move in X,Y,Z) and rotate (heading, attitude, bank) can be set. This allows the limitations of how fast the physical rigid body can translate and rotate to be imposed on the rigid body tracking in the software. Tuning these constraints can help improve the robustness of the tracking.   Rotation constraint values are in angular degrees per frame, with a range from 0 to 180 degrees. A value of 0 disables the rotation constraint. Translation constraint values are in meters per frame with a value of 0 disabling the constraint.   Marker List : This list contains all of the markers used in the rigid body and their positions. The point of origin for a rigid body can be changed by selecting a marker in the list and pressing the "Set Origin" button. Markers can also be removed from a rigid body by selecting a marker in the list and pressing the "Delete" button.   3D Model : Custom 3D models can be loaded and assigned to a rigid body for more intuitive feedback. A model assigned to a rigid body will translate and rotate in the 3D viewport based on the motion of the associated rigid body.   3D Model Translation is in meters, 3D Model Rotation ranges from -180 to +180 and is adjusted by the slider.   6.2.4 Loading and Saving Rigid Bodies Rigid Body definitions created in the software can be saved to a file for later use or manual editing, and easily loaded back into the software.   To save all of the rigid bodies currently defined in the software, follow the steps below. From the File menu choose "Save Rigid Body Definitions" Enter a filename and press the "Save" button To load previously defined rigid bodies into the software, follow the steps below. Note that loading a set of rigid body definitions will replace any rigid bodies currently defined in the software. Once loaded, the rigid bodies will show up in a list in the panel to the right. From the File menu choose "Load Rigid Body Definitions" Select the file with rigid body definitions and press the "Open" button   6.3 Tracking and Recording The OptiTrack Rigid Body software provides real-time tracking with the ability to record 3D point cloud data and play it back offline. The following section covers data preview, capture and export to other applications.   6.3.1 Tracking Status Status about the cameras, 3D point cloud reconstruction and playback can be found in Frame Information area. The feedback provided here can be helpful in assessing the health of the system and troubleshooting.     Marker Count : Indicates the number of markers contained in the current frame of the 3D point cloud.   Rigid Count : Indicates the number of rigid bodies found within the current frame of the 3D point cloud. Rigid bodies defined in the software which are not found in the current frame will not be included in the total.   Frame : During playback of recorded data this will display the current frame number. When operating in real-time mode with the cameras, the value "Real-Time" will be displayed instead.   Display FPS : Indicates the rate in frames-per-second at which the 3D viewport is being updated.   Incoming FPS : Indicates the rate in frames-per-second which 3D point cloud data is being processed. When using V100 cameras this number should close to "100". Large quantities of markers and rigid bodies can load the system down and may reduce this number, indicating that some frames are being skipped or dropped.   Camera Sync : Indicates the synchronization status of the cameras. When the camera-sync cables are connected and the cameras are operating in hardware synchronization mode, the value "Hardware" will be displayed. Using the cameras in hardware synchronization mode will produce the best quality 3D point cloud data. If the camera-sync cables are not connected between all cameras then the software will attempt to synchronize the cameras using the time of delivery for each frame. When software synchronization is being performed, the value "Synchronized" will be displayed.     6.3.2 Real-time Tracking Processing and displaying live 3D point cloud data from the cameras as it is captured is referred to as real-time operation. As markers move within the field of view of the cameras, a 3D point cloud of their current positions is created. The 3D point cloud is searched to determine the location of currently defined rigid bodies. This data can be recorded for later use, or streamed to other applications as it comes in from the cameras.   Loading Camera Calibrations and Starting the Cameras Before the software can begin tracking in real-time the cameras need to be calibrated using the Point Cloud calibration tool as described in Section 4.2 ("Camera Calibration Tool"). Once the calibration has been completed it should be saved to a file. This file can then be loaded into the Rigid Body software by choosing "Load Calibration Profile" from the Tracking menu. As soon as the calibration profile is loaded the software will begin creating the 3D point cloud and displaying it in the viewport.   To stop collecting tracking data from the cameras, choose "Shutdown Cameras" from the Tracking menu. After as shutdown has been performed, a calibration file will have to be loaded again in order to resume tracking.     3D Point Cloud The point cloud engine collects 2D marker data from the individual cameras and reconstructs it into a 3D point cloud containing the identified markers. The rigid body solver then searches this 3D point cloud to find any of the currently defined rigid bodies.   Cameras which observe the same physical markers from different angles can use known information about their positions to triangulate where the marker is in 3D space. The imaginary lines drawn from the focal point of a camera to the projected location of each 2D marker in its field of view are called Rays. 3D points are formed when multiple rays from different cameras cross within a specified distance of each other. A ray which does not cross with other rays, or which crosses at too great of a distance from other rays will not be used for creating 3D markers.   The default settings for point cloud tracking are usually sufficient, however they can be modified to tune the performance for specific camera and rigid body configurations. To change these settings, open the Point Cloud Properties tab and check the "Override Tracking defaults" checkbox.     Residual : Rays from the camera to the marker must cross closer than this value (in meters) to form 3D points. Making this value too large can result in an increased numbers of falsely detected 3D markers. Making the value too small can prevent 3D markers from being formed, especially if the cameras are poorly calibrated.   Min Angle : Rays from the camera to the marker must cross at an angle (in degrees) larger than this to form 3D points. If cameras observing the same marker are positioned very closely together, the this value may need to be reduced.   Min Ray : Rays from the camera to the marker must cross farther than this distance (in meters) from the camera to form 3D points.   Max Ray : Rays from the camera to the marker must cross closer than this distance (in meters) from the camera to form 3D points.   Show Cameras : Controls whether 3D representations of the cameras are shown in the 3D viewport.   Display Capture Volume : This provides a way to visualize the physical area of the capture volume. It is generated by estimating the 3D region where the field of view of multiple cameras overlap. It can be used to determine which areas have better tracking coverage.   6.3.3 Recording and Playback The real-time 3D point cloud data from the cameras can be recorded and played back offline. When using recorded data, rigid bodies are tracked in the same way as when live data from the cameras is used. Recorded data provides the ability to play the data backward as well as review a specific set of motion in close detail as many times as needed. Since the recorded data can be streamed to other applications, it can also be used in testing and validation for downstream tools which consume the rigid body tracking data.   Playback Button Controls Rewind to the First Frame Play Backward Stop Pause Play Forward Advance to the Last Frame Record   How to Record Tracking Data Before 3D point cloud data can be recorded, make sure that a calibration file has been loaded and the cameras are tracking as described in Section 6.3.2 ("Real-time Tracking").   To begin recording, press the record button. When the desired amount of data has been captured, recording can be stopped by pressing either the Record button again, or the Stop button. To save the data that was recorded, choose "Save Capture Data" from the File menu, enter a file name and press the "Save" button.     Playing Back Recorded Data If recorded 3D point cloud data is already loaded, then playback can be started by pressing the Play button. To play the data in reverse, press the Play Backward button. If data has not yet been loaded then choose "Load Capture Data" from the File menu, select a file and press the "Open" button.   While data is being played back the software will display it in the 3D viewport and the timeline scrubber will advance. The current frame number will be displayed in the "Frame Information" tab (if the tab is expanded). Playback can be stopped using the Pause or Stop buttons.     Switching Between Real-time and Recorded Data Playback The Rigid Body software makes it easy to switch between capturing real-time data and playing back recorded data. The value "Real-time" will be displayed when the software is in real-time mode, otherwise it will display "Frame" plus the current frame number.   If recorded data is currently loaded in the software, pressing either of the Play buttons will begin playing it back. Using the Pause button will momentarily halt playback, but will not switch to real-time mode. Pressing the Stop button switches the software into real-time mode until any of the other playback controls are pressed again.     Editing Recorded Data The Rigid Body software provides a set of basic tools for editing recorded data. These tools are located under the "Recorded Data" section of the Edit menu.   Delete All Frames : Removes all of the recorded frame data.   Delete Before Current Frame : Removes all of the recorded frame data before the currently displayed frame.   Delete After Current Frame : Removes all of the recorded frame data after the currently displayed frame.     Recording and Playback Options A number of options for recording and playback are available, they can be found under the "Recording and Playback" tab.     Playback Rate : The speed at which the recorded 3D point cloud data is played back can be adjusted. Setting the value to smaller numbers will make the data play back slower, larger numbers will result in accelerated playback. A value of 50% is 1/2 the real speed, and 200% is double the real speed.   Loop Playback : When the last frame in a set of recorded data is reached, playback will continue at the first frame if this setting is checked.   Marker Locations : Controls whether 3D point cloud marker locations are displayed in the 3D viewport.   Rigid Body Locations : Controls whether identified rigid bodies are displayed in the 3D viewport.   Rigid Body Markers : Controls whether the marker positions which a rigid body is composed of are displayed in the 3D viewport. Showing these can be helpful in determining how much the marker locations of a physical rigid body differ from the expected marker positions.   Rigid Body 3D Models : Controls whether 3D models assigned to rigid bodies are displayed in the 3D viewport.   6.3.4 Exporting and Streaming Tracking Data The 3D point cloud and rigid body tracking data can be exported using files or streamed in real-time for use in other applications. The details below explain how to use these features.     Exporting to CSV format Recorded tracking data with rigid body locations can be exported in the CSV format for use in other applications. The exported file will contain comments at the top which describe the data formatting.   Before exporting data to a CSV file, make sure that recorded data is currently loaded and rigid bodies are defined. Then choose "Export" from the File menu, enter a file name and press the "Save" button.     Streaming tracking data Real-time and recorded data can be streamed over the network for use in other applications. Several different methods of streaming are supported including industry standards VRPN and Trackd. All of the streaming methods are built on a network transport which allows the data to be made available on the local computer as well as remote network computers.   Before trying to stream, make sure that either recorded data is currently loaded or the cameras are tracking in real-time. It is also necessary to have rigid bodies defined.   Since data is streamed over TCP/IP on the network, it may be necessary to review your firewall settings. If the firewall is blocking traffic it may prevent the Rigid Body software from properly sending the data or the client applications from reading it.   The settings which control streaming can be found under the "Streaming" tab.     NaturalPoint Engine (NatNet) : The NaturalPoint streaming engine is a custom streaming implementing which can be used to access real-time and recorded tracking data over the network. NaturalPoint provides sample source code for writing a client which receives the streaming data. The standard sample client can be used to verify that streaming is functioning properly.   Streaming is enabled by checking the "Broadcast Frame Data" checkbox.   Streams : rigid bodies and markers   Network Details : port 1510, multigast group 1001, UDP milticast   VRPN Engine : VRPN is an open source set of classes and a protocol which can be used to access real-time and recorded tracking data over the network. It features low latency and has built-in auto-renegotiation if a connection is temporarily dropped. NaturalPoint provides sample source code for a listener that receives tracking data. Additional sample code is available from the VRPN website .   In VRPN objects are identified and accessed by a name. In the Rigid Body software's implementation of VRPN the name used is the one assigned to a rigid body under the "Rigid Body Definition" tab.   Streaming is enabled by checking the "Broadcast Frame Data" checkbox.   If needed, the VRPN port can be changed from the default by editing the port number setting.   Streams : rigid bodies   Network Details : selectable port, default port 3883, TCP + UDP   Trackd Engine : Trackd is a standard created by VRCO/Mechdyne which can be used to access real-time and recorded tracking data over the network. NaturalPoint provides sample source code for a listener that receives tracking data. Additional information is available from the VRCO website.   The Trackd server must be configured before data can be streamed, use the following steps to get it set up.   A Trackd module (dll) and sample configuration file are provided by NaturalPoint.   Configure Trackd by calling it the "naturalpointtracker".   Define the number of rigid bodies desired to track.   The streaming host defaults to localhost, but can be changed to a different network address.   Once the configuration is complete, run the Trackd server.   Streaming is enabled by checking the "Broadcast Frame Data" checkbox.   Streams : rigid bodies   Network Details : port 4994, TCP + UDP 6.4 Application Preferences Several aspects of the Rigid Body software can be customized to better suit your needs. The settings which control customization are found in the Preferences window, which can be launched by choosing "Preferences" from the Edit menu. Changes to these settings will be preserved when the software is closed, and restored when it is started again.