Payload developers can add custom sensors or extra compute power to Spot through mechanical, electrical, communication, and software interfaces. Installing the bosdyn-client, bosdyn-choreography-client and bosdyn-mission packages will also If you have any questions, please email support@bostondynamics.com. Send commands like body posing, velocity, and direction to the robot. The Boston Dynamics Python API has a corresponding set of clients for each service, which are created off of the robot object. Learn more about our partner Vinsas computer vision pipeline. This command returns once the robot is powered on or throws an error if the power command fails for any reason. Now we can create a RobotStateClient for the robot-state service, and obtain information about Spot: The structure of the robot state message retrieved above is defined by its protobuf definition. Pythons futures architecture is used as the underpinning of asynchronous communication. Act on data insights and performphysical work in people-centric environments. Assuming that the username is user and the password is password, issue the following command. Above, we've used the default IP when the robot is hosting its own access point. Hello_spot will fail because there is not an E-Stop endpoint. installation. There are thousands of pounds of pressurized combustible material out there. Equip Spot with Spot CAM+IR to detect issue-indicating hot spots on machines or electrical conductors. Program expressive poses to use Spot as an entertainer in a show or amusement park. Conceptual documentation. NOTE: If you installed the SDK using a zipfile, be careful to understand what changes youve made, as users sometimes inject errors into the SDK code unintentionally. Create autonomous routes or drive the robot to remotely inspect facilities and improve awareness of plant operations. Users with multiple python versions, anaconda, etc., are responsible for maintaining separation between those installs. Try out the wasd programming example. Inspect progress on construction sites, create digital twins, and compare as-built conditions to Building Information Modeling (BIM) autonomously with Spot. Balance dynamically in uncertain surroundings with payloads of up to 14kg. Program dynamic movements and expressive poses through the API or drive the robot in real-time as part of a performance. For this tutorial, we will just issue stand and safe power off commands. Access our SDK on our GitHub to download code and begin developing on Spot. Services are what you communicate with on Spot, use them to issue commands, retrieve information, etc. This call returns both high level feedback (is the robot still processing the command?) as well as command specific feedback (in the case of stand, is the robot standing?). All communication to the robot is over a secure HTTPS connection. Both the endpoint and the E-Stop systems stop level is ESTOP_LEVEL_CUT - if a single Endpoint wants to cut power, the entire system will cut power. Control the robot from afar using an intuitive tablet application and built-in stereo cameras. In this example, we only create one robot object, but it is possible to create and control multiple robots in the same program with the Boston Dynamics API. To install a different version of the packages from PyPI, for Watch Spot pull Adam Savage around San Francisco or perform with Cirque du Soleil. Privacy Statement | This is standard boilerplate for the API, where we're initializing the SDK and logging in. Optionally rotate the image, depending on which camera it is from (only the left and back cameras on Spot are upright). If we could have a robot with the proper sensors out there, wed much rather do that., Adam Ballard, Facilities Technology Manager, bp, Every 12 hours, we follow the same path through the plant, and were assessing situational awareness. In addition, the stand command can be modified to control the height of the body as well as the orientation of the body with respect to the footprint frame. ../../protos/bosdyn/api/robot_state.proto. Leases can be returned when the client no longer wants to control the robot. A wide variety of example programs and a QuickStart guide are also included. All Boston Dynamics API programs start by creating an SDK object with a client name argument. Energy giant National Grid is using Spot to keep employees safe and ensure uptime at a critical facility. All rights reserved. Now that youve authenticated to Spot, created an E-Stop endpoint, and acquired a lease, its time to power on the robot. <_InactiveRpcError of RPC that terminated with: debug_error_string = "{"created":"@1585323526.280242100","description":"Error received from peer ipv4:192.168.80.3:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Deadline Exceeded","grpc_status":4}", get_robot_state_async programming example, ['back_depth', 'back_depth_in_visual_frame', 'back_fisheye_image', 'frontleft_depth', 'frontleft_depth_in_visual_frame', 'frontleft_fisheye_image', 'frontright_depth', 'frontright_depth_in_visual_frame', 'frontright_fisheye_image', 'left_depth', 'left_depth_in_visual_frame', 'left_fisheye_image', 'right_depth', 'right_depth_in_visual_frame', 'right_fisheye_image'], stop_level_details: "Not all endpoints are registered", stop_level_details: "Endpoint requested stop", client_name: "HelloSpotClientlaptop-kbrandes01:hello_spot.py-32049". Try it again, and this time, push the E-Stop button and watch the robot do a glide-stop. Remember, E-Stop is your friend. Lets make a LeaseClient for the lease service and list the current leases: The lease-able resources are listed: body, mobility, (full-arm, gripper, arm for robots with an arm). Attach and integrate unique outside hardware using mounting rails and payload ports. Power off the robot using the power_off command. The robot can have multiple clients but only one can control the robot even as other clients may be requesting data or acting as E-Stop endpoints. Our goal is to collect images of the dog toy in the same environment we want to test in. It typically takes several seconds to complete. If it didnt, be sure to check that the Motor power enable button on the back of Spot was properly turned on. The command above installs Check if pip is installed by requesting its version: If pip is not found, youll need to install it. Add payloads provided by Boston Dynamics or third-parties to enhance Spots sensing and data processing capabilities. Add processing to visuals collected by the Spot CAM+ payload to detect water and steam leaks around plants and note equipment with degraded performance. Note the robot-id service is listed, which we just used in the previous section. Payload developer documentation. Spot Python SDK works with Python 3.6 or Python 3.7 only. Hear how real developers have utilized the platform to meet customer needs. From a safety perspective, this allows users to define a period of time for which a command will be valid. (or stop in an emergency!!). Easily configure sensors, cameras, and other devices, process the data Spot collects into actionable insights with a CPU and GPU, and relay those insights to your team over 5G/LTE. This session will provide you with a proven playbook of the processes our customers have used to start testing and deriving value with Spot. DEVELOPER SPOTLIGHT: DESIGNING APPLICATIONS ON AN INTUITIVE PLATFORM. Common failures include using the wrong version of python, installing python packages to the wrong python, or using pip associated with the wrong python. Please note however that Boston Dynamics recommends that you first have your designated robot administrator log onto the robot with admin credentials and change passwords to increase security. The robot class maintains a timesync thread. Contact sales@bostondynamics.com to get a Spot robot. bp uses Spot to improve safety and efficiency. Conduct virtual patient interviews or deliver medicine and food by mounting a tablet and other equipment on Spot. The Spot API exposes on-robot capabilities via a set of network-accessible services - similar to a microservice architecture. We want a diverse set of viewpoints, so we'll walk Spot around while taking the images. Safer, more efficient and more predictable operations. The body resource covers all of the motors on Spot and for most use-cases it is sufficient to issue all commands just using the body lease since services on Spot will break apart the lease to use the minimal set of resources necessary. Youre now ready to go! The robot object provides a method to check the power status of the robot. If it has been too long since a check-in, the robot will commence a Comms Loss Procedure - sitting down if it can, and then powering off. Please review the Release Notes to see what has changed. The power_on helper function first issues a lower level power command to the robot and then waits for power command feedback. The third line creates a client of Spots robot-id service. or upgraded from PyPI with the following command. Terms of Use, Machine Learning with the Network Compute Bridge, Fire Extinguisher Detector with the Network Compute Bridge, Test Image Service Implementation with Get Image, GraphNav and Recording Service Command Line Interfaces, Part 5: Detecting People and Playing Fetch, Configuring Docker containers in SpotCORE, Spot CORE system management tool: Cockpit, open source off-the-shelf labeling program. Before you label the rest of your images: Let's make sure things are working before we spend a lot of time labeling.
Issue the following command to get your Spot robots ID: If this worked for you, SUCCESS! We use python in this guide butif you have multiple versions of Python installed then running python might reference an incorrect version (e.g.
Run Hello Spot - lets see the robot move! Each client is issued a clock identifier which is used to validate that the client has performed timesync, for services that require this functionality. install bosdyn-api and bosdyn-core packages with the same version. Specify the verbose flag with --verbose , and youll get a lot of information! When taking lease control of Spot, the lease should first be acquired, and then the keepalive should be created to retain ownership of the lease resource. WHITEPAPER: Benefits of Reliable Data for Asset Performance Management. If you want your ML model to work more reliably, capture more pictures in a wider variety of locations. Set Spot up to do autonomous inspection rounds or use the robot to create digital twins of a plant in advance of rework. Make sure that the robot is in a safe spot, in a seated position, with a charged battery, and not connected to shore power. IP address of the robot. Take a look, they are right here in your distribution! Label a few more images with d and w then open your annotations folder in a file browser. You can also try the -v or --verbose to get more information to debug the issue. Rename the top-level directory spot-sdk-master to spot-sdk. Each time we: Now that we have a script to capture images, let's run it (use Ctrl+C to stop capturing): First we make a directory to store our images. An E-Stop Endpoint is the client component of the E-Stop system which lets a user immediately kill power. example 3.1.2.1, use the following command. Applications using the Python library can control Spot and read sensor and health information from Spot. Access our Software Development Kit (SDK) documentation and learn how to develop on Spot. Utilize pre-built solutions from an existing network of third-party software and hardware providers. If you just unboxed your Spot robot, you will find a sticker inside the battery cavity with wifi, admin, and username user credentials. You can also optionally pass arguments to the launcher to control what version of Copyright 2022 Boston Dynamics. Leave it running, and open a second python window, and again run hello_spot: Your Spot robot should have powered up its motors, stood up, made a few poses, taken a picture, and sat down. To retrieve the robot id like we did in Spot Python SDK Quickstart well first need to create a robot object, using its network address as an argument. Finally, the RobotIdService responds to the GetRobotId RPC with a response including information about the robot. Use our video tutorials, how-to articles, and detailed documentation to jumpstart your journey with Spot. Contact support@bostondynamics.com for help. Create routine tunnel inspection routes and attach additional payloads to take measurements and ensure safe working conditions. Learn how GlobalFoundries is automating thermal data collection for condition monitoring and predictive maintenance. Automate sensing and inspection, capture limitless data, and explore without boundaries. See the get_robot_state_async programming example for how to use these functions. The client library is written such that most implementation details of timesync are taken care of in the background. Create a virtualenv, being careful to point at the proper python executable. Learn more about Trimble (pictured) and FARO's laser scanning integrations. The Spot Python SDK distribution is available at https://github.com/boston-dynamics/spot-sdk. You'll also want to run the script with a few different cameras to capture a diverse set of images. # Command Spot to rotate about the Z axis. If you cant import bosdyn.client without an error, you may have multiple instances of Python on your computer and have installed bosdyn to one while running the other. Its important to communicate across shift changes and make sure youre handing off critical information. After timesync is established, this thread will make periodic calls to maintain timesync. With its ability to move through a construction environment, and then, its ability to integrate, we could access the platforms software and build with APIs if we needed to., Tristen Magallanes, Innovation Analyst, Swinerton. The id command is one of the simplest Spot commands, so its useful to understand how it works, since every command to Spot performs many of the same basic functions. See the Concept documents for more details about Spots API Architecture. Try making simple modifications to the code. The GitHub repo where all of the Spot SDK code is hosted. NOTE: the lease keepalive object must remain in scope for the entire duration of the program that is using the lease for commands. As a platform, Spot is designed to give your team the flexibility to configure the system to maximize ROI. Before robot state can be retrieved, you need to authenticate to the robot. In the above output, a blocking GetRobotId RPC can be seen being made to the bosdyn.api.RobotIdService.
(Note you could also supply a full pathname c:\path\to\install\python.exe to directly call the executable). To change E-Stop status and allow power, the endpoint needs to check in on a regular basis. Lets make an asynchronous call for the robot id and wait for the result from the returned future object: The robot-state service contains dynamic information about the robot such as location, battery status, etc. You'll learn how to: At the end, youll have a fully-functional API example that enables Spot to autonomously play fetch with you. For example, an interactive UI should give the E-Stop system a keep_running_cb function which blocks until the UI thread has run a cycle. Carry and power up to 14kg of inspection equipment. Perform more detailed inspections with a thermal camera in addition to Spot CAM+s color panorama, PTZ, and communication capabilities. Develop applications and payloads for Spot using the Boston Dynamics Spot SDK. In this example, I've captured about 400 images. Drive Spot remotely to get eyes on dangerous situations or inspect hazardous packages from afar.
This command wraps several RobotCommand RPC calls. Program repeatable autonomous missions to gather consistent data. If all else fails, contact support@bostondynamics.com for help. Read through the protocol buffer definitions and the Spot Python SDK source code documentation to understand even more. Once you're done, you should have a large number of XML files which contain the labels. For example, to run python 3 on Ubuntu 18.04 you would run python3 and on Windows you could use the Python launcher and run py -3. Congratulations, you are now a full-fledged Spot Programming Example Operator! Cruise over loose gravel, grass, curbs, and stairs. Machine Learning with the Network Compute Bridge, Fire Extinguisher Detector with the Network Compute Bridge, Test Image Service Implementation with Get Image, GraphNav and Recording Service Command Line Interfaces, Part 5: Detecting People and Playing Fetch, Configuring Docker containers in SpotCORE, Spot CORE system management tool: Cockpit, ../../protos/bosdyn/api/robot_state.proto. Note that in many implementations, you should specify the keep_running_cb argument to EstopKeepAlive, a function called by the background thread to see if check-ins should continue. The robot should now be standing. With python and pip properly installed and configured, the Python packages are easily installed Spot SDK completely exposes the protobuf, so to really understand Spot programming you want to look at and understand the protobufs. The force_simple_setup call issues a few API calls to make your E-Stop Endpoint the sole endpoint in a new E-Stop configuration. The distribution contains programming examples, protobuf definitions and API documentation. Drive Spot underground remotely post-blast to look for cracks and instabilities and ensure safe conditions for workers. Enhance the range and accuracy of Spots autonomy system on large and changing sites with LIDAR. The stop_level is now ESTOP_LEVEL_NONE, indicating that power can start up. git clone https://github.com/boston-dynamics/spot-sdk.git (recommended). (only for consistency with this document, nor required). The guide defaults to a Linux setup. SAFETY NOTE: the act of registering an endpoint will trigger an emergency stop on the robot, so only perform this step when Spots motors are already powered off. Often, it is useful to know about the position of Spot and how it relates to the world around it. Record maps and create your own missions with behavior treesby programming actions associated with specific points on the map. That's about the minimum you need to get something to work. Well use the EstopKeepAlive class to do these checkins on a regular basis from a background thread. Attach a Spot CAM+ to the base platform to collect color visuals and read analog gauges measuring pressure, flow, temperature, and more, as well as 30x optical zoom and inspect gauges from afar. This command id can then be passed to the robot command feedback RPC. Theres one more step before powering on Spots motors, and thats to acquire ownership of the robot. installed. Integrate Spot with sensors and software to develop applications with industry partners. Use Spots mechanical and electrical interfaces, documented in the SDK, to integrate sensors or custom payloads. OK, now we have an E-Stop. If you dont see the 4 bosdyn packages with your target version, something went wrong during Python client library. There are many out there, but this one is simple and works great for our purposes. Lets take a look at the initial E-Stop state of the robot by creating a client to the E-Stop service and requesting status: The stop_level: ESTOP_LEVEL_CUT line indicates that power will not be enabled since the E-Stop level is CUT.
There are a few options: pip comes preinstalled with all Python 3 versions >= 3.4 downloaded from python.org, Use an OS-specific package manager (such as the python3-pip package on Ubuntu). 
Learn the basics of Spot Operation and Safety with our comprehensive video tutorial series. Terms of Use, Machine Learning with the Network Compute Bridge, Fire Extinguisher Detector with the Network Compute Bridge, Test Image Service Implementation with Get Image, GraphNav and Recording Service Command Line Interfaces, Part 5: Detecting People and Playing Fetch, Configuring Docker containers in SpotCORE, Spot CORE system management tool: Cockpit. Here are the next steps we recommend: Read our next section, Spot Programming Highly recommended! The Boston Dynamics Spot Python SDK works with most operating systems including: Windows WSL use is discouraged due to so many examples having graphics. Your username or password is incorrect. If you provided the wrong credentials, an exception will be raised. The list_image_sources RPC returns valid camera source names. Set up imports for images and the Spot API: Here, we add the common API arguments including --username, --password, and hostname. The following example sets a too short timeout and fails.
Git users can simply use git diff to understand all changes they have made. gRPC is a high-performance networking connection for services which supports a wide variety of programming environments. Using the source names listed above, we can capture an image from one or more image sources. A valid lease must be presented with every mobility command to the robot. For that reason, I didn't label the flying dog toy. Verify your python install is the correct version. Note: Please ensure the virtualenv was created and uses the correct version of python by Learn more about HoloBuilder's custom-built app SpotWalk. Change your working directory to the E-Stop example and run the nogui version: You should now have a big red STOP button displayed on your screen. Privacy Statement | The robot checks some basic pre conditions (powered on, not faulted, not E-Stopped) and returns a command id. Payloads add additional sensing, communication, and control capabilities beyond what the base platform provides. Are they where youd expect them to be? If you see a version incompatibility error during pip install such as: Try uninstalling the bosdyn packages (Note: unlike install, you will need to explicitly list all 4 packages) and then reinstalling: Make sure that the packages have been installed. Well explain the pieces. In addition to blocking calls, clients support non-blocking asynchronous calls. Replace my_spot_env with the name of the virtualenv that you created using the Spot Quickstart Guide: Install OpenCV's python bindings in your virtualenv: Make a folder called ~/fetch that we'll put everything into: Copy (or download) the script below into a file called capture_images.py and place it in the ~/fetch folder. Were here to change your idea of what robots can do. Learn more about our tablet interface and autonomous route planner Autowalk, built on the SDK. Make sure to not overwrite existing files. The latter uses the Python launcher, which by default starts the most recent version of Python Wait for the fans to turn off (and maybe 10-20 seconds after that). I should wait until it's on the ground. The commands proto can be quite expressive, and therefore, if going beyond default parameters, non-trivial. Copyright 2022 Boston Dynamics. But, if you also want to be a full-fledged Spot Programmer, you need to understand more about how Spot works. 
Install packages as needed, including Spot SDK. line. While simply installing the Boston Dynamics Python packages is sufficient to deploy solutions, developers need to download the full Spot SDK distribution to actually develop solutions. Learn how Brigham and Women's Hospital is using Spot to protect healthcare workers. This can be useful in high performance applications where a thread of execution can not stall waiting for an RPC to complete. python to launch: Pip is the package installer for Python. Like the E-Stop, lease holders need to periodically check in with Spot to indicate that they are still actively controlling the robot. Then we run the code with parameters: While capturing, drive the robot around to get a variety of viewpoints. Then run hello_spot: HELPFUL HINT: When working with any Spot SDK programming example, always use the associated requirements.txt to install dependent third party packages. Don't worry about this stuff too much. Note that in some circles the term E-Stop implies a hardware power short-circuit, hence our semantic dancing, as Spots Motor Power Authority is a networked software solution, not a hardware solution. Note that the output text itself demonstrates Spots use of Pythons logging facility, we recommend you perform your logging with the same. This prevents a frozen client from continuing to allow power to the robot. Do I want the network to find flying dog toys? No check-ins from the E-Stop Endpoint have happened yet. Spot may have more than one E-Stop Endpoint registered at a time - for example, during operator training the trainee may have a tablet which lets them control the robot and the E-Stop, and the trainer may have a tablet which also lets them e-stop the robot. The Spot SDK and the third-party packages used by many of its programming examples use pip to install. The majority of services require the user to be authenticated - this prevents random network attackers from being able to control the robot or intercept information which might be sensitive. Application developers can use the Spot Software Development Kit (SDK) to control the robot, access sensor data, and integrate with Spots autonomy system. Python 3.8 is not supported. Consider using virtual environments (see above). Manage Multiple Python Environments with virtualenv, Get a copy of the full SDK distribution from github. Spot has 5 fisheye cameras in addition to 5 depth cameras. You are now successfully communicating with Spot via Python! Equip Spot with a microphone or utilize the one on the Spot CAM+ to detect abnormal operating noises like ticking, grinding, or whirring. Here are some tips to working with virtualenv. Lets retrieve the left_fisheye_image and display it (unless you are on MacOS or WSL). The stop_level_details: "Not all endpoints are registered" line indicates that there are no E-Stop Endpoints registered. The endpoint itself says ESTOP_LEVEL_CUT, with a very long ago time_since_valid_response. All rights reserved. Privacy Statement | Now that we are confident things are working, label the rest of the images. gRPC uses Protocol Buffers as the messaging format, which has a compact over-the-wire representation and supports backwards and forwards compatibility. All rights reserved. Change your working directory to the hello_spot example in the distribution. Go back and try to get your ping to work. Application developers who wish to use a language other than Python can implement clients that speak the protocol. Later, use those missions to autonomously navigate your planned routes. We use the term E-Stop below and in our functions as shorthand for Motor Power Authority. Use Spot to remotely search and scan hazardous environments or disaster areas. To gain control of the robot, a client needs to acquire a Lease. If it installed correctly, you'll see a window like this: Open the image directory with File > Open Dir. These documents explain the key abstractions used by the Spot API. Spot API uses gRPC as its underlying RPC (Remote Procedure Call) transport. Tip: A good rule of thumb if you're unsure about a label is to ask, "what do I want the network to do?". This is the language the robot speaks. Learn how Cognite integrates with Spot to process visuals on AkerBP's sites. project. Use 360 perception to map terrain and avoid obstacles as they appear. This is a more detailed example built on top of the Python API which lets you interactively control Spot using the keyboard on your development machine. For example, in one of my images I threw the dog-toy. For a robot with an arm, you will also take ownership of the sub-resources full-arm, arm, and gripper when acquiring the body lease. Create autonomous routes or drive the robot to remotely perform inspections in electrified or radiation dense areas. First a stand command is issued. Process critical data in real-time and trigger actions using intelligent models created by our AI partners. To express this information, Spot uses frames to represent objects and locations (e.g.
NOTE: the lease_owner field is empty since no one has acquired the body lease. To verify your packages work correctly with Spot, you need: A Spot robot on the same version as your packages.
