Robot Arm Project Briefing

Build an SO-101 arm that sorts M&M's by color and follows typed commands. Then send a 60–90 second demo video to Standard Bots by Jan 24.

  1. You type a command“put three blue ones in the cup”
  2. A language model writes a planmove · blue · 3 · cup
  3. Your code checks the planagainst what the camera sees
  4. The arm actspicks the candies the detector found

1Order

Nov 12–15Done when everything is ordered

  1. 1Order

    Order the six servos first

    Feetech STS3215, 7.4 V, 1/345 gear (C001), $83.34 for six. All six must match; a wrong gear ratio makes a joint weak. They're the slowest to arrive and every arm step waits on them, so pick a seller that ships domestically. Overseas shipping pushes arm work back up to two weeks.

  2. 2Order

    Order everything else

    ItemQtyCost
    Waveshare Bus Servo AdapterMotor control board1$10.60
    Power supply5 V, 4 A or more, 5.5 × 2.1 mm barrel plug1$10
    USB-C data cablesLong enough to reach the laptop2-pack$7
    Table clampsHold the arm's base to the table2$5
    Screwdriver setMust include Phillips #0 and #11$6
    USB webcamThe model the printed overhead mount fits1~$30
    LED desk lampSteady, even light1~$20
    PLA+ filamentArm, camera mount, fingertips, tray1 kg~$20
    M2 and M3 screw assortmentSpares for the ones you'll lose1 box~$10
    Silicone grip padsStop the candy shell slipping1 pack~$8
    Plain M&M'sA bag just for the robot1 large~$5
    White matte boardPlain background under the camera1~$5
    With the servos$219.94

    Arm parts follow the official SO-101 parts list. Other prices are estimates. Any short, wide kitchen cup works as the target cup.

2While parts ship

Nov 16–29Done when the arm is printed, LeRobot is installed and color thresholds find candies

  1. 3Build

    Print the arm on the P2S

    Lego test gauge, then Ender_Follower_SO101.stl at 0.2 mm layers, 15% infill and tree supports, then the 3 webcam-mount parts. Block supports in the screw holes in Bambu Studio or the screws won't go in. The arm plate prints overnight. Files and settings are in the SO-101 repo.

  2. 4Build

    Install Python and LeRobot

    In a fresh environment used only for LeRobot: pip install 'lerobot[feetech]'. Mixed packages cause most install errors. Follow the install guide.

  3. 5Build

    Start the GitHub repo and a build log

    Log every session: what you did, what broke, the fix. It becomes the README.

  4. 6Learn

    Learn Python, 30 minutes a day until the demo

    Automate the Boring Stuff with Python. Write all the project code yourself, because the job tests Python written without AI help.

  5. 7Learn

    Watch the SO-101 assembly videos end to end

    Watch them before the servos arrive, so assembly week is only doing. Official guide and videos, plus a builder's write-up for the gaps.

  6. 8Build

    Set up the camera scene

    White board on the table, lamp fixed in place, webcam on its printed mount pointing straight down. Shifting light breaks color detection.

  7. 9Build

    Find candies by color, no ML

    An OpenCV script that finds M&M's with color thresholds. It's your baseline, and it shows how a camera sees color.

3Arm and detector

Nov 30–Dec 13Needs the servosDone when your code moves the arm and YOLO labels candies live

  1. 10Build

    Give each motor its ID, before assembly

    Run lerobot-find-port and set both Waveshare jumpers to B (USB). Then, one motor at a time: lerobot-setup-motors --robot.type=so101_follower --robot.port=<port>. The gripper is ID 6 and the base is ID 1; tape-label each. If it fails, check the cable, the power and the jumpers, then retry that motor.

  2. 11Build

    Assemble the arm

    Clean the supports off first. Build joints 1 to 5, then the gripper, following the official videos. M2 × 6 mm screws go into motors, M3 × 6 mm hold parts and horns.

  3. 12Build

    Wire it, clamp it, get it checked

    Daisy-chain the 3-pin cables, plug motor 1 into the board, mount the board on the base and clamp the arm to the table. Power off whenever you plug a motor cable, and use the 5 V supply only; 12 V destroys these servos. Have Connor check the wiring before the first power-on.

  4. 13Learn

    Learn how calibration works

    NVIDIA: Calibrating the SO-101.

  5. 14Build

    Calibrate

    lerobot-calibrate --robot.type=so101_follower --robot.port=<port> --robot.id=sorter_arm. Set each joint to mid-range, then move it through its full range. A rushed calibration is why picks miss later.

  6. 15Build

    Write your first script

    Connect, print all six joint positions, move one joint slowly. Add slow speeds, joint limits and a stop before anything else.

  7. 16Build

    Teach a pose by hand

    Torque off, move the arm, record the joint positions, torque on, replay. Film it for the progress video.

  8. 17Learn

    Learn how image detectors work

    3Blue1Brown: Neural Networks, chapters 1 and 2, then “YOLO algorithm for object detection explained”.

  9. 18Build

    Photograph and label the candies

    100 to 150 webcam photos: candies scattered, touching, overlapping, in slightly different light. Box and color-label every candy in Roboflow.

  10. 19Build

    Train YOLO and run it live

    Train in Google Colab and run it on the live webcam. Use Roboflow's YOLOv8 tutorial and the Ultralytics docs.

  11. 20Ship

    Dec 13: send a progress video and apply

    Show the hand-taught replay and live detection. If the posting disappears sooner, apply that day with whatever works.

4Gripper and picking

Dec 14–Jan 3Spans Christmas and New YearDone when the arm clears a scattered board into the right bins

  1. 21Build

    Compare three ways to count candy

    Color thresholds, YOLO, and a VLM asked “how many of each color?”, all on the same photos. Write down where each one fails: touching candies, shadows, brown vs. orange. It's a strong interview story.

  2. 22Learn

    Learn camera-to-robot coordinates

    Automatic Addison: convert camera pixels to robot coordinates.

  3. 23Build

    Calibrate pixels to the table

    Tape a printed grid under the camera and work out pixels per centimeter. Lock the camera mount; if it moves, recalibrate.

  4. 24Learn

    Learn Fusion 360 basics

    Autodesk's free beginner tutorials. Model a simple bracket before the fingertip.

  5. 25Build

    Design the fingertips

    An M&M is about 1.3 cm across. Modify the stock SO-101 jaw to cup it, and add grip pads. Expect three versions. Flat tips and fast approaches make candy roll away.

  6. 26Build

    Print a six-bin sorting tray

    One bin per color, with a fixed spot on the table.

  7. 27Build

    Pick from fixed spots (version 1)

    Teach a few spots by hand. The camera reads each candy's color and the arm drops it in the right bin.

  8. 28Build

    Pick from anywhere (version 2)

    Convert camera position to table position to joint angles (inverse kinematics). Show a live count by color and stop when the board is clear. If the holidays eat the time, demo version 1.

5Talk to it

Jan 4–17Done when every command works and bad ones get a reason instead of a move

  1. 29Learn

    Learn how language and vision models work

    Hugging Face: Vision Language Models Explained. The Claude API docs: getting started, sending images, fixed-format answers. Anthropic's prompt engineering guide.

  2. 30Order

    Buy about $5 of API credit

    For the Claude API. A few dollars covers the whole project.

  3. 31Build

    Define the commands and the plan format

    Commands: “sort everything,” “put the red ones in the cup,” “put three blue ones in the cup,” “how many green ones are left?” Plan: action (move or count), color, count, destination (cup or bin).

  4. 32Build

    Turn commands into plans

    Send each command to the model with instructions to answer only in that format. Print every plan before anything moves.

  5. 33Build

    Check every plan before the arm moves

    Refuse unknown colors or destinations, or more candies than the camera sees, and say why. The model never drives the motors directly. Test with “put the purple ones in the cup” and “put 50 in the cup.”

  6. 34Build

    Connect plans to the arm

    Approved plans run the version 2 pick code. Answer “how many green ones are left?” from the detector's count, then compare a VLM's answer.

6Ship

Jan 18–24Done when the video is with your Standard Bots contacts

  1. 35Ship

    Write the README

    What it does, how it works, the wiring diagram, what failed and how you fixed it. Build it from the log.

  2. 36Ship

    Record the 60–90 second demo

    Typed commands visible on screen: detection boxes, “put three blue ones in the cup,” “sort the rest” with the running count, “how many green ones are left?”

  3. 37Ship

    Run it for Dad like a customer job

    He types the commands. Then write one page on how the same cell could pull hardware kits at Forest Iron Works.

  4. 38Ship

    Do a mock interview, then send it

    Connor runs a mock technical interview against the job posting. Then send the video to your Standard Bots contacts and add it to the application.