However, I have never actually explained, in a concise way, what deep learning is, so here we go.
Deep learning is a technique for teaching a computer how to make predictions based on a set of inputs.
Input Data —–> Deep Learning Algorithm (i.e. Process) —–> Output Data
To make predictions (i.e. the “Process” part of the line above), deep learning uses deep neural networks. A deep neural network is a computer-based, simplified representation of neurons in the brain. It is computer science’s attempt to get a computer to process information just like real neurons in our brains do.
Deep neural networks are well suited for complex applications like computer vision, natural language processing, and machine translation where you want to draw useful information from nonlinear and unstructured data such as images, audio, or text.
In this tutorial, we will program our robot to perform a pick and place task.
Pick and place involves picking up an object in one location and placing it in another.
In this case, we will have our do-it-yourself SCARA robotic arm pick up a hex nut in one location and drop it off in another. Take a look at the video below of what you will build.
Real-World Applications
SCARA robots have a number of real-world applications:
Manufacturing: Picking up items off a conveyor belt and dropping them off on another conveyor belt.
Logistics: Picking up an item and placing it in a bin.
Factory: Tightening screws.
You can see an image here of a SCARA robot packaging cookies into bins.
Prerequisites
You completed this tutorial to build a two degree of freedom robotic arm.
You Will Need
Assuming you have gone through the prerequisites and acquired all the components in there, you will also need the following components to complete this project (#ad):
Let’s test it. The linear actuator has the following specifications:
Input voltage: 5-7.4V
Stall current: 650mA
Grab the 6 channel digital servo tester.
Connect the positive red lead to the positive terminal of the power supply.
Connect the negative black lead to the negative terminal of the power supply.
Connect the linear actuator to the digital servo tester so that the:
Black lead connects to (-)
Red lead connects to (+)
White lead connects to (S), which means signal
Now turn on the power supply. Set it to 6.0V with a current limit of 0.50A (ie. 500mA).
Turn the corresponding knob on the digital servo tester, and watch the linear actuator go back and forth.
Control the Linear Actuator Using Arduino
Now we want to write code so that we can automatically control the lowering and raising of the linear actuator. We will write code that makes the linear actuator go back and forth repeatedly.
Open your Arduino IDE, and type the following code. Save the file as control_linear_actuator.ino.
/*
Program: Make a linear actuator go back and forth automatically
File: control_linear_actuator.ino
Description: This program makes the Actuonix L16 Actuator 100mm 150:1 6V RC Control
go back and forth.
Author: Addison Sears-Collins
Website: https://automaticaddison.com
Date: October 5, 2020
*/
#include <VarSpeedServo.h>
//////////// Linear Actuator Variables ///////////////
// Create the object.
VarSpeedServo linearActuator;
// Attach motor to digital pin on the Arduino
const int linearActuatorPin = 12;
// Set the home and extended positions of the linear actuator
const int linearActuatorHomePos = 0;
const int linearActuatorExtPos = 180;
// Stores the current position of the linear actuator
int linearActuatorPos = 0;
// The amount of time in milliseconds alloted to enable the linear actuator to move into position
// The more delay given, the more the linear actuator extends. 100ms, for example, will result
// in full extension of the linear actuator.
const int linearActuatorDelay = 50;
//////////// End Linear Actuator Variables ///////////////
void setup() {
// Attach the linear actuator to the pin
linearActuator.attach(linearActuatorPin);
}
void loop() {
// Linear actuator goes to extended position and then back to home position
moveLinearActuator();
}
void moveLinearActuator() {
for (linearActuatorPos = linearActuatorHomePos; linearActuatorPos <= linearActuatorExtPos; linearActuatorPos += 1) { // goes from home position to extended position
linearActuator.write(linearActuatorPos);
delay(linearActuatorDelay); // wait for the actuator to reach the position
}
for (linearActuatorPos = linearActuatorExtPos; linearActuatorPos >= linearActuatorHomePos; linearActuatorPos -= 1) { // goes from extended position to home position
linearActuator.write(linearActuatorPos);
delay(linearActuatorDelay); // wait for the actuator to reach the position
}
}
Turn on the power supply (6.0V with a current limit of 0.50A), and then plug in the Arduino to the 9V battery. Watch the linear actuator go back and forth.
So that you don’t damage your Arduino Mega, turn off the DC external power supply first before unplugging the Arduino.
Test the Electric Lifting Magnet
Now that we’ve tested the linear actuator, let’s test the electric lifting magnet.
Grab the electric lifting magnet.
Attach the positive (typically red) lead of the magnet to the red positive lead of the power supply.
Attach the other lead (typically blue or black) of the magnet to the black lead of the power supply.
Set the voltage to 5V (or 6V depending on what it says on your magnet) and the current limit to 0.50A (the current limit of this component is 0.68A).
Turn on the power supply.
Grab a small metal object like a screw or hex nut, and place it near the magnet.
You should see the hex nut move towards the magnet.
If you are using a 5V electric lifting magnet, you can power the magnet directly with the Arduino by placing the leads in the 5V and GND (ground) pins.
Control the Electric Lifting Magnet Using Arduino
Now let’s control the electric lifting magnet using Arduino.
Using a 6V Electric Lifting Magnet
If you have a 6V electric lifting magnet, follow the instructions in this section. Otherwise, if you’re using a 5V electric lifting magnet, proceed to the next section.
Grab the 5V Relay. This relay acts as a switch for the electric lifting magnet, enabling you to activate and deactivate the magnetism using Arduino.
There are 6 pins on the relay:
NO: Normally Open Terminal (peak load of DC28~30V/10A)
NC: Normally Closed Terminal
COM: Common Terminal
DC+: Connects to the 5V pin (power supply) of the Arduino
DC-: Connects to the GND (ground) pin of the Arduino
IN: Control pin that receives either a HIGH (3-5V) or LOW (0-1.5V) signal from the Arduino (Pin 10 in our case)
If you decide to use the NO terminal instead of the NC terminal, when a HIGH signal is received on the IN pin from the Arduino, the relay switch will turn ON, allowing electric current to flow from C to NO. We’ll use this NO terminal in our work.
If you decide to use the NC terminal instead of the NO terminal, when a LOW signal is received on the IN pin from the Arduino, the relay switch will turn ON allowing electric current to flow from C to NC.
Here is the code to test the setup: control_electric_magnet_arduino_v1.ino.
/*
Program: Make an electric lifting magnet turn ON and OFF repeatedly.
File: control_electric_magnet_arduino_v1.ino
Description: This program makes an electric lifting magnet turn ON and OFF repeatedly.
Author: Addison Sears-Collins
Website: https://automaticaddison.com
Date: October 6, 2020
*/
const int magnetSignalPin = 10; // select the pin that will control the magnet
void setup() {
pinMode(magnetSignalPin, OUTPUT); // sets digital pin 10 as output
}
void loop() {
activateMagnet();
delay(10000); // waits for 10 seconds
deactivateMagnet();
delay(3000); // wait for 3 seconds
}
void activateMagnet() {
digitalWrite(magnetSignalPin, HIGH); // sets the digital pin ON
}
void deactivateMagnet() {
digitalWrite(magnetSignalPin, LOW); // sets the digital pin OFF
}
For your Arduino, you’ll need to upload the same code from the previous section: control_electric_magnet_arduino_v1.ino.
Your magnet will activate for 10 seconds and then deactivate for 3 seconds.
Connect the Electric Lifting Magnet to the Linear Actuator
Now that we’ve tested the electric lifting magnet, let’s connect it to the end of the linear actuator. I placed a 3mm diameter screw in one side of the magnet so that it can more easily attach to the end of the linear actuator.
Perform Pick and Place Using Arduino
Before we dive into the pick and place task, let’s see how to perform pick and place using Arduino and our robotic arm. Check out this tutorial for the details on the math.
What we want to do is:
Place a hex nut at a predetermined x and y coordinate. This is the pick location.
Move the end effector to the pick location.
Activate the electric lifting magnet.
Lower the linear actuator.
Pick up the hex nut.
Raise the linear actuator.
Move the end effector to the place location.
Deactivate the electric lifting magnet.
Place the hex nut in the toy trash can by lowering the actuator.
Return to the home position (both servos at 0 degrees).
To get started, measure the link lengths, a1, a2, a3, and a4. Use the kinematic diagram below as your guide. You’ll need to use a ruler for this.
a1 is measured vertically from the board’s surface to the top of Link 2.
a2 is measured horizontally from the middle of the first servo (where the central screw is located on Joint 1) to the middle of the second servo (where the central screw on Joint 2 is located).
a3is measured vertically from the top of Link 2 to the top of Link 4.
a4 is measured horizontally from the middle of the second servo to the middle of the electric lifting magnet.
Place a metal object (e.g. hex nut) at position x = 4.0, y = 10.0 on the board.
Place the toy trash cash at position x = -4.0, y = 10.0.
Set the power supply to 6V with a 0.60A current limit.
Here is the code for the Arduino: pick_and_place_no_vision.ino.
/*
Program: Perform a pick and place task with a SCARA-style robotic
arm using inverse kinematics (graphical method)
File: pick_and_place_no_vision_v1.ino
Description: This program performs a pick and place task with a SCARA-style robotic arm.
No computer vision is used.
Author: Addison Sears-Collins
Website: https://automaticaddison.com
Date: October 9, 2020
*/
#include <VarSpeedServo.h>
//////////// Servo Motor Variables ///////////////
// Define the number of servos
#define SERVOS 2
// Create the servo objects.
VarSpeedServo myservo[SERVOS];
// Speed of the servo motors
// Speed=1: Slowest
// Speed=255: Fastest.
const int servoSpeed = 50;
// Attach servos to digital pins on the Arduino
int servoPins[SERVOS] = {3,5};
//////////// End Servo Motor Variables ///////////////
//////////// Linear Actuator Variables ///////////////
// Create the linear servo object.
VarSpeedServo linearActuator;
// Attach motor to digital pin on the Arduino
const int linearActuatorPin = 12;
// Set the home position of the linear actuator
const int linearActuatorHomePos = 0;
// Stores the current position of the linear actuator
int linearActuatorPos = 0;
// The amount of time in milliseconds alloted to enable the linear actuator to move into position
// The more delay given, the more the linear actuator extends. 100ms, for example, will result
// in full extension of the linear actuator. Tweak this input here until you get desired results.
const int linearActuatorDelay = 45;
//////////// End Linear Actuator Variables ///////////////
// Link lengths in centimeters
const double a1 = 4.85;
const double a2 = 5.80;
const double a3 = 5.15;
const double a4 = 6.0;
// Select the pin that will control the magnet
const int magnetSignalPin = 10;
void setup() {
// Move servos to the home position of 0 degrees
setupServos();
// Set magnet pin
pinMode(magnetSignalPin, OUTPUT);
// Attach the linear actuator to the digital pin of the Arduino
linearActuator.attach(linearActuatorPin);
linearActuator.write(0);
delay(1000);
}
void loop() {
// The pick location in centimeters in the
// robot's base frame (pick up the object from here)
double xPosPick = 4.0;
double yPosPick = 10.0;
// The place location in centimeters in the
// robot's base frame (place the object here)
double xPosPlace = -4.0;
double yPosPlace = 10.0;
// Pick up an object from one location, and place it in another
pickAndPlace(xPosPick, yPosPick, xPosPlace, yPosPlace);
while(1) {
// Return home
returnHome();
}
}
/* This method converts the desired angle for Servo 1 into a control angle
* for Servo 1. It assumes that the 0 degree position on the kinematic
* diagram for Servo 1 is actually 90 degrees on the actual servo.
* The angle range for Servo 1 on the kinematic diagram is
* -90 to 90 degrees, with 0 degrees being the center position.
* The actual servo range for the physical motor
* is 0 to 180 degrees. We convert the desired angle
* to a value within that range.
*/
int calcServo1Angle (int inputAngle) {
int result;
result = map(inputAngle, -90, 90, 0, 180);
return result;
}
/* Move servos to the home position */
void setupServos() {
// Attach the servos to the servo object
// attach(pin, min, max ) - Attaches to a pin
// setting min and max values in microseconds
// default min is 544, max is 2400
// Alter these numbers until both servos have a
// 180 degree range.
myservo[0].attach(servoPins[0], 544, 2475);
myservo[1].attach(servoPins[1], 500, 2475);
// Set initial servo positions to home position
myservo[0].write(0, servoSpeed, true);
myservo[1].write(calcServo1Angle(0), servoSpeed, true);
// Wait to let servos get into position
delay(3000);
}
/* Move arm to the x and y position indicated by the parameters */
void moveArm(double xPos, double yPos) {
// Define the inverse kinematics variables
double r1 = 0.0;
double phi_1 = 0.0;
double phi_2 = 0.0;
double phi_3 = 0.0;
double theta_1 = 0.0;
double theta_2 = 0.0;
/* Calculate the inverse kinematics*/
// (1) r1 = square_root((x_0_2)^2 + (y_0_2)^2)
// Value is in centimeters
r1 = sqrt((xPos * xPos) + (yPos * yPos));
// (2) ϕ1 = arccos((a4^2 - r1^2 - a2^2)/(-2*r1*a2))
// The returned value is in the range [0, pi] radians.
phi_1 = acos(((a4 * a4) - (r1 * r1) - (a2 * a2))/(-2.0 * r1 * a2));
// (3) ϕ2 = arctan((y_0_2) / (x_0_2))
// The atan2() function computes the principal value of the
// arc tangent of y/x, using the signs of both arguments to
// determine the quadrant of the return value.
// The returned value is in the range [-pi, +pi] radians.
phi_2 = atan2(yPos, xPos);
// (4) θ1 = ϕ1 - ϕ2
// Value is in radians
theta_1 = phi_2 - phi_1;
// (5) ϕ3 = arccos((r1^2 - a2^2 - a4^2)/(-2*a2*a4))
// Value is in radians
phi_3 = acos(((r1 * r1) - (a2 * a2) - (a4 * a4))/(-2.0 * a2 * a4));
//(6) θ2 = 180° - ϕ3
theta_2 = PI - phi_3;
/* Convert the joint (servo) angles from radians to degrees*/
theta_1 = theta_1 * RAD_TO_DEG; // Joint 1
theta_2 = theta_2 * RAD_TO_DEG; // Joint 2
/* Move the end effector to the desired x and y position */
// Set Joint 1 (Servo 0) angle
myservo[0].write(theta_1, servoSpeed, true);
// Set Joint 2 (Servo 1) angle
myservo[1].write(calcServo1Angle(theta_2), servoSpeed, true);
// Wait
delay(1000);
}
/* Activate the electric lifting magnet */
void activateMagnet() {
digitalWrite(magnetSignalPin, HIGH); // sets the digital pin ON
}
/* Deactivate the electric lifting magnet */
void deactivateMagnet() {
digitalWrite(magnetSignalPin, LOW); // sets the digital pin OFF
}
/* Move linear actuator down and then up */
void moveLinearActuator(int extension) {
// goes from home position to extended position
for (linearActuatorPos = linearActuatorHomePos; linearActuatorPos <= extension; linearActuatorPos += 1) {
linearActuator.write(linearActuatorPos);
delay(linearActuatorDelay); // wait for the actuator to reach the position
}
// goes from extended position to home position
for (linearActuatorPos = extension; linearActuatorPos >= linearActuatorHomePos; linearActuatorPos -= 1) {
linearActuator.write(linearActuatorPos);
delay(linearActuatorDelay * 2); // wait for the actuator to reach the position
}
}
/* Pick up an object from one location, and place it in another */
void pickAndPlace(double xPosPick, double yPosPick, double xPosPlace, double yPosPlace) {
// Move arm to the pick location
moveArm(xPosPick, yPosPick);
activateMagnet();
// Pick up the object
moveLinearActuator(180);
// Move arm to the place location
moveArm(xPosPlace, yPosPlace);
deactivateMagnet();
// Place the object (tweak this input here until you get desired results)
moveLinearActuator(120);
delay(1000);
}
/* Return to the 0 degree home position */
void returnHome() {
// Set servo positions to home position
myservo[0].write(0, servoSpeed, true);
myservo[1].write(calcServo1Angle(0), servoSpeed, true);
}
Future Challenge Project: Add Computer Vision to Automate Pick and Place
Up until now, we have placed an object at a predetermined location, and we hardcoded that location in our Arduino code. If you’re feeling really ambitious, what you can now do is integrate the Raspberry Pi camera so you can determine the object’s location automatically.
What you’ll want to do is:
Turn on the Raspberry Pi camera.
Place a hex nut on the board, somewhere where the camera can see it, and the robotic arm can reach it.
Determine the coordinates of the object using the Raspberry Pi camera. This is the target location.
Send the coordinates of that object (in the robot’s base frame) from Raspberry Pi to Arduino.
Move the end effector to the target location.
Activate the electric lifting magnet.
Lower the linear actuator.
Pick up the hex nut.
Raise the linear actuator.
Move the linear actuator to the toy trash can.
Lower the linear actuator.
Deactivate the electric lifting magnet.
Place the hex nut.
Raise the linear actuator.
Return to the home position
Pick and place complete!
You will need to tweak the code in this tutorial to get everything working.
That’s it for today. Keep building!
Disclosure (#ad): As an Amazon Associate, I earn from qualifying purchases.
Imagine we are the captain of a ship. We want the boat to maintain a heading of due north automatically, without any human intervention.
We go out and hire a smart robotics software engineer to write a control algorithm for the ship.
The variable that we want to control is the ship’s heading, and we want to make sure the heading maintains a “set point” of due north (e.g. 0 degrees).
Proportional Control
If the ship is heading slightly off course to the left, we want the ship to steer slightly to the right.
If the ship is heading slightly off course to the right, we want the ship to steer slightly to the left.
If the ship is heading strongly off course to the left, we want the ship to steer strongly to the right.
If the ship is heading strongly off course to the right, we want the ship to steer strongly to the left.
In short, the steering magnitude is directly proportional to the difference between the desired heading and the current heading. The further you are off course, the harder the ship steers.
Mathematically, proportional control is:
Control Output = Kp * (Desired – Actual)
Where:
Control Output is what you want to control (e.g. steering intensity, speed, etc.)
Kp is some non-negative constant
Desired = Set Point (e.g. the heading you want to achieve…i.e. 0 degrees north)
Actual = Process Variable (e.g. the heading you measured)
Error = Desired – Actual
The Proportional term is all about adjusting the output based on the present error.
Integral Control
Now let’s suppose that there is a crosswind. Even though the ship is adjusting the steering in proportion to the heading error, there are still differences between the actual and desired heading that are accumulating over time as a result of this ongoing crosswind.
The ship continues to drift off course even though proportional control is being applied.
The integral term looks for the residual error that is generated even after proportional control is applied. In this example, the residual error is caused by the wind, and it keeps adding up over time. The integral term seeks to get rid of this residual error by incorporating the historical cumulative value of the error. As the error decreases, the integral term will decrease.
Integral control looks for factors that keep pushing the ship off course and adjusts accordingly.
Mathematically, we now have to add the Integral term.
Control Output = Kp * Error + Ki * Sum of Errors Over Time
The Integral term is all about adjusting the output based on the past error.
Derivative Control
Let’s suppose that the ship is steering hard to the left in order to correct the heading. You don’t want the ship to steer so hard that momentum causes the ship to overshoot your desired heading.
What you want to do is have the ship slow down the steering as it approaches the heading.
The derivative term of PID control does this. It is used to slow down the heading correction as the measured heading approaches the desired heading. This derivative term helps the ship avoid overshooting.
Mathematically, we now have to add the Derivative term to get the final PID Control equation:
Control Output = Kp * Error + Ki * Sum of Errors Over Time + Kd * Rate of Change of the Error with Respect to Time
The Derivative term is all about adjusting the output based on the future (predicted) error. If the error is decreasing too quickly with respect to time, this term will reduce the control output.
Summary
PID is a control method that is made up of three terms:
Proportional gain that scales the control output based on the difference between the actual state of the system and the desired state of the system (i.e. the error).
Integral gain that scales the control output based on the accumulated error.
Derivative gain that scales the control output based on the rate of change of the error (i.e. how fast the error is changing).