Difference Between Supervised and Unsupervised Learning

In this post, I will explain the difference between supervised and unsupervised learning.

Table of Contents

What is Supervised Learning?

new_home_for_sale_4

Imagine you have a computer. The computer is really good at doing math and making complex calculations. You want to “train” your computer to predict the price for any home in the United States. So you search around the Internet and find a dataset. The dataset contains the following information  for 100,000 houses that sold during the last 30 days in various cities across the United States:

  1. Square footage
  2. Number of bathrooms
  3. Number of bedrooms
  4. Number of garages
  5. Year the house was constructed
  6. Average size of the house’s windows
  7. Sale price

Variables 1 through 6 above are the dataset’s features (also known as input variables, attributes, independent variables, etc.). Variable 7, the house sale price, is the output variable (or target variable) that we want our computer to get good at predicting. 

So the question is…how do we train our computer to be a good house price predictor?  We need to write a software program. The program needs to take as input the 100,000-house dataset that I mentioned earlier. This program needs to then find a mathematical relationship between variables 1-6 (features) and variable 7 (output variable). Once the program has found a relationship between the features (input) and the output, it can predict the sale price of a house it has never seen before.

1-ipo

Let’s take a look at an analogy. Supervised learning is like baking a cake. 

Suppose you had a cake machine that was able to cook many different types of cake from the same set of ingredients. All you have to do as the cake chef is to just throw the ingredients into the machine, and the cake machine will automatically make the cake.

The “features,” the inputs to the cake machine, are the following ingredients:

  1. Butter
  2. Sugar
  3. Vanilla Extract
  4. Flour
  5. Chocolate
  6. Eggs
  7. Salt

The output is the type of cake:

  1. Vanilla Cake
  2. Pound Cake
  3. Chocolate Cake
  4. Dark Chocolate Cake
cake_cakes_sweet_bake

Different amounts of each ingredient will produce different types of cake. How does the cake machine know what type of cake to produce given a set of ingredients? 

Fortunately, a machine learning engineer has written a software program (containing a supervised learning algorithm) that is running inside the cake machine. The program was pre-trained on a dataset containing 1 million cakes. Each entry (i.e. example or training instance) in this gigantic dataset contained two key pieces of data: 

  1. How much of each ingredient was used in the making of that given cake
  2. The type of cake that was produced

During the training phase of the program, the program found a fairly accurate mathematical relationship between the amount of each ingredient and the cake type. And now, when the cake machine is provided with a new set of ingredients by the chef, it automatically “knows” what type of cake to produce. Pretty cool huh!

2-ipo-2

What I described above is called supervised learning. It is called supervised learning because the input data (which the supervised learning algorithm used to train) is already labeled with the “correct” answers (e.g. the type of cake in our example above; or the sale price values for those 100,000 homes from the earlier example I presented in this post.). We supervised the learning algorithm by “telling” it what the output (cake type) should be for 1 million different sets of input values (ingredients). 

The fundamental idea of a supervised learning algorithm is to learn a mathematical relationship between inputs and outputs so that it can predict the output value given an entirely new set of input values. 

Let’s take a look at a common supervised learning algorithm: linear regression. The goal of linear regression is to find a line that best fits the relationship between input and output. For example, the learning algorithm for linear regression could be trained on square footage and sale price data for 100,000 homes. It would learn the mathematical relationship (e.g. a straight line in the form y = mx + b) between square footage and the sale price of a home. 

3-square-footage

With this relationship (i.e. line of best fit) in hand, the algorithm can now easily predict the sale price of any home just by being provided with the home’s square footage value. 

For example, let’s say we wanted to find the price of a house that is 2000 ft2. We feed 2,000 ft2 into the algorithm. The algorithm predicts a sale price of $500,000.

4-price-square-footage

As you can imagine, before we make any predictions using a supervised learning algorithm, it is imperative to train it on a lot of data. Lots and lots of data. The more the merrier.

In the example above, to get that best fit line, we want to feed it with as many examples of houses as possible during training. The more data it has, the better its predictions will be when given new data. This, my friends, is supervised learning, and it is incredibly powerful. In fact, supervised learning is the bread and butter of most of the state-of-the-art machine learning techniques today, such as deep learning.

Now, let’s look at unsupervised learning.

Return to Table of Contents

What is Unsupervised Learning?

Let’s suppose you have the following dataset for a group of 13 people. For each person, we have the following features:

  • Height (in inches)
  • Hair Length (in inches)

Let’s plot the data to see what it looks like:

5-height

In contrast to supervised learning, in this case there is no output value that we are trying to predict (e.g. house sale price, cake type, etc.). All we have are features (inputs) with no corresponding output variables. In machine learning jargon, we say that the data points are unlabeled

So instead of trying to force the dataset to fit a straight line or some sort of predetermined mathematical model, we let an unsupervised learning algorithm find a pattern all on its own. And here is what we get:

6-hair-length-vs-height

Aha! It looks like the algorithm found some sort of pattern in the data. The data is clustered into two distinct groups. What these clusters mean, we do not know because the data points are unlabeled. However, what we do suspect given our prior knowledge of this dataset, is that the blue dots are males, and the red dots are females given the attributes are height and hair length. 

What I have described above is known as unsupervised learning. It is called unsupervised because the input dataset is unlabeled. There is no output variable we are trying to predict. There is no prior mathematical model we are trying to fit the data to. All we want to do is let the algorithm find some sort of structure or pattern in the data. We let the data speak for itself. 

Any time you are given a dataset and want to group similar data points into clusters, you’re going to want to use an unsupervised learning algorithm.

Return to Table of Contents

How to Connect Arduino to ROS

How do we connect ROS to an actual embedded system that operates in the real, physical world? I’ll show you how to do this now using Arduino

Arduino is a popular microcontroller for building electronics projects. A microcontroller is a bunch of circuits that do stuff, such as accepting data input, doing calculations, and producing output. With respect to robotics development, Arduino would be the “brain” of a robot.

Fortunately, ROS can integrate with Arduino. We will install some software that will enable your Arduino to be a bonafide ROS node that can do everything a normal node can do, such as publish and subscribe to ROS messages.

Here are the official steps for interfacing Arudino with ROS, but I’ll walk you through the process below.

Table of Contents

Directions

How to Install Arduino on Ubuntu Linux

First, let’s download the Arduino IDE (Linux 64 bit version) to our computer. I will follow these instructions for installing the Arduino IDE on Ubuntu. Go to this website, and download the software:

63-download-arduino-file

Save the file. It will be saved as tar.xz format to your Downloads folder.

Open a new terminal window.

Move to the Downloads folder (or wherever you saved the tar.xz file).

cd Downloads
64-change-to-downloads-directory

Run this command to extract the files (substitute FILENAME with the name of the file you just downloaded):

tar xvf <FILENAME>

In my case, I will run:

tar xvf arduino-1.8.10-linux64.tar.xz

You will see a bunch of file names print out to your screen.

If you type the dir command, you will see the new folder. Let’s move that folder to the home directory. You can cut and paste it into the home directory using the file manager (the file cabinet on the left of the screen. Go there, then go to the Downloads folder, cut the file and paste it into the Home directory.

65-downloads-dir
66-arduino-in-home-directory

Now open up a new terminal window and type:

cd arduino-1.8.10

Type dir to see what files are inside.

67-what-files-are-inside

To install the IDE, type:

sudo ./install.sh
68-install-ideJPG

Here is the output. You should see a “done” message. A desktop icon will also be present. You can activate it by clicking on it and allowing permissions at the prompt.

Now, get your Arduino and connect it to the USB port on your computer.

Start Arduino by going into a new terminal window and typing:

arduino

You might see an error message like this:

69-arduino-startupJPG

Failed to load module “canberra-gtk-module” … but already installed

To resolve that error, press CTRL+C and close the terminal window.

Open a new terminal window, and type:

sudo apt-get install libcanberra-gtk-module

Now open a new terminal window, and type:

arduino

Let’s see if everything works by trying to blink the light-emitting diode (LED) on your computer.

Go to File – > Examples -> 01.Basics, and choose Blink.

Try to upload that code to your Arduino by clicking the right arrow on the upper left of your screen.

70-right-arrow-to-uploadJPG

You should see an error about the “Serial port not selected”.

Close out of your Ubuntu Virtual Machine completely. Do not save the state.

Set Up the Serial Port for VirtualBox With Ubuntu

Assuming you are using Windows, go to your Device Manager. Search for that in your computer in the bottom left of your desktop.

Under Device Manager, you should see Ports (COM & LPT). Note that Arduino is port 3 (Make sure your Arduino board is connected to the USB port of your computer).

71-arduino-com-portJPG

Open your VirtualBox.

Click Settings and go down to Serial Ports.

72-settings-serial-portsJPG

Make sure your settings look exactly like this:

73-settings-after-serial-portJPG

Note that the little box next to “Enable Serial Port” is checked.

Side Note: Any time you unplug your Arduino….say perhaps after using it within Ubuntu Linux or if you shutdown your PC….be sure to disable the “Enable Serial Port” option before restarting Ubuntu Linux in your Virtual Box. Otherwise, your Ubuntu Linux session will NOT launch. I’ve made this mistake numerous times, and it is frustrating.

After you are done, click OK.

Restart the VirtualBox with Ubuntu.

Open a new terminal window and type:

ls -al /dev/ttyS0

Here is the output:

74-serial-port-outputJPG

Now we need to give the IDE permissions to access the device.

In a new terminal window, find out your username. Type:

whoami

Now type the following commands, replacing YOUR_USER_NAME with what you found above:

sudo usermod -a -G dialout YOUR_USER_NAME
sudo chmod 660 /dev/ttyS0
76-add-dialoutJPG

Reboot your machine:

sudo reboot

Open up a terminal window and launch Arduino by typing:

arduino

Go to Tools -> Port, and you should see /dev/ttyS0. This is your Arduino board that is connected to the USB port of your computer. Make sure /dev/ttyS0 is checked.

77-arduino-board-selectedJPG

Now open the Blink sketch again. Go to File – > Examples -> 01.Basics, and choose Blink.

Click the Upload button…the right arrow in the upper left of your screen.

The LED on your Arduino should be blinking! If it is not, go back to the beginning of this tutorial and follow the steps carefully.

To turn off the blinking light, open up a new sketch and upload it to your board. Go to File -> New.

Integrate Arduino With ROS

Now that we know Arduino is working, we need to integrate it with ROS. Here are the official instructions. I’ll walk you through the steps below.

Let’s install the necessary packages. 

Close Arduino. Then type the following commands in a new terminal window (these will take a while to download so be patient):

sudo apt-get install ros-melodic-rosserial-arduino
sudo apt-get install ros-melodic-rosserial

Open the IDE by typing arduino and go to File -> Preferences. Make a note of the Sketchbook location. Mine is:

/home/ros/Arduino

78-sketchbook-locationJPG

Open a new terminal window and go to the sketchbook location you noted above. I’ll type:

cd arduino
79-libraries-folderJPG

Type the dir command to see the list of folders.

Go to the libraries directory.

cd libraries

Within that directory, run the following command to build the Arduino library that will be used by ROS (don’t leave out that period that comes at the end of the command):

rosrun rosserial_arduino make_libraries.py .

Type the dir command to see the list of folders. You should now see the ros_lib library.

80-ros-lib-libraryJPG

Make sure the Arduino IDE is closed. Now open it again.

You should see some sample code. Now, let’s take a look at the Blink example.

Go to File -> Examples -> ros_lib

81-ros-lib-on-examplesJPG

Return to Table of Contents

How to Blink an LED (Light-Emitting Diode) Using ROS and Arduino

The Blink example is analogous to a “Hello World” program. Blinking an LED is the most basic thing we can do to make sure the hardware is working properly and that it accepts the software we are developing on our laptop. The goal of the Blink example is to toggle an LED on and off. 

In this example, Arduino is going to be considered a Subscriber node. It will subscribe to a topic called toggle_led. Publishing a message to that topic causes the LED to turn on. Publishing a message to the topic again causes the LED to turn off. 

Go to File -> Examples -> ros_lib and open the Blink sketch.

Now we need to upload the code to Arduino. Make sure your Arduino is plugged into the USB port on your computer.

Upload the code to your Arduino using the right arrow button in the upper left of your screen. When you upload the code, your Arduino should flicker a little bit.

Open a new terminal window and type:

roscore

In a new terminal window, launch the ROS serial server. This command is explained here on the ROS website. It is necessary to complete the integration between ROS and Arduino:

rosrun rosserial_python serial_node.py /dev/ttyS0
83-rosrun-rosserial-pythonJPG

Now let’s turn on the LED by publishing a single empty message to the /toggle_led topic. Open a new terminal window and type:

rostopic pub toggle_led std_msgs/Empty --once
84-publish-message-led-onJPG

The LED on the Arduino should turn on. Note the yellow light is on (my Arduino is inside a protective case).

2019-10-21-193316

Now press the Up arrow in the terminal and press ENTER to run this code again. You should see the LED turn off. You might also see a tiny yellow light blinking as well. Just ignore that one…you’re interested in the big yellow light that you’re able to turn off and on by publishing single messages to the /toggle_led topic.

85-publish-message-led-offJPG
2019-10-21-193535

Return to Table of Contents

That’s it! You have now seen how you can integrate Arduino with ROS. To turn off your Arduino, all you need to do is disconnect it.

How to Launch the TurtleBot3 Simulation With ROS

In this tutorial, we will launch a virtual robot called TurtleBot3. TurtleBot3 is a low-cost, personal robot kit with open-source software. You can read more about TurtleBot here at the ROS website.

Below is a demo of what you will create in this tutorial. You will get experience with SLAM (Simultaneous localization and mapping) and autonomous navigation.

turtlebot3-simulation

TurtleBot3 is designed to run using just ROS and Ubuntu. It is a popular robot for research and educational purposes.

48-turtlebotsJPG

Table of Contents

Directions

I’m assuming you have ROS installed and are using Linux. If you don’t have ROS installed, install ROS now.

Let’s install the TurtleBot3 simulator now.

Open a terminal window and install the dependent packages. Enter the following commands, one right after the other:

cd ~/catkin_ws/src/
git clone https://github.com/ROBOTIS-GIT/turtlebot3_msgs.git
git clone https://github.com/ROBOTIS-GIT/turtlebot3.git
cd ~/catkin_ws && catkin_make

TurtleBot3 has three models, Burger, Waffle, and Waffle Pi, so you have to set which model you want to use before you launch TurtleBot3. Type this command to open the bashrc file to add this setting:

gedit ~/.bashrc

Add this line at the bottom of the file:

export TURTLEBOT3_MODEL=burger
49-update-bash-settingsJPG

Save the file and close it.

Now reload .bashrc so that you do not have to log out and log back in.

source ~/.bashrc

Now, we need to download the TurtleBot3 simulation files.

cd ~/catkin_ws/src/
git clone https://github.com/ROBOTIS-GIT/turtlebot3_simulations.git
50-turtlebot-simulationsJPG
cd ~/catkin_ws && catkin_make
51-run-catkin-makeJPG

Return to Table of Contents

Simulate TurtleBot3 Using RViz

Now that we have the TurtleBot3 simulator installed, let’s launch the virtual robot using RViz. Type this command in your terminal window:

roslaunch turtlebot3_fake turtlebot3_fake.launch

If you want to move TurtleBot3 around the screen, open a new terminal window, and type the following command:

roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch

Click the terminal window and use the keys below to control the movement of your TurtleBot (e.g. press W key to move forward, X key to move backward and S to stop).

52-move-turtlebot-aroundJPG
52-turtlebot-rvizJPG
53-move-turtlebotJPG

And remember, use the keyboard to move the robot around.

54-how-to-control-turtlebotJPG

Press CTRL+C in all terminal windows.

Return to Table of Contents

Simulate TurtleBot3 Using Gazebo

Now let’s use Gazebo to do the TurtleBot3 simulation.

First, let’s launch TurtleBot3 in an empty environment. Type this command:

roslaunch turtlebot3_gazebo turtlebot3_empty_world.launch

Wait for Gazebo to load. It could take a while. Here is what your screen should look like:

55-gazebo-simulationJPG

Press CTRL+C and close out all windows.

Return to Table of Contents

How to Change the Simulation Environment for TurtleBot3

Let’s look at our TurtleBot3 in a different environment. This environment is often used for testing SLAM and navigation algorithms. Simultaneous localization and mapping (SLAM) concerns the problem of a robot building or updating a map of an unknown environment while simultaneously keeping track its location in that environment.

In a new terminal window type:

roslaunch turtlebot3_gazebo turtlebot3_world.launch
56-gazebo-slam-navigation-viewJPG
57-the-burgerJPG

Press CTRL+C and close out all windows.

We can also simulate TurtleBot3 inside a house. Type this command and wait a few minutes for the environment to load.

roslaunch turtlebot3_gazebo turtlebot3_house.launch
58-turtlebot-in-a-houseJPG

To move the TurtleBot with your keyboard, use this command in another terminal tab:

roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch

Press CTRL+C and close out all windows.

Return to Table of Contents

Autonomous Navigation and Obstacle Avoidance With TurtleBot3

Now let’s implement obstacle avoidance for the TurtleBot3 robot. The goal is to have TurtleBot3 autonomously navigate around a room and avoid colliding into objects.

Open a new terminal and type:

roslaunch turtlebot3_gazebo turtlebot3_world.launch

In another terminal window type:

roslaunch turtlebot3_gazebo turtlebot3_simulation.launch

You should see TurtleBot3 autonomously moving about the world and avoiding obstacles along the way.

59-autonomous-navigation-robotJPG

We can open RViz to visualize the LaserScan topic while TurtleBot3 is moving about in the world. In a new terminal tab type:

roslaunch turtlebot3_gazebo turtlebot3_gazebo_rviz.launch
60-open-rviz-visualize-laserscanJPG

Press CTRL+C and close out all windows.

Return to Table of Contents

Simulating SLAM With TurtleBot3

Let’s take a look at how we can simulate SLAM with TurtleBot3. As a refresher, Simultaneous localization and mapping (SLAM) concerns the problem of a robot building or updating a map of an unknown environment while simultaneously keeping track its location in that environment.

Install the SLAM module in a new terminal window.

sudo apt install ros-melodic-slam-gmapping

Start Gazebo in a new terminal window.

roslaunch turtlebot3_gazebo turtlebot3_world.launch

Start SLAM in a new terminal tab.

roslaunch turtlebot3_slam turtlebot3_slam.launch slam_methods:=gmapping

Start autonomous navigation in a new terminal tab:

roslaunch turtlebot3_gazebo turtlebot3_simulation.launch

Watch the robot create a map of the environment as it autonomously moves from place to place!

61-slam-using-turtlebot3JPG
62-slam-2-using-turtlebot3JPG
62-slam-3-using-turtlebot3JPG

And that is all there is to it.

When you’ve had enough, press CTRL+C and close out all windows.

Return to Table of Contents

That’s it for TurtleBot3. In case you want to try other commands and learn more, check out the official TurtleBot3 tutorials.