Lab 4: HTTP Access from a SPOT

Background Reading:

SunSPOT Developer's Guide, pages 41-42: These two pages of the Developer's Guide demonstrate how to open and use HTTP connections on SPOTs. This lab is based on the code provided here, so it is useful to read this beforehand.

Introduction to fetching files with HTTP in Java: http://www.davidreilly.com/java/java_network_programming/. Read all of section 2: “HTTP Questions”. This will introduce you to using HTTP connections with Java's URL and URLConnection objects. It will also provide you with some good-to-know details about working with HTTP in Java and things you might encounter.

Prelab Questions:

What are the steps and classes used to set up an HTTP connection and receive a file in normal Java networking and with a SPOT?
When reading characters from an InputStream, how does one know when to stop (no more chars are available to be read)?

Introduction:

You've seen how to perform radio communication between SPOTs. Now we see an example of an application opening and using HTTP connections. The program that we create will grab an HTML file from a web server with the HTTP protocol. We will then have the SPOT parse the page and flash its LED's whenever certain tags are encountered. Obviously SPOTs do not have direct connections to the Internet. To use HTTP, a basestation must be running from a PC with access to the Internet, and the Socket Proxy program must be started.

Using HTTP Connections with a SPOT: HtmlRetriever

Start new project. First let's go ahead and start a new project for our HTTP application. This time we won't be starting from any preexisting project's code, so go to File->New Project and choose for your project to be a “Sun SPOT Application.” For the project name, I chose HtmlRetriever, but if you prefer another name, you are welcome to use that one instead. Netbeans will automatically use the project name for the name of the project folder unless you tell it to do otherwise. You can place the project within the Demos directory if you want, but it might be a better idea to make a new folder, calling it MyProjects or [your name]Projects or something along those lines, and place it in the main “SunSPOT” directory (one level up from the “Demos” folder). You will then have a place to store all your future projects as well (you might want to move your RadioCount project there, too).

Review the page retrieval code. You should have already read the brief section in the SPOT Developer's Guide about opening and using HTTP connections. Provided (page 41) is some example code that will open a connection, read an HTML page and display it, and then close the connection. For convenience, this code is pasted below. Notice how similar establishing our HTTP connection is to how we set up our Radiogram connection in the last lab (and how using the Radiostream protocol works as well). To get our connection we call Connector.open() just like before; the only difference is that this time, we supply a URL string as the argument and assign the returned connection as an HttpConnection. We then use connection.openInputStream() to set up an InputStream, from which we will read in our HTML file one character at a time. As the characters are read from the InputStream, we place them in a standard StringBuffer object (“ch”, our temp variable that holds each character when it is read, is declared as an int, but the value is cast to a char when it is appended to the buffer). Once the InputStream is empty (in.read()<= 0), the program prints out the contents of the buffer, which will display the same HTML source that one's browser receives when pointed to http://www.sunspotworld.com/. Lastly, the InputStream and HttpConnection are both closed.

HttpConnection connection = (HttpConnection)Connector.open("http://www.sunspotworld.com/");

connection.setRequestProperty("Connection", "close");

InputStream in = connection.openInputStream();

StringBuffer buf = new StringBuffer();

int ch;

while ((ch = in.read()) > 0) {

buf.append((char)ch);

}

System.out.println(buf.toString());

in.close();

connection.close();

3. Prepare the startApp() method. Now open up the source file for your new project that Netbeans has provided you, called “StartApplication.java.” It would make sense to rename this file to coincide with the name of your project, but is not necessary. If you wish to rename it, right-click on the source file and select refactoring->rename. This will bring up a dialog where you can change the name of the file (and the class inside). After you enter the desired name, click “next.” The output window at the bottom of the screen will now display a refactoring tree. Click the “Do Refactoring” button, and the file and class will be renamed. Inside the source file, you will notice that Netbeans has taken care of most of the necessary imports for you, and has given you three methods. The only method that has any code in it so far is startApp(); you have probably already guessed that this is the starting point of all Sun SPOT applications, similar to main() in normal Java programs. For the HtmlRetriever, you can either put the code that establishes the connection, grabs the page, and parses it (provided later) directly into startApp() or create another method that is called from startApp(). The latter approach is usually preferred, since it is considered cleaner to keep your startApp() method as short as possible, and provide important code pieces with their own methods. Regardless of what you choose to do, we need to clean up the contents of startApp() that we have been given. You can get rid of everything that Netbeans put in startApp(). You will notice that one line Netbeans put in for you was this:

private ITriColor leds[] = EdemoBoard.getInstance().getLEDs();

We want to keep this line, however, we want leds to be a class field and not a local variable (unless you want to paste all your code directly into startApp() and modify flashLights() to take an additional argument, in which case you would leave the line as it was); add this line inside the class, but outside of any method:

private ITriColorLED leds[] = EdemoBoard.getInstance().getLEDs();

One last thing you will need to do as preparation is to add the following import:

import com.sun.spot.sensorboard.peripheral.LEDColor;

4. Add the flashLights() method. After our application reads in a web page it will parse the HTML file and flash its LEDs when we tell it to (when certain tags are encountered). We need to provide a method to do this, so here it is:

protected void flashLights(int num, LEDColor color) {

for(int i=0; i < 8; i++) {

leds[i].setColor(color);

}

for(int i=0; i < num; i++) {

for(int j=0; j < 8; j++) {

leds[j].setOn();

}

Utils.sleep(500);

for(int k=0; k <8; k++) {

leds[k].setOff();

}

Utils.sleep(500);

}

It should be pretty clear what is happening here. First we set the color of the LEDs (all eight of them), based on the “color” parameter. Then we turn each LED on, wait a little bit, and then turn them off. One tells flashLights() how many times to flash the LED's with the “num” argument.

5. Add the rest of the code. Now all we have to do is add the code that opens an HTTP connection, uses an InputStream to grab the page and place it in a StringBuffer, and finally goes through the buffer and flashes the lights green every time an opening or closing script tag (“</script>” or “<script”) is encountered and blue whenever an image occurs (“<img “). Much of the code is identical to the Developer's Guide example. Here we use a second StringBuffer object to hold the current word as we go through the page (stored in “buf”) and check for our target tags whenever whitespace characters (a newline or a space) occur. This code first has the lights flash orange right before it starts parsing the web page, then flashes three times when it finds either of our two target strings, and flashes the lights red once after finding a tag to separate different occurrences of the same tag. Place this code either directly in startApp() or in another method that you have startApp() call. If you leave it as is, this code will only parse the page once; if you want it to happen more than once, you can placing the parsing code (the “for” loop) inside a new loop and do it as many times as you wish.

HttpConnection connection =

(HttpConnection)Connector.open("http://www.sunspotworld.com/");

connection.setRequestProperty("Connection", "close");

InputStream in = connection.openInputStream();

StringBuffer buf = new StringBuffer();

StringBuffer word = new StringBuffer();

flashLights(2, LEDColor.ORANGE);

int ch;

while ((ch = in.read()) > 0) {

buf.append((char)ch);

}

for(int i=0; i < buf.length(); i++) {

char c = buf.charAt(i);

if(c == ' ' || c == '\n') {

int flashes = 3;

if (word.toString().equals("</script>") || word.toString().equals("<script")) {

flashLights(flashes, LEDColor.GREEN);

flashLights(1, LEDColor.RED);

} else if (word.toString().equals("<img")) {

flashLights(flashes, LEDColor.BLUE);

flashLights(1, LEDColor.RED);

}

word.delete(0,word.length());

} else {

word.append((char) c);

}

in.close();

connection.close();

6. Start the Socket Proxy application. Now its time to plug in the basestation and start the Socket Proxy program. Once your basestation is connected to the PC, start a shared basestation with the SPOT Manager. Then, open up a command line, navigate to any project folder (C:\Program Files\Sun\SunSPOT\Demos\BounceDemo\BounceDemo-OnSPOT will do, or even better, use the folder of your current project), and run the command “ant socket-proxy.” A command line can be opened from the desktop shortcut we have placed for you. Once Socket Proxy is running, you should be ready to deploy and test your application.

7. Deploy and test the application. Its time to check out what we've done. Deploy your new application to a SPOT and see what happens. You should see your SPOT flash its lights blue and green a number of times in sets of four flashes. As of this writing, there are three places where </script> tags occur on www.sunspotworld.com, and many more occurrences of “<img .” Open up a web browser and go to view->source (or something similar, depending on your browser); you should be able to determine which tag is making the SPOT flash its lights each time.

Challenges:

1. Add functionality to recognize more tags (or other frequently occurring strings), and assign a new color to each. Do this enough, and you will get quite a colorful display. Try having your SPOT access different sites and see how the results change. You could also have your SPOT parse several pages one after another on one run (make the URL string an argument to your method and call it several times with different values in startApp()).

2. Have your program use the Radiogram protocol to send out messages to another SPOT when special strings encountered, sending it the arguments to flashLights(). You can make both SPOTs flash their LEDs on each tag, only have the remote SPOT (the one that is not parsing the page) do the flashing, or alternate which SPOT is to flash. You will need to create a new application for the second SPOT that includes the flashLights() method and allows the SPOT to receive Radiograms that contain the arguments to flashLights(). Come up with some ideas for utilizing even more SPOTs, or to make sure your SPOTs only act on communication from each other, and not SPOTs programmed by other groups (as in last lab's challenge).

Conclusion:

The goal of this lab was to demonstrate how SPOTs can interact with web sites over HTTP connections. You should be getting more comfortable writing SPOT applications and starting to become familiar with the code behind common SPOT tasks, like working with the LEDs and opening and using connections for various protocols.