Parsing XML with SAX

There are a few ways to parse XML. The first you maybe familiar with is the DOM parser. The DOM is a tree based structure which makes up XML's structure. You can traverse the DOM to find a tag, attribute or value you are seeking. The other method I will be talking about is SAX or Simple API for XML.

Typically the difference between DOM parsing and SAX parsing is that SAX parsing reads the file as a stream, whereas DOM parsing constructs the complete structure in memory. The advantage of SAX parsing is on mobile devices with limited memory or slow connection speeds. It's also really good at parsing very large XML files that can be gigabytes in size. A disadvantage is that you cannot traverse or backtrack. SAX parsing is also a little more complicated.

In order to setup your program to work with SAX you need to understand a little bit about event handling and a quick big picture. When the stream starts reading in data, certain handlers that you setup to are called via SAX's callback methods that look for tags, attributes and values which will be called if they match the criterium that you specified. Handlers in SAX are basically method overrides that allow you to take control over what happens. There are 3 methods you typically override in SAX, the startElement method, the endElement method and the characters method. When you instantiate your Handler class and feed it your XML stream, startElement is called. You conditionally check the tag that is passed into your method. With each tag that is passed into startElement and matched to a conditional statement, you flip boolean flags that you setup to keep track of which element was traversed.

Once startElement is finished executing (the length of one tag) the characters method is called to extract the values or attributes out of that tag. Keep in mind, these methods are called by the SAX library, not you, so you are simply catching values and flipping booleans to determine if you are in a location you want to subscribe to. Inside characters you receive a String containing the values you are looking for. Typically this looks like the startElement and endElement methods, which the exception that the string is passed to an class through public mutator method that you could name ParsedData. ParsedData would contain fields such as Title, Author, ISBN for example.

Once the tag has been passed over, the endElement is called. This method does the same as startElement with the exception that the booleans you flipped to true in startElement are now flipped to false. The next element in then streamed through and the process continues.

A Diagram Representing SAX's Possible States

In my project WattDroid (my first Android application) I've implemented SAX handlers with documentation to get you started.

Some more information is located here.

WattDroid GUI Improvements and Settings Added

Recently I talked about adding a settings feature to allow any WattDepot server to be queried for data. This has now been implemented and the settings will be saved once set. Here is a screenshot of this feature:

As you can see the top field is where you type the URL of the WattDepot server and the bottom image is what will pop up if you don't have that set when you first start the application. If the URL in invalid, meaning it does not link to a WattDepot server, then you will get an error message and it will tell you to enter the correct URL.

Beyond those necessary additions, we started to style our application to make it more visually pleasing. Here is a screenshot of a temporary placeholder image. It will change once we go over a few more designs. We have added transparency to each listview item so the background is visible.

We also added a feature where if the menu button is pressed it displays our settings panel. This is useful if the user would like to make changes to the WattDepot URL and also houses a placeholders for future settings.

We still have some pending issues to resolve. These issues are creating test cases and a continuous integration system. We also would like to add locational awareness that will be used to sort the list by closest proximity.

While Android is quite a fun framework to work with, getting to learn the libraries, like any framework, can be quite a task; so I recommend spending a lot of time read others code, it has helped me a lot.

WattDroid with Hudson and Unit Testing

Over the last few days I have been studying all the options for Android Continuous Integration and Unit Testing and well there is a plethora of choices to review.

Here are a few that I have reviewed:

Ant Related:

http://code.google.com/p/autoandroid/wiki/AndroidAnt

Hudson Related:

http://www.facebook.com/note.php?note_id=499519075367

JUnit Related:

http://mylifewithandroid.blogspot.com/2008/11/junit-in-android.html

http://marakana.com/tutorials/android/junit-test-example.html

http://code.google.com/p/robotium

What I have found is that Android requires its own unit testing libraries to perform tests that use Android libraries. Normal JUnit testing will work, however, it will not replicate actual functions the user will encounter. For example, it will test that a method returns the correct value given input, but it will not test if the user presses on a certain button and a result is shown in an Activity.

The classes that deal with specific Android testing are:

ActivityUnitTestCase and ActivityInstrumentationTestCase2

(ActivityInstrumentationTestCase is now deprecated)

The android.jar file also contains normal JUnit tests. I'm in the middle of evaluating a tool called Robotium ( http://code.google.com/p/robotium/ ) as well and it provides a simpler way to test Android applications, but this is mainly geared at the blackbox perspective, otherwise the users actions and the applications responses.

Robotium offers some interesting features and members of the Android community also say it is a valuable tool to learn. Here are a few of its features:

• You can develop powerful test cases, with minimal knowledge of the application under test.
• The framework handles multiple Android activities automatically.
• Minimal time needed to write solid test cases.
• Readability of test cases is greatly improved, compared to standard instrumentation tests.
• Test cases are more robust due to the run-time binding to GUI components.
• Blazing fast test case execution.
• Integrates smoothly with Maven or Ant to run tests as part of continuous integration.
• The Syntax is quite easy to understand as you can see in these examples.

Some others have recommended EasyMock as well. I am starting to feel all these extras are becoming a distraction and will stick to learning the standard Android test case classes and look further into Hudson CI after that.

Back to WattDroid...

Currently we are working on GPS integration and theming out application. The book we have doesn't really talk much about these two topics so we ordered a more advanced book Pro Android 2. However, in the meantime I have been trying to decipher and wad my way through some of the documentation and decent progress.

Until next time...

Remy

WattDroid Beta

The first beta of WattDroid is ready to explore. It's bare bones right now, but soon there will be a lot more functionality. Currently the WattDroid application displays information about specific sources when you select the source from a list. For those of you who are wondering, "What's a Source?" well it's just a node or sensor that contains data about anything you desire. WattDroid simply finds the list of sources and displays the XML of that source as of now. We have a lot in store for this application.

Here are some screenshots to give you a good idea:

WattDroid has a Watson Icon, taken from WattDepot.



















WattDroid's old intro screen.
















About WattDroid




















Source List


















Source Info










The next features we have planned are:

Add functionality

• Incorporate GPS (locational awareness for closest dorm)
  
• Add ability for user to specify WattDepot source URL
• Add ability for user to specify refresh interval in settings

Incorporate Android Unit Testing

• Research Android Unit Testing
• Research Integration with Hudson
• Better coding standards
  

Improve the look and feel of the application

• Needs to be more graphically enticing for the target audience (freshman, sophomores)
• Develop Killer Icon (Merge Android with Watson)
• Work with Themes
• Look into transition effects
  

Make the interface more user-friendly

• Add settings when standard menu button is pressed

Check out the project and follow the User Guide or Developer Guide Wiki to Try it out.

Android Ahoy!

After a long time wishing I had the time to develop an Android application, I finally got the chance. We are calling this app "WattDroid." The idea is quite simple, we will be pulling data from WattDepot to display energy usage data from specific dorm floors at the University of Hawaii at Manoa.

When the user launches the application he or she will see a main menu where they can enter (through the settings) a URL that the application will pull the data from. Currently this will work only with WattDepot servers. Then the user will see a list of sources to choose from (sources are dorm floors). When they choose a location, the app will display a meter reading with various other data readouts.

The idea here is to keep it intuitive and consistent with other Android applications and of course fun for the users. I see this application as a way to bypass the need for an expensive energy meter as every student can now have their own meter display for free.

So far we have read the first 5 chapters of the excellent book Android and Me (A Pragmatic Bookshelf publication) and are thoroughly enjoying working tidbits of examples into our application. Some of the difficulties we have are the networking aspect, but that should be resolved shortly after this writing. We have already been able to get a list of sources (not from the server yet) into a list that displays a Toast type message. The next step will be to create another activity (Android view) to display the updated readings.

The development cycle is quite fun as well. I simply plug my Android phone into my computer and place it into debug mode, when I hit Run in Eclipse, it sends the APK over to my Android device instead of the AVD (Android Virtual Device/ the emulator), this keeps my Macbook at a lower temperature (sometimes it can get up to 180 degree Fahrenheit).

I will post screenshots next Tuesday of this application.

Brainstorming the Next Phase

We've taken a bit of a break from working on the HEEP widgets for a little while and decided to focus our time on creative ideas for the next round of design. Kevin and I had a few brainstorming sessions over Spring break to come up with ideas that would benefit the users of the HEEP website, the people of HEEP and the students who will be using the larger aspect of the project my team is working on The UH Dorm Energy Competition.

The Dorm Energy Android Application
The idea here is to make the competitors visually see how their dorm and floor is doing compared to other dorms and floors.
Let's put this into a story/scenario:
Sally grabs her Android phone and presses the UH Dorm icon on her home screen.
She sees a layout that displays many floors of a particular dorm. She swipes the screen to change which dorm that she wants more information about. When she gets to the dorm she wants to view, she sees numbers 1-6 on a building representing the floors. She wants to find out how her friend Mark is doing on the 4th floor of Freer (a Dorm at UH). She taps the number 6 on a vertical structure that represents a building. A pretty layout with live energy consumption and a simple one week graph is displayed.

The idea here is to not push too much information on the students and allow an alternative mobile vector for the competition. We believe a simple mobile application has that potential.

A Visual Javascript Application to Display Household Energy Data
This idea is still under the works. So far our idea is to create an application that can visually demonstrate a graphical depiction of Honolulu energy usage based on household data. There are many possibilities with this and we intend to explore them when the data becomes available. Currently we are looking forward to seeing what kind of infrastructure we can obtain to house the 1TB+ of data on a SQL server that obtains fresh data nightly. I predict the model will be based on OLAP and contain highly detailed information in the form of a Google Map overlay. More to come on this exciting project.

Milestone 2

We have completed 3 of our gadgets. I'd like to share what we have so far.

First off, let me mention that these gadgets are Google Gadgets for placement of your iGoogle homepage. They report information by querying a Google Spreadsheet from javascript and then displaying the appropriate visualization or dynamic content.

Our Don’t Waste Money iGoogle Gadget displays the HEEP Rebate + Money Saved. It gives the consumer a birds eye view of how much they will earn by choosing an EnergyStar model.

When you select a certain brand, it queries the spreadsheet for the relation between the brand, rebate and savings.

Our NegaWatt iGoogle Gadget displays the amount of watts saved for certain types of appliances.

The following graph displays the HCEI (Hawaii Clean Energy Initiative) goals for the projected years. It also shows real statistics for given years that have passed.

Here are two of them together on the iGoogle Page:



There are some things we would like to continue to work on in order to perfect them. Most of these have to do with white space surrounding the visualizations and other CSS refinement.

You can search for these gadgets on Google Gadgets Search or simply visit our project page:

http://wiki.github.com/remyb/heepoverflow/

Most Prototypes Complete

Today I'm glad to say that we have completed most of our prototypes based on the mock-ups I previously demonstrated.

Let's have a look:

Mock-up:

Prototype:


Mock-up:

Prototype:


After this iteration, we will be tweaking the CSS to make it look as clean as possible to a deliverable state. Next week the applications will be ready for HEEP to review.

On another note there is a quite a cool trick I found that I thought I'd share. You can use any font you like in CSS without any PHP or GD tricks. It will actually load a TTF into the client browsers memory. You can upload a TTF here:

http://www.fontsquirrel.com/fontface/generator

and get a few lines of code that will work on all browsers. Here is a cut out of what the TTF looks
like and what you would place in your CSS file.

@font-face {
 font-family: 'TizaRegular';
 src: url('tiza.eot');
 src: local('Tiza Regular'), local('Tiza'), url(data:font/truetype;charset=utf-8;
base64,AAEAAAASAQAABAAgRkZUTU5mIEMABZvIAAAAHEdERUYA2gAGAAWXPAAAACBHUE9T/ykFxgAFl3wAAAR
MR1NVQmyRdI8ABZdcAAAAIE9TLzKFzi0LAAABqAAAAGBjbWFwGBFXGwAABLwAAAHKY3Z0IAAwBj8AAAgUAAAA
EmZwZ20GWZw3AAAGiAAAAXNnYXNwABcACQAFlywAAAAQZ2x5ZrZ+zyUAAArgAAWF9GhlYWTzweoFAAABLAAAA
DZoaGVhCfIE9gAAAWQAAAAkaG10eDgKBEwAAAIIAAACtGxvY2ECplwQAAAIKAAAArhtYXhwBIou4AAAAYgAAA
AgbmFtZfXm/qwABZDUAAAE1HBvc3RZOs/WAAWVqAAAAYRwcmVwHP99nAAAB/wAAAAWAAEAAAABAABpZxRFXw8
89QAfA+gAAAAAx7FSegAAAADHsVJ6/7b/aAV2BMMAAAAIAAIAAQAAAAAAAQAABMP/aAAABcL/tv8eBXYAAQAA
AAAAAAAAAAAAAAAAAK0AAQAAAK0TpQDyE24A3gABAAAAAAAKAAACAAfLAAIAAQADA1EBkAAFAAQCvAKKAAAA
jAK8AooAAAHdADIA+gQBAgYIBwAAAAIABIAAACMAAAAAAAAAAAAAAABweXJzAEAAISAeAu7/BgAABMMAmAAA
AAEAAAAAA18DVgAAACAAAgH0AAAAAAAAAU0AAAH0AAABwwAbAv0AGwNdACIC4QAaAr8AGgPzABoBjAAbAd4A
FQH6ACECSwAVAokAJgFPACECBwAiAR8AIQKJAAgDQQAlAnQAIgPDAB8DQQAhA38AFQMfABkCuQAaA38AGgN/
ABsDNAAbAToAIQEtABsBrgAbAugAUwH0ABsCpAATAqQAGwSj//cD/f/5A9j/6QQI//MDvv/wA5X/+QQS/+UE
bv/zAln/+QLU//YERAAhA7UABAVL//cEO//zA5r/9gOuABsDcf/9BBL/+gN+AAAD1f//BDsAAARw//UFwv//
BB//6wOK//UDp//1AQj//QKJAA0BNAAOAr8AEwKyABUBkwAbBNX/9wP2//kDzP/pA83/8wPo//ADef/5BAj
/5QRu//MCHv/5Aqn/9gREACEDtQAEBTX/9wRm//MDnP/2A64AGwNd//0EBP/6A34AAAPV//8EOwAPBGX/9Q
XC//8EH//rA4r/9QOa//UBeAAVAVoAVQFHAB0BoQABAToAGwJ9ABUCkAAaA0EA1wKrABsCWP/+A0EAawGSA
AACWAACBKP/9wSj//cEo//3BKP/9wSj//cEo//3A9j/6QO+//ADvv/wA77/8AO+//ACWf/5Aln/+QJZ/+s
CWf/5A83/8wQ7//QDmv/2A5r/9gOa//YDmv/2A5r/9gQfAKwDmv/2BDsAAAQ7AAAEOwAABDsAAAOK//UEo
//3BKP/9wSj//cEo//3BKP/9wSj//cDzP/pA77/8AO+//AD6P/wA77/8AJZ//kCWf/5A0H/9wJZ//kEZv/
zA5z/9gOc//YDnP/2A5z/9gOc//YBOv+2A5z/9gQ7AA8EOwAPBDsADwQ7AA8Div/1A4r/9QNBAKYB4gAEA0
EABAES//wBJgAAAmwAAgKjAAMCjAAhAAAAAwAAAAMAAAAcAAEAAAAAAMQAAwABAAAAHAAEAKgAAAAkACAA
BAAEAH4AowCpAKsArgC0ALsAxQDdAOUA7wD9AP8C3CAUIBkgHv//AAAAIACgAKgAqwCuALQAuwDAAMcA4A
DnAPEA/wLcIBMgGCAc////4wAA/73/vP+6/7X/r/+r/6r/qP+n/6b/pf3J4JPgkOCOAAEAAAAiAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwBiAGMAZAAAAQYAAAEAAAAAAAAAAQIAAAACAAAAAAAAAA
AAAAAAAAAAAQAAAAQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4vMDEyMzQ1
Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGEAb3Bxc3uAhom
==) format('truetype');

Enjoy :)

Refactoring Designs

Since the last post, we have refactored our designs to better communicate with our target audience. This is more of a marketing stage in our design. We still have not yet reached the implementation phase but we hope that changes in the next week.

This is what we have for our Negawatts Generated by Rebates application now:

This is a visualization of the percentage of total rebates that were generated by certain appliances. HEEP provides rebates for certain appliances and shows consumers how much energy they did not use because of their rebates.

We decided to get rid of the Appliances Energy Savings calculator because it did not relate to HEEP at all. It also is available on other websites like EnergyStar.

We constructed a Top EnergyStar Appliance Calculator that displays the potential loss caused by NOT purchasing an EnergyStar appliance. Studies show that people are more inclined to pay attention to things that cost them money, then things that save them money.


This is our Projected Energy Generation Visualization. It shows the past and projected means of generating energy up to 2030.

This visualization shows our past and present oil consumption and a bubble that appears when you mouse over a data point for the given year.

The next few weeks we are going to produce the final prototypes of our models, a fully working example that we hope will meet the needs of HEEP and their visitors.

On a side note, I really enjoy using CampFireNow.com and BaseCampHQ.com to manage our project and communicate. Campfirenow is a great web chat based collaboration tool that lets you share files and display them in the chat area while retaining the format and markup of pasted code. It also logs your chats so if a member of the group comes late he or she can catch up, search and see what happened earlier. Basecamphq.com is an amazing project management tool that lets you do everything Google code does, but a lot more. Both are written in ROR (Ruby on Rails) and I really enjoy the way it all works. You'll have to see for yourself to know what I mean.

Stay tuned for our next update...

HEEP Design Revisions

Realizing that our last designs were a bit too complicated for the average user, we decided to play them down a bit and make a few of them simpler to use and more visually appearing. I'm learning that high level design is much more challenging then the actual implementation. As such, I think this is a really good, but difficult exercise. I am starting to see the benefits after much frustration.

Let me walk you through a few of our mock-ups:

This is the EnergyStar Refrigerator Calculator. You select an approximate date and size and it will calculate your savings based on predefined average energy usage by the appliances. The energy cost, which will fluctuate the most will be either scraped from an external source, or updated manually on a Google Spreadsheet.

This is the Oil Consumption Timeline. Looking at it will hopefully tell you what is happening. It calculates the change from last year in the amount of oil consumed (number of barrels).

This is the Energy Generation by Source Chart. It breaks down the recent and future use of certain methods of power generation.

We also have the idea of working with NegaWatts (from Wiki):

Negawatt power is the idea of creating incentives to reduce demand for electricity to ease the load at peak times or alleviate the need to build more generation plants. In theory, these negawatts can be aggregated and an arbitrage market could be created to trade these.
The term was coined by Amory Lovins, who saw a typo — "negawatt" instead of "megawatt" — in a Colorado Public Utilities Commission report. He adopted the term to describe electricity that wasn't created due to energy efficiency.[1]

The Negawatts is almost complete, but the visualization needs a bit more work. Basically it comes down to totaling up the Negawatts for the appliances.

As we get closer to picking a few top selections of these visualizations and calculators, we will start to really refine them and take feedback from different people to see what needs more work.

HEEP Update - Milestone 1

Today is the first milestone of our HEEP project. I think significant progress has been made as far as creative thinking and design has been concerned. It's like a paradigm shift from constructing from specifications to designing a project for the client. They have basically handed us the reigns to pursue what we think is best. I hope it ends that way as well.

Our plan is now to come up with as many ideas as we can think of, outside the box. We are no longer focusing solely on visualizations. We want to create applications that reflect what the user of the site would want, not necessarily what the organization would like for itself. For example, a report card for the organization is not really at the best interests of the organization, rather, a report card for products that it gives rebates on would be a wiser choice since the consumer will find that information much more useful.

Currently we have thought of a few calculators that will serve as applications in the tabs. These are being implemented in plain javascript with the use of google visualization api to fetch data from the spreadsheets into our function that does the calculations. The result is displayed for the user in an easy to interpret interface.

The Appliance Report Card

• Get top 5 Energy Star Efficient Models
• Get Cheapest Non-Energy Star Models
• All models must have similar specs
• Compare/Determine How Long Until Break Even
• Less Amount of Time is Better
• Grade Based on Time for Break Even

The Appliance Calculator

• User enters info in for 2 models
• Calculate Break Even Point
• Total Savings and amount of time (lifetime estimate)

The Cooling Degree Day Calculator

• User enters two years, their corresponding
  power usages, and a base temperature
  
• Calculates total change in power usage per cooling degree day and percent of change

The Cooling Degree vs Oahu Generation

• Power generation for two years are read
• Cooling Degree Data is read for corresponding years
• Calculates the amount of power generated per Cooling Degree Day and percent change

We expect to have 2 of these calculators available for demo in two days.

We also plan on giving HEEP a live demonstration so they can give feed back in 2 weeks.

HEEP Status Update

We have implemented a raw form of the requested Google Chart visualizations embedded in JQuery Tabs. The problem with this is that the data is displayed in a form that is not quite readable or understandable by the user. Our next step is to take the Cooling Degree Day chart and the HEEP Report card and figure out a way to make them more usable. Brainstorming ways to achieve this over the next day hopefully will give us a good result.

Our current idea for the the Cooling Degree Day data is to display a settings tab, as part of the chart, that will allow the user to input his usage values to obtain a chart that provides a useful user-centric visualization of the request.

As far as the HEEP Report Card, currently we are going to design simply that, a report card. Letters /[A-F]/ will be distributed over months to indicate HEEP's success. In order to calculate this data is a different story, it is something that we need to work with HEEP on and discuss in detail. For now we'll focus on creating creative solutions.

By Thursday we hope to have, at the least, a mockup design of the charts we will be designing.

Google Visualizations with Energy Data

After much discussing we decided that Google Visualizations would be the best way to develop an application that can be easily used and maintained by the folks at http://hawaiienergy.com/. The idea is to create 3 or 4 applications within a tabbed environment on a section of their page. This will allow visitors to interact with the data that HEEP intends to make transparent. Our applications have the following design specs:

Stoplight Style Representation for Current Power Status

• Red, yellow, and green to represent high, moderate, and low demand times
• Use their socket icon(logo) to represent the colors.

Oil Price Variance over Time

• Create Google visualization to represent oil prices as stacks of barrels
• Information updated daily

HEEP Report Card

• Report HEEP's performance Displays cost vs. power savings Metrics updated weekly

Cooling degree days

• Display relationship between cooling degree days and power usage
• One cooling degree day per degree above 65 degrees

Our first task is to learn the visualization API and then show HEEP a demo.

I will post the demo here when it is complete.

New Project Underway

Kevin, Alex and I are going to be developing an embedded web application for The Hawaii Energy Efficiency Program. They are currently contracting a web design group from Maryland to construct their website. We will be working with this both of these groups to integrate our applications together. The interesting aspect of this project is that it is entirely up to us to design, implement Kevin, Alex and I are going to be developing an embedded web application for The Hawaii Energy Efficiency Program. They are currently contracting a web design group from Maryland to construct their website. We will be working with both of these groups to integrate our applications together. The interesting aspect of this project is that it is entirely up to us to design, implement and manage to the project. There are no assignments or deadlines except the ones we make. I think this freedom will allow us to really engage the project and teach us a lot about planning and real development. Working with people in the industry will also be a plus. So far we don’t have much planned until our initial meeting on Tuesday. I hope to gain perspective on the requirements and wishes of the client during this initial meeting.