Eric Sword's list is more focused on problems with the naive user experience while my list is more of a component by component analysis based on HCI knowledge. These two methods capture quite different errors.
The user's goal is to check the weather - the item by item breakdown does not capture this piece of context, and therefore does not discover the problem that no-one knows the grid coordinates of New York. The HCI principles merely point out that there are easier ways to choose a point on a grid instead of a text input. For example, a natural mapping might exist, such as pointing on a globe, or scrolling the window as a view onto a larger map. (Norman)
Both methods catch the fact that the error messages are poorly done, but the user perspective fails to offer suggestions for improving the interface. A combined approach would work well - the user perspective (perhaps gained from a UI walkthrough) points out problems. Application of HCI analysis should suggest alternatives.
Eric Sword points out that the keyboard interface is ambiguous - pressing Tab and Return have uncertain effects. HCI principles from Norman tell us that we could get rid of the arbitrary text boxes all together in favor of more natural mappings: scroll bars for the Map center and magnification fields, scrolling lists of possible values for the date. By removing the need to type in input fields, we also remove the need to press enter, and improve feedback - moving a scrollbar should immediately change the picture.
Sword also notes "why is the Map Center a Zoom Specification?". This is an interesting point. It would make more sense if the group were named "Map Window Specification" or similar, since the two controls both affect the map-to-window mapping. Changing the title is an easy fix, but doesn't solve the deeper confusion. A bit of analysis yields the result above: more direct manipulation of the map.
Applying a bit of naive-user viewpoint shows up more problems:
The analytic solution to this is to change the controls from radio buttons to checkboxes, thus allowing more than one option to be selected at once. Distinguishing the different details can be done with icons (rain/shine/cloudy) or with colorcoding for each option.
The (obvious) fix is to stop resetting the text input after showing the error message box. The deeper solution is to redesign the interface so as to make errors impossible by making everything visible and directly manipulable (Norman).
Problems fall into three broad categories:
One of the nice things about a heuristic evaluation is that it can be done on paper prototypes. This makes it easy and cheap to set up, and hence it can be done early and often in the design process. One problem is that it can become rather boring after the tenth repetition.
The law of diminishing returns works here as everywhere else. After the fifth or so analyst, you have covered about 90% of the possible errors. In order to discover the remaining 10% requires as much effort as the first 90% - much more time and effort than they warrant (usually). Of course, if all five analysts use the same analysis methods and have the same viewpoints, then many more errors are likely to go unseen. Variety is therefore important to ensure in these analyses.
Combining methods should also help inform the analysis: play-acting the interface components should help point out information flow problems: "I'm the Map viewer", "I'm the Fahrenheit/Centigrade control".
Heuristic analysis tends to focus on components, and can miss higher level problems. Applying higher level heuristics (the naive user viewpoint) can uncover higher level problems. However, the heuristic analysis method is unlikely to uncover problems in timing (an implementation issue) and in interaction between different controls - i.e. the effect of an undo button on user behavior.
Things like problems in the dialogue, the holistic problems in the interface, are hard to catch using heuristic analysis simply because so many details start to become significant. At some point a mockup has to be built and tested.