Reactions: 6. December

Christian mogensen@cs.stanford.edu http://www-pcd.stanford.edu/mogens/377/reaction-1206.html

Commercial Memory Systems

Von O. Leirer, E. Decker Tanke, D. G. Morrow

Business is risk management.
minimize the chances of really big and costly false starts
The commercial process is thus geared towards ensuring that reasonable risks are taken given the expectations of rewards at a given point. Given the exploratory nature of their enterprise, it makes sense to prototype quickly and rapidly so as to understand the problem better.
It's interesting to note that they developed the prototype with the intent to test it out at such an early stage. Traditional engineering life-cycle models perform testing at a much later stage.
The other significant point about the early user testing is that they built up a lot of understanding about the problem through the tests, instead of relying on interviews and surveys. The scientific studies and research functioned mainly to discover problem areas and suggest solution methods. The early studies also act as a filter - irrelevant or low-impact aspects get cut out quickly.
They do not mention any ethnographic studies or user interviews to study the user interface. In fact they mention that this is the most challenging aspect of the development. It would be interesting to see how it fits in with the rest of the system
Phase two appears to be a full scale test of the system and all of its components. Had this been as software journal, I would want to see more discussion on the design and development that must have taken place between phase one and two.
The results are very good. (p<0.001) indicates that the results are pretty strong, especially given the larger sample size and increased number of tests in phase two.
The discussion of measurements (change in non-adherence) is illuminating: something to remember. The same goes for the discussion of commercial success and how to measure it.
It's interesting to note that IBM and HP are both moving towards the open team approach, where researchers intermingle with the commercial developers.

All in all, a very interesting and thought provoking paper. The final point about needing concrete results to point to when asking for more NSF grants is well made.