Four months have passed since our first usability test. We’re not neglecting the usability of EditGrid, but we’re resorting to real users’ feedbacks after gaining more publicity.

But after all, we still believe observing users with their keyboard and mouse actions is the best way to discover usability problems. So, a second usability test was held in our office. It was a hot day, with air-conditioning extended after office hours. The subjects were our 6 newly contracted software testers. They are all new to EditGrid.

EditGrid Usability Test Report 2, 2006-07-21

System

EditGrid, Beta 7

Subjects

6 undergraduate students, with IT background.

Procedure

Ask subjects to create a budgeting spreadsheet for a student association, and email it out to invite members to edit. Record the whole process for evaluation.

Observations

1. Context Menu is missing or wrong in various locations. Including row/column headers, and when a cell is in edit mode (F2).
2. Toolboxes not covering windowed controls. This is an IE-only bug, which has been worked around in our Menu Bar implementation. We need to extend this to the Toolboxes also.
3. Focused Toolboxes not bubbled to the front. Something we sacrified for more important features. Should consider giving it a higher priority.
4. Toolboxes can move out of visible area. Which should be bounded.
5. Improper way of word-wraping long text/formula under edit mode (F2). The current implementation hinders the input of long line of text.
6. ‘Create New’ is not prominant. New users still find it hard to locate the button to create new spreadsheet.
7. Various spreadsheet operations require grid reload, which can be avoided. Including changing spreadsheet permissions, adding tags, sharing, etc. To be ajaxified.
8. No visual indication of cell references. This is a very useful visual aid in creating formulas, but was in low priority.
9. Toolbar icons not flowing together in groups. For small screens, our toolbar icons will overflow to the next line, but not having the grouping being kept intact.
10. Cannot merge cells. Again, something we sacrified for more important features, but is now on the top of everybody’s wishlist. Must give it a higher priority.
11. Incomplete keyboard and mouse support for range operations. Including range selection using Shift+Click, drag to fill with Ctrl being pressed, etc.
12. Insert and delete row not disabled on column selection, and vice versa. This should be disabled as these kinds of operations give no sensible outcome.
13. Permalink feature is hard to discover. Perhaps we should show it up in more places.

Self-Evaluation

The test reveals no major usability problem. But having many minor usability problems is even worse than having a big one. We have been adding many new features in the past few months, but were not giving high attention to usability improvement and the feature harmonisation.

On completing more and more items in the list of most wanted features, I strongly believe it’s time to come back to usability issues.

END OF REPORT

Usability is everything. Tech people has their fingertips pretty integrated with the machines, but to a general user it’s a different story. Starting from private release 2, we’ll rely on usability tests rather than the intuition of developers to lead our product design.

Today, there were 4 subjects, sitting in front of the computer one after another to perform a simple register / create sheet / logout test case under no external intervention. That’s how we get started our first lesson.

EditGrid Usability Test Report 1, 2006-03-11

System

EditGrid, Private Release 2 (as of 2006-03-10)

Subjects

A law firm’s account clerk, a civil engineering undergraduate student, a garment trader’s manager, a fashion designer

Observations on Spreadsheet View

1. All subjects failed to handcraft a formula correctly - They get used to building formula by mouse clicks.
2. Not awared of the save sheet notice - Even it is red in color!
3. Save sheet sequence is unexpected to user - This includes requiring to login, and getting out of the spreadsheet view.
4. Failed to distinguish between functions provided by browser and the system - Using browser’s right-click context menu to manipulate cells, searching for functions in the browser’s menu bar, pressing CTRL-S to save a sheet on the destop!
5. To save or not to save - User is very unexpected to find no save button, but is also unexpected when requiring to save a newly created / imported sheet. Also, we are unable to tell users clearly that whether a sheet shall be saved prior to closing, because in our implementation, that depends!
6. Some users misunderstood that there is no copy & paste - Because there’s no visual effect after doing a copy or cut.
7. Some are unfamilar with import and export - Feel confused when presented with a file input box, and even would types in it. Also feel confused when presented with a download window after clicking the export link. The file formats in the export links are mostly unfamilar, and that cause distractions.
8. Confused of the terms private and public - They have no idea what sheet sharing is, not to mention the access control mechanism behind it.
9. Not awared of the Permlink feature - Have no way to know about it. Have no idea what the chain icon means.
10. Problem with delayed cell updates after actions - If nothing happens immediately after, e.g. pasting on a cell, users may feel they have not performed the action. Some loading visual effect can help a lot here.

Observations on Website

1. Failed to comprehend product vision from cover page - Only know its a spreadsheet software.
2. Can’t get started - Can’t locate the register combo box and quick start link. Perhaps two new tabs could help: sign-in and register.
3. Decide slowly on quick start - The options in the quick start page is confusing, and nobody clicks the “start new blank sheet” at the first place.
4. Unexpected ordering of search results - Currently, username matches come before sheet name matches.
5. Links are too subtle - Perhaps links should be underlined, more bold, more colorful. How about replacing some with buttons?
6. Can’t back home - The “home” button is now the EditGrid logo. A bad idea.
7. Forget to logout - The logout link is, yes, too subtle.

Feature Requested by the Subjects

1. Graphing
2. Manipulate cell data by mouse dragging
3. Currency and thousands mark (,)
4. Multiple cell selection by SHIFT+ARROW
5. Cell borders
6. Inserting symbols
7. Auto-logout on closing browser
8. Printing
9. Pasting single cell data to multiple cells

Self-Evaluation

First, nobody really talks about the speed.

Second, many compares features and UI with common desktop spreadsheet apps in a very strict sense. Perceptions are totally that we’re making another desktop spreadsheet app. So the feature must be more comprehensive in order to make any sense. We failed to convey our objective as a collaborative app. Perhaps, the look and feel of our app is too close to a desktop app, and many don’t even aware that our app is running inside a browser.

END OF REPORT