Main

June 10, 2007

Adventures in Optimization: Benchmarking

My first job at Apple in 1992 was benchmarking the quality of a dozen different handwriting recognition software libraries from different companies. We like to pretend that everything from health to cars to wars can be measured by a single number but real systems are multi-dimensional. Benchmarks are like standards: everyone likes to make their own. I approach software measurements with my old Stanford Physics lab training. I treat the software as tangible physical system, experiment to determine what I can measure reproducibly, look for mathematical relationships between variables, and attempt to construct a useful basis set of axes upon which to characterize their phase space. This did not make me any friend while benchmarking for Apple. Each company or internal group with a solution to sell had strong ideas about comparing systems to put their own solution in the best light. Furthermore, handwriting recognition was and is still an unsolved research problem. Comparing different research projects is quite different from comparing products of a mature technology. Ultimately, none of the choices proved adequate.

For Graphing Calculator, I use performance measurements to make programming choices when comparing high-level algorithms, low-level implementation details, different compilers, and compilation options. Holding the Command key while pressing the slider Play button will initiate a stopwatch to time the slider. When I'm focusing on a particular routine, this is useful to compare small code changes from one build to the next. In the Graphing Calculator menu, the Graphing Calculator > Benchmark command runs through thirty-two tests timing calculation and drawing. These help me compare how GC behaves on different hardware and how its behavior changes across versions years apart. The report looks like this:

Continue reading "Adventures in Optimization: Benchmarking" »

December 11, 2006

Google Videos

Peter Norvig's Google Research Picks for Videos of the Year. The Graphing Calculator Story Hour came in at #2!

December 01, 2006

Lessons from our Usability Study: Part I

The most educational part of the original Graphing Calculator 1.0 development process was sitting behind a two-way mirror for five days watching first-time users, students and teachers, trying to use our software. It was a humbling and enlightening experience. Anyone doing any aspect of software development: engineering, design, support, documentation, as well as user interface folks, should be subjected to the experience. Though the our study was done with version 1.0 way back in 1993 many of the lessons we learned were timeless.

Two recent blog posts bring one particular lesson to mind. Coding Horror warns This Is What Happens When You Let Developers Create UI and Arno Gourdol discusses The Design of the Mac OS X Shutdown Feature:

It is the job of the software designer to make choices on behalf of the user. That's what designing is all about.

Continue reading "Lessons from our Usability Study: Part I" »

October 11, 2006

Cooperative Multitasking

I've been deep in the bit mines all week, wired on Mountain Dew, reviewing the 35,000 lines of code which implement the actual graphing classes of Graphing Calculator, making the Compute method of each different graphing algorithm thread-safe and able to run preemptively. It's been fun to revisit a decade worth of code evolution and simplify it with the benefit of hindsight. Much of the complication was due to the necessity of preserving state across multiple calls to the Compute method of each class so that it could be time-sliced to keep the UI responsive in a single-threaded environment.

As I sat down to describe this, I remembered that I had already wrote what I wanted to say four years ago to a colleague who asked: I've always wanted to know how to do threading. Under OS X that will become more and more valuable as multi-processor machines become increasingly common. Where is there a good discussion of using threads?

At 3:05 PM -0800 10/25/02, Ron Avitzur wrote:

Continue reading "Cooperative Multitasking" »

September 14, 2006

Hardware

On the occasion of the 50th anniversary of the hard disk drive I've been pondering the evolution of the hardware Graphing Calculator grew up on.

The original hard disk drive (IBM 305 RAMAC) in 1956 stored only 5MB of data on fifty 24-inch disks and weighed over 250 kg (over 550 pounds). About the size of two large refrigerators and as tall as a man, you could lease the whole unit for about $250,000/year in today's dollars - John Cole

Continue reading "Hardware" »

August 08, 2006

Breaking into Apple

Larry Osterman comments on The Graphing Calculator Story

It's an "interesting" story, and I have to say that I was aghast when I read it.  And my jaw dropped even further when I read the Digg comments about it....

For some reason, the people reading the story actually thought it was COOL!

Now think about it.  These people were so dedicated to their product that they were willing to BREAK THE LAW to get it into Apple's OS....

Apple CANCELED their project. Apple made an executive decision to not produce the product.

And the developers decided to override it by BREAKING INTO APPLE.

There are a bazillion other ways they could have handled it: Building it on their own and trying to convince Apple to put it in the product, etc.  But no, they chose to STEAL from Apple.  And the policies of Apple were sufficiently lax that they let them get away with it.

The discussion going on the comments on Larry's blog is a fascinating consideration of the ethics of the situation. I find myself largely sympathetic to his reaction. He is not wrong.

Continue reading "Breaking into Apple" »