What Alan Kay's Programming and Scaling Says About Molecular Data

Woolf Software
software design systems biology alan kay data engineering
A misty glowing forest where one organism repeats at four sizes, from tiny polyps to a tower, linked by pulsing fibers.

This essay is prompted by a talk Alan Kay gave at the Hasso Plattner Institute in Potsdam on July 21, 2011. It was announced there as “Next steps for qualitatively improving programming” and it circulates online as Programming and Scaling. Kay coined the term object-oriented programming, and the talk is his account of what he meant by it and why most software built under that name does not scale. His argument passes through biology twice. He uses the Internet and living things as the two large systems whose complication matches their complexity, and he uses the size of a bacterium to show how far our software abstractions sit from the ones that build organisms. This series is about the traffic between molecular biology and computation in both directions, and the talk is a clean case of the reverse flow. We want to lay out what Kay says and check his numbers against real sources. Then we ask what the argument implies for a data system whose parts are a genome, a blood panel, and a glucose trace.

Software measured in books

Kay opens the technical part of the talk with a unit of measure. A book of software is about 20,000 lines of code, and a meter of shelf holds about a million lines. On that scale Windows Vista, at 120 million lines, is 6,000 books. The applications that ran on it add another 140 million lines and 7,000 more books, for 13,000 in total. He asks whether that much code is needed to produce that much functionality, or whether something has gone wrong.

His answer is a pair of curves. Complexity is the amount of behavior the system has to have, and complication is the amount of machinery written to get it. He puts the gap for personal computing at a factor of 100 or 1,000, and says that at those ratios the difference is between a system one person can comprehend and one that nobody can. His example is a selection bug in Microsoft Word that he says has been reported tens of thousands of times over 25 years without being found. A system 13,000 books long cannot be searched by anyone.

The one large system that is tidy

Against that background Kay singles out one artifact that is huge and yet tidy, and it is the Internet. By his count the protocol layer that runs it is under 20,000 lines of C and could be written in about a thousand. TCP/IP is the pair of protocols that carry packets between machines and reassemble them in order, and it was specified by Vint Cerf and Bob Kahn in 1974 1. Kay’s description of it in the talk is that “TCP/IP is a kind of a universal DNA”. He notes that the network has run continuously since September 1969 and has replaced every one of its physical parts at least once. It has been changed and improved in that time without ever being stopped. He excludes the web from this praise.

His explanation for why the Internet came out this way is that its designers made a qualitative jump from gear-like things to biological things. Gears have to fit closely, which is how he characterizes data structures and procedures joined by APIs. Living systems are made of protected parts that repair and grow while running, and some of the people who designed the Internet had been biologists and brought that model with them. The rest of software, in his telling, is a missed chance to take the Internet as the model for how modules relate.

Objects as cells

The biological model is older than the talk. In his 1993 history of Smalltalk, Kay describes the original idea as replacing data structures and procedures with a scheme of protected universal cells that interact only through messages 2. Each object was to be a recursion on the whole computer rather than a weaker piece of one, so that a program’s semantics resembled thousands of computers joined by a fast network. He credits the recursive principle to Bob Barton, who designed the Burroughs B5000 3 and taught the systems design course Kay took as a graduate student. Two consequences follow, and Kay states both in the talk. The first concerns setters, the methods in mainstream object-oriented languages that let outside code write directly into an object’s fields. He says that adding a setter turns an object back into a data structure, because outside code is once again assigning into its fields. The second concerns size. Looking at the gap between a protein and a cell, he says that “one of the mistakes we made years ago was to make objects too small”. A cell is a far more capable unit than an integer or a record, and building from more capable units is where he thinks the scaling comes from.

The scale numbers, checked

Kay’s biological illustration in the talk comes from David Goodsell’s The Machinery of Life, a molecular biologist’s book of watercolor paintings of the interior of cells 4. An E. coli cell is about two microns long. A medium-sized protein has about 5,000 atoms, and Kay puts the number of non-water molecules in one bacterium at about 120 million. A protein could be assembled by hand. A bacterium could not, so it has to be grown. The jump in scale from polymers to a living cell is much larger than the jump from primitive data types to the objects we write.

The talk stops at the bacterium, so the next numbers are ours rather than his. A 70 kilogram reference human contains about 30 trillion (3.0 × 1013) human cells, and roughly 90 percent of them are red blood cells 5. The same study puts bacterial cells at 38 trillion, close to a one to one ratio. The instruction set for all of the human cells fits in a genome that GRCh38 sizes at about 3.1 billion base pairs 6. The complete T2T-CHM13 assembly comes to 3.055 billion base pairs and carries 19,969 predicted protein-coding genes 7. At two bits per base the whole genome is under 800 megabytes. Kay’s 120 million lines of Vista, at a modest 40 characters per line, is several gigabytes of source text. That comparison is loose, since one is a description and the other an implementation, but its direction is the point. The genome does not contain 30 trillion cells. It contains the rules for one cell, and the trillions arise from cells that build and regulate each other by exchanging signals.

STEPS

Kay’s group at Viewpoints Research Institute tried to build software the way a genome builds a body, and the experiment has a name and a budget. STEPS, funded by an NSF grant awarded in 2006, set out to build a complete personal computing system in about 20,000 lines of code 8. The first-year report lays out the arithmetic: 20,000 lines is a 400-page book, and hitting the target would be a reduction of three to four orders of magnitude from shipping systems. The budget counted only lines of meaning. Optimizations could be added on the side, and the system had to run correctly with all of them switched off.

The method was to find the mathematics of each subsystem and write a small language in which that mathematics runs directly. In the talk Kay shows the graphics case. Dan Amelang and his mother, a high school geometry teacher, worked out a formula for the exact coverage of a pixel by an arbitrary polygon. He then wrote a stream language called Nile in which 40 lines of mathematics became about 80 lines of runnable code. The 26 standard compositing rules took about 90 more, and the whole of 2D personal computer graphics came to under 400 lines. Kay sets that against the one or two million lines of Cairo, the rendering library behind Firefox. The 2012 final report puts the finished Nile graphics at 435 lines 9. The languages were made with OMeta, a parser generator written in itself, and Kay reports that OMeta can build itself in 76 lines. TCP/IP, treated as a non-deterministic parser over packet headers, came to under 200 lines against the 10,000 to 20,000 of a typical C implementation 8.

The final report is candid about what was and was not achieved. Frank, the document editor the group used to give its presentations and write the report, runs on about 10,000 essential lines and 17,358 in total with bindings and tests 9. The group concludes that much of personal computing can be built in thousands to tens of thousands of lines. It also notes that it covered less ground than planned, because making the system run in real time on ordinary laptops fought with the original goal. Two principles from the reports carry over to what follows. One is extreme late binding, which means that decisions about types and compilation and deployment are deferred until the moment they are needed. The other is the group’s statement that the real language of an object system is the messages that pass between its parts. Imperatives and names do not scale, they write, whereas cellular biology has scaled by noticing and reacting for billions of years 8.

What this means for a personal molecular data system

Now consider a system whose parts are a whole genome, a series of blood panels, and a continuous glucose trace. RNA and protein measurements arrive later. The usual way to build it is a pipeline. Each file goes through a stage, each stage knows the internal layout of the files it consumes, and the last stage emits a wide table with one row per person. That is the Vista design in miniature. Every new assay adds a stage, every change to a file format touches every stage downstream of it, and after a few years the pipeline is a shelf of books that nobody has read. It also has setters everywhere, because stages write into shared tables and later stages cannot tell who wrote what.

The Kay design starts from the observation that the parts are already close to being cells. A VCF file is the standard format for genetic variants. It begins with a header that declares its version, the reference genome it is relative to, and the definition of every field it contains 10. It is a file that describes itself. A blood panel is a set of analytes with units and reference ranges, plus a lab and a draw time. A glucose trace is a sensor identifier and a series of timestamped readings with a known lag behind blood. Each of these is a self-contained object with its own description, its own error model, and its own clock. The FAIR principles for scientific data, which ask that data carry machine-readable metadata about itself, are the same idea stated for a different audience 11.

Messaging then has a concrete meaning. A message is a question posed to one of these objects, and the object answers from its own contents in its own units. Ask the genome for the genotype at a locus and it answers with the two alleles, the reference build, and the read depth behind the call. Ask the glucose trace for the two hours after a meal and it answers with a series and the sensor’s lag. Ask the blood panel for fasting glucose and it answers with a value, a unit, and the lab’s reference range. Integration takes the form of a conversation between parts. The part that asks the question is responsible for reconciling units and time, which keeps that logic in one place. No part writes into another. A reanalysis of the genome with a new caller produces a new genome object with a new version, in the same way that a cell does not reach into a neighbor’s cytoplasm.

Late binding is the rule for when meaning gets decided. A pipeline decides at ingest what a glucose reading means to a genotype, because it has to pick the columns of the wide table before it knows what will be asked. A late-bound system stores each object as it arrived and resolves the relationship at the moment a question about it exists. This costs some compute at query time. It buys the ability to ask questions nobody thought of when the data was collected, which in a longitudinal system is most of the questions. The 2007 STEPS report suggests that modules should carry a working model of their own environment 8. For a molecular object that model is the assay’s coefficient of variation, the reference population, and the sampling interval. It belongs inside the object.

We would build it this way. A small set of object types, each self-describing and versioned. A shared vocabulary of questions they can answer. A line count kept in the open so that the complication curve stays visible. A genome runs 30 trillion cells from under 800 megabytes by giving every cell the same instructions and letting them talk. A data system for one person is a far easier problem, and Kay’s talk is the argument that it should be sized accordingly.

Questions people also ask

What is Alan Kay’s Programming and Scaling talk about? A 2011 lecture at the Hasso Plattner Institute on why software has grown to hundreds of millions of lines without a matching growth in what it does. Kay contrasts that with the Internet and with living cells.

What does Kay mean by objects as cells? Objects that hold their own state and interact only by sending and receiving messages, in the way that cells signal across membranes rather than editing each other’s contents 2. Setters that let outside code write into an object break the model.

What does late binding mean for a molecular data system? Storing each data source as a self-describing object and deciding how a glucose reading relates to a genotype at the moment someone asks, rather than fixing the relationship in a table at ingest. It costs compute at query time and preserves the ability to ask new questions of old data.

Woolf Software builds longitudinal molecular profiles of individuals: whole-genome sequencing, RNA sequencing, proteomics, blood biomarkers, and continuous glucose data, integrated into one model of you. Build your profile.

Footnotes

  1. Vinton Cerf, Robert Kahn. A Protocol for Packet Network Intercommunication. IEEE Transactions on Communications, 1974. https://doi.org/10.1109/TCOM.1974.1092259 ↩

  2. Alan C. Kay. The early history of Smalltalk. ACM SIGPLAN Notices, 1993. https://doi.org/10.1145/155360.155364 ↩ ↩2

  3. R. S. Barton. A new approach to the functional design of a digital computer. Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference, 1961. https://doi.org/10.1145/1460690.1460736 ↩

  4. David S. Goodsell. The Machinery of Life. Springer, 2009. https://doi.org/10.1007/978-0-387-84925-6 ↩

  5. Ron Sender, Shai Fuchs, Ron Milo. Revised Estimates for the Number of Human and Bacteria Cells in the Body. PLOS Biology, 2016. https://doi.org/10.1371/journal.pbio.1002533 ↩

  6. Genome Reference Consortium. Human Genome Assembly GRCh38.p14, assembly statistics. NCBI, 2022. https://www.ncbi.nlm.nih.gov/grc/human/data ↩

  7. Sergey Nurk, Sergey Koren, Arang Rhie, et al. The complete sequence of a human genome. Science, 2022. https://doi.org/10.1126/science.abj6987 ↩

  8. Viewpoints Research Institute. STEPS Toward the Reinvention of Programming, First Year Progress Report. VPRI Technical Report TR-2007-008, 2007. https://tinlizzie.org/VPRIPapers/tr2007008_steps.pdf ↩ ↩2 ↩3 ↩4

  9. Yoshiki Ohshima, Dan Amelang, Ted Kaehler, et al. STEPS Toward the Reinvention of Programming, 2012 Final Report. VPRI Technical Report TR-2012-001, 2012. https://tinlizzie.org/VPRIPapers/tr2012001_steps.pdf ↩ ↩2

  10. Petr Danecek, Adam Auton, Goncalo Abecasis, et al. The variant call format and VCFtools. Bioinformatics, 2011. https://doi.org/10.1093/bioinformatics/btr330 ↩

  11. Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 2016. https://doi.org/10.1038/sdata.2016.18 ↩