A Horror Story

For over a year I've been publicly writing about digital factories.

The goal is to disrupt the popular conception of software as a static asset like a building or a sculpture. I want to direct people to something closer to the truth - it is a dynamic system that produce a field of outputs. Software traditionally behaves consistently provided a number of prerequisites are met - handshake agreements. Whether these pre-requisites are in fact met is often invisible to consumers of digital products.

When they're not met, people involved find themselves frustrated and asking questions like "why did this happen?" or making remarks like "this didn't happen before", "this used to work", or in the case of more subjective software, "this used to be good". Developers may respond with "it works on my machine", or "it works for me". And they may be correct.

Contemporary digital experiences have many dependencies. Devices, accounts, multiple sources of truth, network quality, and other dimensions as we have previously explored. These help to explain how failure modes can happen. There are other more abstract reasons such as a more substantial differentiation across digital products than we have become accustomed to in the physical world.

To put it in terms that people younger than me might use today - software delivery is a tough caper that is getting tougher by the day. I've written a lot about systemic forces, today is about a horror story.

Years ago I worked in a digital agency. We were a pretty small and fairly tight-knit team and there were projects we would collaborate on together and other projects where it might be a one-person operation.

The project I want to talk about today was a digital transformation for a non-profit organisation. What was taking place was not only a process of digitisation (e.g. here is what we are doing, now take this online) but also a process of standardisation, where employees of the client would align on their practices to a previously unprecedented level of precision.

I'm not sure whether this differs in other parts of the world, but in Australia it is very common for staff of many non-profit organisations to be on fixed-term contracts with no promise of extension. Many non-profits have concentrated revenue streams, e.g. they may be wholly dependent on government support in order to operate. On top of this, special initiatives like a digital transformation may take a back seat to the core purpose of the organisation.

We had a few projects like this, and they often had the same shape: a small handful of meetings to sketch out requirements with months between them. My first meeting was with a young man who emphasised that this would be a really small project. A group of people who would probably be sharing a single Android phone, we would just have to make sure it works on that device and we'd be all good. A simple app. Every client I have ever worked with wanted a simple app.

Back then I might have said they were wrong about what they want. Today I would use some of the terminology I've built out here to rewrite what they are saying into what they mean. Everyone wants a simple experience of using the app. This is one level removed from the complexity of the digital product on an objective basis - e.g. we could make an app simpler by having it do less. The work that developers do is one additional step removed - the complexity of the digital factory that delivers that product.

More meetings went on between our client and our agency that I was not privy to. There was a changing of the guard. New requirements - the one device claim from earlier wasn't true, they wanted a web application in addition to a mobile application, and they also wanted to be able to export data from the system via PDF. In good faith, particularly early on and when working with non-profits, we tended to accept these kinds of change requests, even though any one of these requests is not only potentially large, they could also affect the quality of any decisions that were previously made.

The worst was still to come.

A very important task in software development is forming the data model, which is how the real-world problem is going to be represented by data. One of the big problems on this project was that we were not able to model the data until relatively late in the process as our client owned the standardisation aspect of this initiative - they didn't know what data they needed, so they weren't in a position to tell us.

I provided a formatted spreadsheet to create some degree of structure based on what I understood about their needs, expecting a few A4 pages worth of questions to be filled out. What I received back was about 10x larger in volume than my most conservative estimate. More than that, the data was unexpectedly deeply-nested in how it is organised.

• maybe
  • you
    • get
      • the
        • picture
          • now

This would eventually turn out to be a big problem. The technical strategy for sending and receiving information over the internet was optimised for a flat structure, e.g. a bullet point list where no items had their own sub-bullet-points, and buckled only once this project had been in production for weeks and the total pool of data had reached some invisible critical mass.

We're not quite done taking inventory of fires.

From my first-hand communication with the client it was obvious that capturing images was going to be part of the digital product. What was not obvious was the volume. We had forecast that 20 images might be attached to a single case. We had forecast that users would be in metropolitan areas at all times. What happened in production instead was that users had no signal and were attempting reasonably often to upload over 300 images on a single case.

The requirements gathering process had monumentally failed to capture the requirements, and I was left holding the bag alone on a project that probably needed a budget 10x larger than it had.

Needless to say it was a pretty harrowing experience, and it's largely why the digital factory framework exists. Most people treat software as a static and highly commoditised asset, and can't understand why defects can't be remedied in a timely manner.

You can make the best decisions at every step in the process, leave everything out on the field and still walk away worse-off than empty handed.