Air Traffic, Autonomy, complexity, Corporate Career Path, Limits of Expertise, Limits of Knowledge, Optimization, Professional Fulfillment, Sequential Planning vs. Adaptive Emergence, Subject Matter Expert
So you think you’re an expert and pretty much have a handle on your domain and the keys to a comfortable ride through life? What if waiting for that 3% raise a year is a form of subtle imprisonment? What if there is much more satisfaction in seeing your life and career as the cumulative list of things you still have to learn rather than as the accumulated knowledge that keeps you safe and comfortable?
My own story starts at Lockheed Martin’s air traffic systems business unit, where I was living the life of a young engineer, trained in analytical thinking, and being part of a huge “machine” operation that churned out software for safety critical applications – in this case air traffic management. I was at the bottom of the hierarchy but optimism was high: I had been selected to partake in the three year long Leadership Development Program (LDP), specially designed to groom Lockheed’s future managers and leaders. And I had even gotten a few 8 and 10% raises a year, so the compounding looked great.
At the time (cca. early 2000’s) the entire air traffic management community was looking at designing the next generation system based on an ideal vision of a future with non-existent flight delays, increased flexibility for airlines, reduced flight costs and pollution.
Much of the focus to achieve these ambitious benefits was based on a trajectory management paradigm. The community envisioned a highly intelligent algorithm called alternatively “The Optimizer” or “The Evaluator”, which would provide the means for an optimization between the global air traffic system’s constraints and airlines’ (flight) trajectory preferences. And so it was that I become Program Manager for a number of research and development programs at Lockheed and decided to try to build a prototype for this Optimizer, while everyone else was still debating whether such a thing was even possible.
We started with a team of senior subject matter experts (former pilots, etc.), mathematicians, computers scientists and lots of state of the art technologies (agent-based simulation, distributed computing, etc.). I even brought in high level mathematicians to help with the selection of mathematical optimization formalism and eventually settled on a cybernetic model for the architecture of the optimizer. Money was in no short supply since this was the pre-2007 era of business optimism, so work commenced rapidly. While my team was busy building the multi-objective national airspace collaborative optimization or MONACO prototype within a cybernetic optimization mind-set I myself had defined, a paradox started to form in my mind.
Here is a brief description of my apparent paradox at the time: any optimal plan P involving a set of deconflicted flight trajectories had to be in existence for a finite time period and for a certain geography. As circumstances evolved (weather fronts, etc.), plan P would eventually have to be replaced with plan P+1. But flights would be taking off and landing continuously throughout both plans. So many flights would take off in plan P but land in plan P+1. So how could a trajectory be part of two sequential plans that imposed different (and quite possibly mutually exclusive) constraints on individual trajectories at once? And since it wasn’t just one but many trajectories that would fall in this category, how could this continuous nature of flights be reconciled within a discrete-sequential planning context?
The paradox remained unsolved at the time, and it took me another several years and running into complexity science to acquire the mindset necessary to overcome it. In the context of assimilating complexity principles and particularly the behavior of complex-adaptive systems, I realized that a cybernetic control model based on a sequential optimization process simply does not work in a complex-dynamic environment, hence the apparent paradox.
I had reached a paradox because I had tried to apply the wrong thinking to the right problem. Overlaying a complexity mindset on the problem drives the realization that in fact the cycle of the optimization plan is so fast that “sequence” gives way to “morphing”, i.e. a continuously adaptive process where order is an emergent rather than prescribed phenomenon. This in turn requires adaptive mechanisms such as autonomous self-separation technologies on aircraft rather than trajectories that are deconflicted from end-to-end before the aircraft ever takes off. It is sufficient to look at nature to see how self-separation and autonomy beats any optimized control paradigm – after all, geese don’t run into each other at take-off even though no one apriori deconflicts their trajectories.
Since then, the entire air traffic management community has given up on the Evaluator prototype only to resurrect the cybernetic centralized control principles in a new paradigm called Trajectory Based Operations or TBO (you guessed it: optimizing a huge number of trajectories before flights take off). The community has however remained vague about how the optimization would happen under TBO, which avoids for the time being the paradox I ran into more than 10 years ago.
With a hands-on appreciation of complexity, I have since expanded the application of complexity principles to other complex socio-technical systems besides air traffic management. However I remain grateful to air traffic for exposing me to a paradox to which I owe my freedom of thought, and starting a journey that has taken me from the corporate office to entrepreneurship in emerging markets and to the leading edge of strategy in the process.
While many experts are continuing to work on various concepts and “doing things right” in Drucker’s terminology, it takes more than being an expert to ask Drucker’s “are we doing the right thing” question that can lead to paradox and eventually to the freedom to accept that which is beyond expertise and experience: the yet to be explored unknown. This is the source of true professional fulfillment and even material wealth beyond the 3-10% yearly corporate increase, that is if you are interested in that sort of stuff. Drucker’s distinction has another quantitative vs. qualitative connotation within this example: it usually takes many experts to do things right, but only one mind asking a tough question to shift an entire industry to the right problem to be solved.It is also highly likely that the tough question will not come from one of the existing experts who are usually too close to the problem at hand and have a vested interest in keeping the perspective from changing.
The recipe to free one’s mind may well be this: ask yourself four or five consecutive “why” questions starting from the most trivial subject you would like (by the way, children are really good at doing this: why do you go to work?). Once you reach about the fourth or firth why question in the series, you’re sure to run into your limits and if you’re fortunate you’ll even run into a paradox. The end of certainty is where the fun really begins.
Now you may understand my smile when I get asked the question of why I am not more careful with disseminating what could be very valuable ideas: it is because I know that no-one can really steal the context of an intellectual journey. They cannot replicate my neural pathways that took years to develop, forged at the intersection of inquisitive thinking and action that intertwined to form a pathway as unique as human DNA. A more interesting question is how much of the complexity mindset can even be taught.
My final point has to do with the price of intellectual freedom. It is neither easy nor comfortable arguing against the current, and one has to be prepared to hold steady as criticism, ridicule and even job security may be at stake. I can attest to this from my own experience as an “expert” at challenging the validity of other experts’ time-tested thinking.