A large part of my career has been spent vacillating between what I would call entrenched platform teams with increasingly vague notions of how their solutions are utilized and scrappy reactive startup teams hobbling by on bailing wire and duct tape solutions. I’m usually the person sitting on the fence trying to balance the situation out, but some situations are beyond repair. That said, I spent a large part of my early career being in the former camp, until slowly over time, and through owning my own business, I started to realize that part of the utility of software itself is the ability to move fast. Ask someone in microprocessor manufacturing sales where the lead times for development are 3-5 years if they would be happy about being able to push out a product in two months. The tendency for engineers to fall exclusively into the former camp comes from a number of factors, not the least of which is cultural bias amongst their engineering peers.
So the question is how do you bridge those two modes and create a funnel for long term investment which doesn’t divorce it entirely from the opportunities that the business needs to pivot to chase while still creating foundations that can be grown upon and make an impact in the long term?
A lot of us are familiar with the concept of skunk works projects. Companies can embrace this mode of working, and usually reserve it for those who are closest to the customer, data analysts for example. But the universal attitude of engineering towards these projects is that they are “throw away”. IE once the engineering team gets in there, there is almost nothing salvageable that can be leveraged, the solution is so fundamentally flawed that no “proper” engineering can be applied to it, they have to go back to square one.
A more conventional way of solving this problem is through what is referred to as “product development”. This is a process of learning the purely economic and business strategic dimensions of a project and collaborating with engineering to make sure that the two are aligned. The problem frequently with this approach is that product development doesn’t usually have the vocabulary to be able to speak effectively with engineering and visa versa that they are frequently talking past one another.
Another problem is that just the sheer weight of additional process prevents the business from being able to move quickly in reaction to macro-economic fluctuations that affect the business. You don’t want to be purely reactive, but you also don’t want to put your head in a cave and watch the world go by. At the end of the day you want to be able to benefit from both of those mindsets.
These problems become even more exaggerated when you start to operate in a complex space, such as one with heavy data science or machine learning. In these environments, data scientists and analysts start to function more properly like the product development team because the context is too complex for product development to be effective at it. Now we have three parties, product, engineering and analysis from three distinct disciplines speaking different languages.
What’s more, treating the results of data science or machine learning projects as just one offs or skunk works projects is sometimes not possible because engineering usually doesn’t have the capability to start back at square one with those projects, it may be too complex or involve mathematical machinery that’s not relevant to engineering experience. Platforms specifically at this intersection like Tensorflow help with this problem but it eschews a rather narrow path through the space. With engineering being forward focused and thinking about secondary and tertiary effects of engineering decisions engineering teams usually stand distinct from the other two, creating an ever widening gap to analytics and product. The problem becomes exaggerated when engineering teams have their own product development team, which usually has the counter intuitive affect of widening the gap, and entrenching / defining the platform to an exaggerated degree making it drift further and further away from the opportunities which sustain the business. Gasoline to the fire here are various biases (superiority complices?) about throroughness and attention to detail that engineers tend to hold up as a crutch or as an integral part of their identities.
This is where I introduce the idea of “Opportunity Engineering”. Engineers who have a dual purpose of extending and building out the platforms which power the team and working with platform teams to extend their work in measured ways while simultaneously helping to build short term opportunity projects which are not simple one offs but build on and extend the long term vision. If the engineering team and platform are a cruise ship, then the opportunity engineers are the tugboats using engineering to pull them gently making small course corrections to ensure that they are veering into the right dock at the right time. Opportunity engineers wear many hats, sit on the fence between several teams and move rapidly between ideation, implementation, maintenance and extension.
Unlike solutions engineers who are often times extensions of a sales team, the opportunity engineer is a full engineer, capable of working at any level of engineering complexity, but with the specific mission of working projects which accelerate the companies ability exploit opportunities which may otherwise be unattainable by traditional engineering teams. It’s often said that engineering teams need to keep their feet on the ground, whereas sales and product function to pull the company into the clouds, but what this often causes is a misalignment problem, a bimodal working environment in which engineering teams are locked into tunnel vision exclusive prioritizing its own needs, leaving end to end teams to fend for themselves. Over time the gap widens and the only thing to resolve the problem is total reorganization. It’s the job of the opportunity engineer to serve as an elevator between the two to make sure that one is never veering too far away from the other. While its traditionally thought of as the job of the product development team to do this, there are some environments, such as complex machine learning or data science environments where this is practically impossible without having someone with engineering experience embedded on those teams.
Additionally having an engineer function in this space gives freedom for other engineers to focus on the well-defined and more certain tasks in the near term knowing that they don’t have to worry as much about generalization. An opportunity engineer can observe that the business requests fall into a certain pattern of behavior and then advocate for extending the platform to have degrees of flexibility which will serve the business’s end goals. While it seems like have a specific focus on this aspect could lead to over-generalization it actually has opposite effect. Because the opportunity engineers are closer to the opportunities of the business, they know which generalizations are likely to bear fruit for the business. Platform engineering teams on the other hand are often making best guesses of what dimensions to generalize based upon direction from product and trying to “read the tea leaves” of where the business is going based upon second hand signals. This then frees up the platform and product engineers to only build what’s been well defined and laid out before them without having to worry about whether or not its well aligned with the chaotic and ever changing business environment around them.
One thing needs to be clear though: An opporutnity engineer wouldn’t just be shooting from the hip and chasing butterflies and shiny objects. They would be deliberately exploring the space around the team like a reinforcement learning algorithm in an “exploration” mode freeing up the rest of the team to be in the “exploitative” phase of running with what they discovered.
Ultimately every engineer needs to develop a product mindset and orient themselves as a kind of opportunity engineer, after all, you have to know how the bread is getting on the table in order to make sure you are making the right moves to ensure it keeps happening, however, not all engineers have a preference to work in this way. Therefore we should exploit as much as possible the engineers which are willing to be scrappy and teach them how to think long term so that they can bridge the rest of the engineering team to the business and keep their efforts honest and well oriented.
At the end of the day all engineering is opportunity engineering, but it also takes one of every kind to make a ship float.
~ Kyle Prifogle