Recently, I asked IBM’s Distinguished Engineer Jan Gravesen, who will be speaking at the upcoming L.A. Digital Government Summit, to share his thought on how implementing a Hybrid Agile approach could enable government legacy transformations. Jan has written some incredibly pieces; here’s a few of my favorite pieces over the years: Creating a Culture of Innovation, A macro-pattern for public sector systems architecture, What defines success with public sector enterprise architecture?, and CalCloud and IBM: Cloud and Government for California. He has also given some really awesome technical presentations, like the one he gave with Public Sector Partners on “Adoption Patterns for Cloud Computing.” Obvie; I have listened and read his works, which is why I am a fan, but Jan is a talented Executive Architect, Client Technical Leader for California, and a member of IBM’s Academy of Technology. So here’s our interview:
What is Hybrid Agile? What are the differences between Hybrid Agile, Waterfall, and Pure Agile?
Jan Gravesen: What we call Pure Agile was born out of a movement that re-established the software developer at the center of software projects and “working software” as the truest measure of progress. This is a very practical and, frankly, appealing perspective, especially if you are software developer. For a long time, developers felt increasingly marginalized. As a former software developer I can recognize and sympathize with this view.
We can look at [Pure] Agile, Hybrid Agile and traditional Waterfall as different styles on a spectrum. At one end of the spectrum we do a lot of pre-planning and design before we begin to code and while we are coding we have a lot process tasks around project management and risk management. This is traditional waterfall approach. At the other end of the spectrum, we do very little pre-planning and design, just enough to get us going in the same direction, but during the coding we continually and fluidly refine the design, and to some extent, the plan and we only have some light-weight tasks around project management so most of our resources are centered on coding. When we need to change we have self-organizing processes in place for doing this but how we organize or improve our performance are really decisions that are left to the team. This is Agile: a way of working where the development team is self-organizing and self-adjusting, where it is the developers that form the core of the project, not the project manager or architect, and where rigorous management is substituted by light-weight but still structured processes. Somewhere along the spectrum in between agile and waterfall we find Hybrid Agile – this is a style where we use Agile at the core of the project but since we may have several Agile teams working in concert we use some traditional, usually condensed, project management disciplines to ensure the teams communicate and stay synchronized, that we can handle suppliers or external risks and so on.
What are some reasons a government would implement a Hybrid Agile approach over a traditional Pure Agile approach? For example: When would you would implement hybrid, agile, waterfall approach. Essentially, what are some of the common trade-offs governments must make when considering a hybrid agile, waterfall, or pure agile approach?
Jan Gravesen: A good example is when we have a lot of Agile teams that are working on different parts of the same system and when we need to coordinate them. There is still some debate as to how well Agile is able to scale up to lots of teams, especially if the teams are geographically separated over large distances or time zones (as is often the case with very large or complex projects). The Agile community has come up with some extensions to Pure Agile, such as Large Scale Scrum (LeSS) or, when Agile is applied on an enterprise level, the Scaled Agile Framework (SAFe).
Another example is where we have lots of external factors and risks we need to address in the project. When risks and factors scale up, it becomes difficult for an Agile team to address them simply through self-organization and besides if the team had to address them, the team would get bloated and potentially too large. A better option is to stay true to the Agile Manifesto and continue to use smaller, focused teams but surround these teams with one or more Centers of Competence that carry out the project management disciplines (a la supplier, risk, finance, change management) we need, so we still allow the Agile teams to work using the agile style.
On the topic of when we would or should use hybrid, agile or waterfall, I would say the jury is still out, although there are plenty of pundits that will promote one style over another for all circumstances, even with near religious fervor. However there are some boundaries that historically have been generally accepted.
When we for instance are dealing with life-critical systems such as air traffic control systems or space mission control systems, or with embedded software in high precision equipment such as MRI scanners or oncology radiation equipment, most companies maintain a strong preference for waterfall, because it lends itself to a discipline that have been practiced since the 1960s and was pioneered by NASA, called Systems Engineering & Architecture (SE&A). We can call these types of software systems ‘systematic applications’ because they simply can’t be allowed to fail. Conversely, when we deal with systems that change fast and where we continually and fluidly need to integrate new changes, such as web store fronts, or when we work closely with end users in a prototype or spiral way of working, we generally find that Agile is really well suited as the preferred style. These types of systems we can call ‘opportunistic applications’ because they have to be able to react opportunistically to change. But these are just the two ends of the spectrum – in between we find a lot of everyday systems that drive everyday business, such as ERP systems or systems of record, and many of these are quite well suited for either waterfall, Agile or Hybrid Agile. In fact, although it is not practiced so often, yet, I find ERP projects as being in very good harmony with the thinking behind Agile. What we are seeing more of, of course, is that we have very large systems – so called, systems of systems, that – if they fail – could have large short- or long-term socio-economic impacts, for instance Medicaid management systems or large social service eligibility systems. Many of these systems of systems are composed of many discrete systems, some of which are opportunistic and some of which are systematic in nature.
Can you provide 1-2 considerations / cases in which you worked on a legacy modernization project using Hybrid Agile, for this would be super helpful for governments seeking to understand when and how to apply Hybrid Agile to legacy modernization?
Jan Gravesen: A typical case is when we use the strangler pattern to modernize a legacy system. The strangler pattern is often suggested to lead to complete strangulation of the legacy system, but when we look at how the pattern has been applied in finance, insurance, health, and transportation for almost 20-25 years now, we find that the legacy system usually remains in one form or another, so we don’t completely get rid of it. Instead we shrink [the legacy systems] footprint and make what is left non-volatile and systematic; we also encapsulate it in modern APIs or services and begin to build new code on top of it and around it.
Moreover, when the legacy system is large and when there is a lot of knowledge accumulated in it, we often run the risk of not being able to identify all this knowledge or transferring it to the new code, only to discover our modernization has led to an “incompletely modernized system.” Why does this happen? Well, agile purists claim that waterfall produces too much documentation but in reality, in many many cases, legacy systems are not very well documented and their documentation hasn’t been maintained over the past many years. So what’s the consequence for Pure Agile here? We now have to build some application discovery into our sprints, simply to ensure that we bring all of the necessary functionality (and knowledge!) with us into the modernized system. Just relying on users and user stories may be insufficient because the users may be stratified into so many different personas that we can’t represent all of them in the sprint reviews.
Another thing to consider is that legacy system can be quite large and we may need to modernize them using multiple Agile teams. Another good case is when we use a COTS software package such as an ERP system to replace a legacy system. The COTS package already contains most of the code we need, but we still need to do a lot customization of workflows, screens, interfaces and so on, all of which is well suited for the hybrid style, and in fact both SAP and Oracle now support the hybrid style in their respective methods.
Awesome! Since you mentioned methods, what do you think of the Managed Agile Development Framework? And, is this a framework governments should consider implementing into their workflow?
Jan Gravesen: The Managed Agile Development Framework (MADF) is one among a growing number of hybrid frameworks that recognize the benefits and value of Agile while also recognizing that there is a need to perform disciplines that Agile don’t address. Other examples are LeSS, SAFe, Disciplined Agile Delivery (DAD), or Agile With Discipline (AWD). While SAFe is quite rigorous, and while I can certainly understand why many Agile purists would balk at it, the MADF, DAD, LeSS and AWD frameworks are all quite light weight. What’s important here is to ensure that the resulting project team – and remember that team may be composed of multiple Agile teams – are all trained in the methodology and that they accept the broader scope of the project and the measures in the two concentric circles.
And yes, I think governments should be more mindful of using Hybrid Agile. Right now there is a gravitation toward Pure Agile and to most government organizations this is quite exciting and even a little radical, but we have to remember Agile has been used broadly in many industries since late 1990s, for a good twenty years now and there are lot of experiences that have been accumulated concerning where the limits are with Agile. I think governments should be more interested in learning from these past twenty years. As with any maturing person or organization, there is often a growing realization that things are not black or white. If we say that Pure Agile is white and pure Waterfall is black, then in between there are a lot of shades that governments should really consider, especially for very large projects where they often will find real constraints, or limits, with Pure Agile.
According to some (maybe many) agile purist and some empirical data, Hybrid Agile projects are more likely to fail than Pure Agile or Pure Waterfall. What are some ways to mitigate the risks of failure when starting a hybrid agile project? For example, what high-level or specific metrics should governments track when implementing a Hybrid Agile approach?
Jan Gravesen: First of all, have to we recognize that projects still fail at a staggering rate irrespective of whether we are using agile or waterfall – numbers are reported around 50 percent by a variety of different sources and this number has stayed steady over time. We may actually suggest that we haven’t really become better at executing large projects during the past 50 years. This is where my perspective perhaps deviates from that of both the Agile purists and the disciples of PMI’s PMBoK or CMMI: Projects often fail because of inadequate ability, both on the part of the organization and on the part of the software vendor that may have been chosen to implements the project on behalf of the organization of they fail if one the two parties play moral hazard. But projects, especially large scale projects, often fail for two very important reasons that have nothing to do with whether Agile or waterfall are used. One reason is what we can call strategic deception. Another is optimism bias. This has been quite well documented by Prof Bent Flyvbjerg’s team at Oxford University.
But coming back to the premise of the question. A recent survey by HPE did suggest lower success rates for Hybrid Agile than for pure agile projects while a different survey by CAST suggested the opposite. More data is obviously needed to settle but one difference within the data sets could also be that hybrid agile, as I have alluded to, could be larger and more complex, involving more teams and special performance, regulatory or security requirements, or multiple external suppliers. All of this drive up complexity and risk. What we really need is a measure of project complexity and risk and then analyze Pure Agile vs Hybrid Agile relative to different levels of complexity and risk. Experienced software implementation firms, like Accenture and IBM, usually apply an actuarial approach to risk and complexity rating and evaluate whether projects can be executed through agile, waterfall or hybrid, and this approach is really one any organization should use. I bet Pure Agile will shine for smaller, less complex projects but as complexity increases we will see Hybrid Agile catching up and eventually overtaking Pure Agile as being a more efficient style of working. That said, one challenge with the hybrid approach is that it can cause a contradiction in culture: Agile purists may well resist the need for overall project management because they are unaccustomed to it or because they don’t believe it’s necessary. Conversely, traditional project managers and architects that are brought up with traditional waterfall thinking may not realize or truly understand the freedom that must be bestowed to Agile teams or the self-organizing nature that is essential to Agile. They may instead try to interfere and regulate how the teams work, causing a contradiction. The answer to this is more training and more respect for each other’s viewpoints and for the overall needs of the project whether these needs are predicated on fast progress and flexibility or on tight financial and supplier control. But part of the answer is also structural: Instead of using hybrid in a way where there is a Project Management Office (PMO) that manages the Agile teams in a traditional hierarchy, the PMO should instead be organized as a network. It should focus on coordination and synchronization, among the teams and between the teams and the external environment where risks and constraints exist. It should offer the Agile teams different centers of competence, for instance around managing external suppliers, managing finance, managing organizational change and so on.
In terms of metrics, I like the core Agile metrics; I like the burndown charts and the measures for team velocity. These are incredibly healthy and spurs the teams to continually improve their performance through retrospectives. But we have to be ready to combine these core metrics with broader metrics that don’t interfere with them or degrade them such as earned value or Net Present Value (NPV) or risk metrics. We can look at this as two concentric circles. In the inner circle we have traditional agile metrics and in the outer circle we have broader project metrics – I prefer starting with the inner metrics and then move on to establish the outer metrics and when we do that we need to ensure that the outer metrics don’t interfere or are in conflict with the inner metrics. This is an often overlooked impediment in Hybrid Agile.
What are the benefits of implementing a Hybrid Agile approach? Does it improve culture? Do you have an example of the benefits of Hybrid Agile?
Jan Gravesen: So the benefits of implementing a Hybrid Agile approach is that we can combine what works in traditional, well understood project management with what works well in Agile. If we have large scale projects with lots of teams working at the same time, or if we have a lot of external suppliers or a lot of external dependencies and risks, which again is quite common in large scale projects, then there are well-established project management disciplines that deal with this coming out of the big structured frameworks, such as PMBoK, PRINCE2, or CMMI. Scrum doesn’t really per se do all this, and nor should it, I would argue. Also, if we have strong financial management requirements, for instance around earned value or cash flow or if we need to hit certain NPV or ROI points during the project in order for it to continue, we find well established financial management disciplines within traditional project management. Scrum, XP or any other form of Pure Agile do not really concern themselves with these disciplines and implicitly assumes there either is no need for them – which may be true for smaller projects –or that that they are taken care using other means.
According to the Agile Manifesto, which is at the core of Pure Agile, “working software” is seen as the truest measure of success and progress, but of course many organizations look at progress or success in much broader terms, which may include financial results or the ability to effectuate change in the organization at the same time that the project is being developed or by handling broad risks and regulatory constraints that affect the project. This is where the hybrid approach shines: when an organization is able to formulate a multi-faceted definition of what progress or success is – beyond just working software - and when at the same time there is a need to work flexibly and implement code changes fast.
Can implementing a Hybrid Agile approach save governments money?
Jan Gravesen: I guess the truest answer is that it can because hybrid can mean the difference between success and failure for complex projects, and we know that nearly 50 percent of all projects fail and 16 percent of them fail really bad. But I think the greatest benefit lies in the ability to combine two objectives that seemingly are opposed to each other if we only use waterfall or only use pure Agile, namely the ability to integrate coordination, portfolio, supplier and financial management with a group of Agile teams that each flexibly can integrate new changes into their part of the overall system. With pure waterfall, we often introduce too much up-front planning and analysis, and documentation, before we even begin to code because we are afraid we might introduce risks or costs we can’t control. Gradually, as the project becomes bigger and has more earned value, we become more and more risk adverse, and the project becomes more and more stale. Conversely, in pure Agile, we often neglect to consider how important it may be or how time consuming it may be to do supplier, risk, finance management, or change management. Hybrid brings these opposing into perspective together under a single, integrated method.
What is the MIT systems maturity model / method? And, how does it relate to Hybrid Agile? Essentially, how can hybrid agile further enable or progress digital systems maturity?
Jan Gravesen: Well, in 2003 MIT’s Center for Information Systems Research (CISR) published an article that discussed strategic architecture maturity. The CISR team, mainly …., looked at a lot of empirical data from tons of organizations and they suggested that an organization’s architecture maturity, and this was measured across all its various departments and systems, could be at one of four levels.
Level 1: Most organizations start out with a silo’ed architecture, where every new project builds a new system and in that process often decides which middleware to use, so the organization ends up with tons of systems and these systems use a lot of different middleware and hardware. This is a level of maturity that fits really well with Agile. It is also a maturity level that ends up being very expensive for the organization because there are few scale benefits and a proliferation of systems and middleware standards – so we find all the usual uneconomic characteristics that often accompany fragmentation.
Level 2: At the next level, organizations try to come to grips with the high costs and they introduce standards for middleware and hardware that everyone have to follow. This level still works quite well for pure Agile because the Agile teams care about highly productive tools for doing continuous planning, continuous testing and continuous release – in fact this level may force more emphasis on using DevOps.
Level 3: At this level, things begin to get a little complicated because the organization now begins to force end to end business processes and in order to do this the processes have to work on top of a rationalized information architecture. Now we have to begin to look beyond the individual system, project or even department to understand how everything fits together across the organization, and that requires some increased measure of design and planning, and some measure of portfolio coordination. We take decision power away from the projects.
Level 4: At the last level, we have modularity. This is where we come full circle and allow individual departments and projects to build their own services, not systems, on top of the core of information and processes we established during the previous level.
So as California's CTO, you've seen the government apply an agile, modular approach to procurement in the state. How has the GovTech marketplace changed with this new agile contracting and modular procurement approach? And, given your previous commentary, does Hybrid Agile have a role in new modular development?
Jan Gravesen: The market has changed, but it is not done changing. It has swung from one end point – the big monolithic contracts that are awarded to a single bidder – to another end point where public agencies divide procurements into a large number of smaller modules that are tendered to individual vendors and where each vendor is measured on how pure an agile method they can apply. In doing that they substitute one set of challenges for a different set of challenges creating an inflexion point for global governments.
One of the things we have learned from experience is that not everything scales – for instance, when we use “scrum of scrums” to coordinate Agile teams, there is now experiential data to suggest that it works well up to five teams and then it doesn’t scale well after that. Perhaps we will see something similar with modular procurement – perhaps the approach scales well to 5-10 modules. At this point we don’t know. Another uncertainty is that we don’t know if the modular approach coupled with Hybrid Agile will work well within organizations that are not used to working with Agile. Usually it requires that the organization that applies agile on a large scale is used to working with Agile — so it is better to start small and build the experience before scaling up to medium-sized projects and then finally scaling up to large projects. The projects we see in public sector are quite large and they have dependencies to functional units in the organization, or to other state/local agencies or counties. If these other stakeholders resist working with Agile, we will see friction begin to creep into the projects. Large public agencies are in the early stages of a process where they have to find the limits for modular procurement and the balance between the inherent advantages and disadvantages, and eventually they will.
To the second part of your question: Hybrid Agile works well with modular procurement, and in some key ways much better than pure Agile. Modular procurement drives a need for cohesion and orchestration at scale that simply is not addressed with Pure Agile methods, such as Scrum or XP. Frameworks such as LeSS or SAFe, and several others, address this challenge head on, not by ignoring it or diminishing it, but by addressing it through disciplines that mesh well with Scrum without suppressing the basic ideas in Scrum. So yes, Hybrid Agile works well with modular procurement but preferably when it is supported by mature disciplines that complement the Pure Agile methods and where it recognizes that the project isn’t done in a vacuum but has many dependencies and interfaces to other units or stakeholders that may not work with an agile mindset or method. Organizations that try to make it up as they go will find that the lessons they learn can be costly and in many cases quite unnecessary. The IT industry has been mindful of them for a long time already and has been keen to address them so there is a lot of learning out there that has accumulated over the past twenty years.
Government innovation agency 18F has done some great work around providing mini roadmaps for legacy transformation using an encasement strategy. To this end, what advice, if you had to provide a mini roadmap per se, would you give the technical staff seeking to undergo a legacy transformation and / or digital transformation and wanted to leverage Hybrid Agile?
Jan Gravesen: There are really a couple of things we have to be mindful of. One is to ensure that your technical staff understands the true measure of progress and success for your project. Think in terms of the two concentric circles. Team velocity and productivity is important but there are a lot of other measures that have to do with risk, completeness, earned value, and strategic fit that are largely ignored by Pure Agile. If you have a lot of developers that are well versed in Scrum or XP, that’s fine, but your Scrum Masters and Product Owners should really take a greater interest in these broader measures and the other disciplines that must be performed around financial management, supplier management, coordinating the different teams, risk management and change management. Of course, you can decide that these disciplines are not needed but then you also have to live with the consequence that you may end up with a lot of working software but no firm grasp on when the project will end or what it will end up costing or that it will not be accepted by its organizational stakeholders. Project audits that look into why projects fail can be quite harsh after the fact and one of the things they always consider is how well the product owner was able to anchor the project into a broader organizational context.
The other thing is that [they] should devote some time to ensure conceptual integrity. Conceptual integrity is really important in legacy modernization but also in very large and complex systems, as well as, of course, in systems of systems – we often risk doing the modernization in a way where the end result is incomplete. Conceptual integrity is a measure of how well the various pieces of the project fit together after it has been developed by a group of agile teams; and it must be a design objective that is independent of the method we use to build the system. Without conceptual integrity we typically get higher and unnecessary duplication costs and we find that the final system is more costly to maintain. Scrum and XP actually address this topic to some extent and when we go back and look at what the founders [of those methodologies] thought about this, we can see that they recognized the importance of having the Agile teams move in the same direction, not inventing everything anew, not using incompatible standards or technologies and so on.
As with anything in life, it is about striking the right balance. It’s not an either-or choice between agile and waterfall but rather a both-and, and we need to see all this on a spectrum with a lot of shades. The hybrid approach should not constrain or drown out the Agile core or the Agile thinking but it should work in complement to it, on top of it. Conversely, the Agile teams should recognize they need to work together, across teams, in order to ensure conceptual integrity and that there are broader measures of success and progress than they may immediately acknowledge or be concerned with.
Disclaimer: The opinions expressed in this piece are not official IBM opinions.