SAFE Case Study – CMS – Approach to Scaling Agile Using SAFeยฎ

Nearly 140 million Americans rely on Medicare, Medicaid, the Childrenโ€™s Health Insurance Program, and the health insurance exchangesโ€”all programs administered by the Centers for Medicare & Medicaid Services (CMS). The agency pays out approximately $767 billion in benefits annually and employs 4,100 people to administer programs in partnership with state governments.

Challenge:

Isolated Scrum teams didnโ€™t make much progress within a deeply ingrained waterfall culture and against long-range planning and budgeting.

Industry:

Government, Healthcare

Results:

  • CMS shifted the budget from 100% dedicated to system maintenance to a 40/60 split between maintenance and innovation
  • Help desk tickets decreased by 55%
  • Surveys show a 27% increase in employee satisfaction

Best Practices:

  • Prepare for face-to-face events – CMS found the SAFe Implementation Roadmap and training invaluable to smooth-running PI planning events
  • Establish transparency – Stress the importance of open, honest discussion and engagement
  • Communicate the vision – In opening remarks at PI events, CMS reminded team members that their work directly impacts people’s health and lives

Introduction

Amid the pressures of increasing citizen expectations, the CMS environment is complex and ever-changing as budgets and legislation fluctuateโ€”making for a perfect setting to introduce Lean-Agile principles. A few isolated programs had begun using Scrum practices, but given the size and complexity of programs at CMS, Scrum did not lend itself well to longer-range planning and the identification and mitigation of dependencies among the Scrum teams. In addition, the organization still had cultural battles to overcome.

โ€œWe were still suffering from a โ€˜throw-everything-over-the-wallโ€™ mentality,โ€™โ€ explained Brent Weaver, Director of Systems Implementation at CMS. โ€œThe few Agile teams were requiring more of programs and that created more frustration on both sides. There was no vision or framework where everyone saw how they fit together. As a result, what they delivered was late, with defectsโ€”and not what the market needed.โ€

SAFe: Systems Thinking for a Complex Organization

In 2017, Weaver arrived with the charge of improving the Agile transformation for the Center of Clinical Standards and Quality (CCSQ) within CMS. In the search for a new approach, the Scaled Agile Frameworkยฎ (SAFeยฎ) resonated as the right option.

โ€œSAFe brought a much-needed approach to scaling Agile and systems thinking that was critical to an organization of our size and complexity,โ€ Weaver said.

In preparation to obtain buy-in and funding, Weaver built his knowledge of SAFe by taking some initial courses: Leading SAFeยฎ, and later, Implementing SAFeยฎ. Following the Leading SAFeยฎ course, he made the case for the Framework for leadership and earned the full support of Steve Davidson and Mark Plaugher, Directors of the Information Systems Group within CCSQ. Additionally, Debra Santos, Director of Hospitals, ASC, and QIO Systems was also willing to support the SAFe adoption for one of her systems.

SAFe case study

For help, Weaver tapped Scaled Agile Partner, Agile Six Applications, Inc. With Agile Six, CMS decided to implement SAFe first in a group brand-new to Lean-Agile concepts, rather than with those already using Scrum, for a chance to start from scratch. The first teams on SAFe would be those working on CMSโ€™s Hospital Quality Reporting (HQR) system, which healthcare facilities use to report data to CMS.

With leadership backing, they secured the budget and marked the calendar for the first face-to-face Program Increment (PI) planning eventโ€”to take place just six weeks in the future.

PI Planning Day One: Messy and Chaotic

To meet the timeframe, CMS decided to shortcut the recommendations from the SAFe Implementation Roadmap and skip SAFe trainingโ€”a decision that created significant challenges and that, in hindsight, they wouldnโ€™t recommend to other organizations. The fact that many team members were located outside the area, and many were contractors, played into that decision.

To help prepare for PI planning, HQR conducted a four-hour, half-day mock PI session with about 20 percent of team members to give them an idea of what to expect.

For the actual PI Planning event, CMS brought together more than 120 people, with approximately a quarter of them coming from out of town. The first day, unfortunately, proved to be chaotic and more challenging than expected for several reasons, according to Weaver and Ernie Ramirez, President of Agile Six Applications:

  • They underestimated the refinement status of the backlog and didnโ€™t follow all relevant parts of the SAFe Implementation Roadmap
  • They had a single Certified SAFeยฎ Program Consultant (SPC) in Ramirez (the recommendation is 3 โ€“ 5 per 100 development practitioners)
  • The agency skipped Leading SAFeยฎ, SAFeยฎ for Teams, and SAFeยฎ Product Owner/Product Manager training
  • They did not identify Value Streams
  • CMS simultaneously created the implementation plan and prepared for the Agile Release Train (ART) launch

โ€œIt cannot be overstated how horrible day one of that PI went,โ€ Ramirez said. โ€œWe didnโ€™t lay out an implementation plan as well as we should have, and the development contractor didnโ€™t have the resources or roles we thought they did.โ€

PI Planning Day Two: โ€˜Quarter-Million-Dollar Conversationsโ€™

Day two, however, could not have played out more differently. โ€œAt the end of day one, rather than throw in the towel, we rolled up our sleeves, and resolved to do better in day 2. We came out of day two with a plan that the teams would ultimately deliver on over the next 12 weeks,โ€ Ramirez said.

Ramirez points to a few reasons for the turnaround. After the first day, people returned knowing more of what to expect and came more prepared. Also, the two-day format created a sense of urgency to make progress. Additionally, Ramirez walked around troubleshooting any issues immediately as they arose.

โ€œAfter the first day, everyone had an opportunity to โ€˜sleep on it,โ€™โ€ he said. โ€œA lot of the frustration at the end of the first day kind of washed out and everyone came back with a renewed focus and commitment to get the plan done,โ€ Ramirez said.

Team members and program managers alike left the event more hopeful than ever before, believing they could actually hit the planโ€™s targets. Most promising, Weaver and Ramirez noticed productive discussions happening throughout the roomโ€”often between people who had worked together for several years, but had never actually met one another in person.

โ€œWe witnessed a lot of team and cross-team bonding that just cannot be replicated over WebEx, Hangouts or Zoom,โ€ Ramirez said. โ€œThere is something immeasurably valuable about being in the same room with someone, laughing, joking and yes, respectfully arguing. A lot of trust was earned and built on day two.โ€

โ€œQuarter-million-dollar conversations were happening all over the place,โ€ Weaver said. โ€œThatโ€™s what it would have cost to fix problems down the road if those conversations had not happened.โ€

SAFe case study

Communication, Collaboration across CMS + Contractors

Following that first PI, CMS began adhering to the SAFe Implementation Roadmap. They delivered Leading SAFeยฎ, SAFeยฎ for Teams, and SAFeยฎ Product Owner/Product Manager training. Unlike the first PI, they identified Value Streams.

โ€œFor the second PI, we found a lot of value in identifying Value Streams and ARTs, which helped people understand where they fit in and how teams fit together,โ€ Ramirez said.

Agile Six also delivered training to external contractors, including Leading SAFeยฎ, SAFeยฎ for Teams, and SAFeยฎ Product Owner/Product Manager. Several people at contractor organizations earned their SAFeยฎ Program Consultant (SPC) certification and began training their own peopleโ€”knowing that it is likely to give them one more strength to promote as they seek to win future contracts with CMS.

During RFPs, contract organizations routinely compete against each other. However, once on contract, they must work with team members from competing firms. As an unexpected benefit, SAFe helped unify CMS team members and contractors, as well as contractors from various companies. Face-to-face, they collaborate more effectively and come to personally know the people behind the roles, developing comfortable working relationships with each other.

โ€œItโ€™s fundamentally better for American taxpayers that teams work together and break those walls down,โ€ Weaver said. โ€œIโ€™m really proud of contractorsโ€™ ability to collaborate, share information, and work as a single team. Doing so has helped us reduce trouble tickets, so we know weโ€™re delivering higher-quality solutions.

Because of CMSโ€™s heavy use of contractors, each ART is comprised of people from numerous organizations. That required transformation leaders to be sensitive to job functions and responsibilities across the different companies on a single ART to foster trust and teamwork instead of competition. Having a single backlog for an ART creates further harmony among diverse team members.

27% Boost in Employee Satisfaction

So far, CMS has trained more than 200 people, including 25 โ€“ 30 Certified SAFeยฎ Program Consultants (SPCs). The agency has also since launched four more Agile Release Trains (ARTs).

With training and preparation, participants have been more engaged in PI Planning events after that first learning experience. Communication, says Weaver and Ramirez, has been critical to the acceptance of the new way of working. Especially in the early days, they had to communicate clearly and persistently to convince people to join in the effort and assuage fears about what this meant for their futures.

SAFe Case Study

โ€œWe really had to do a lot of selling on SAFe to get people comfortable,โ€ Weaver said. โ€œPeople were genuinely apprehensive about changing the way they have worked for so long, but as they have seen results, they have embraced it.โ€

And over time, HQR has implemented other SAFe concepts such as Weighted Shortest Job First (WSJF). Well ahead of a PI, the primary stakeholder has time to weigh the value of work and prioritizeโ€”which takes some of the emotion out of the decision, Ramirez says.

They are also in the process of adjusting budgets to fit more with shorter-term planning. Instead of years in advance, they began thinking in terms of three-month increments, in which Ramirez called a halfway step between the traditional approach and the โ€˜wild westโ€™ of Scrum.

Higher Quality, Happier People

After a bumpy beginning, CMS points to measurable progress:

  • Budget shift to modernization versus maintenance โ€“ Instead of 100% of the budget going to maintain the existing HQR system, now only 40% is dedicated to it. A full 60% of the budget goes toward innovation for the system, helping the agency deliver on citizen expectations.
  • Higher quality โ€“ The HQR group reports a 55% decrease in help desk tickets from hospitalsโ€”demonstrating a direct impact to customer satisfaction.
  • Happier people โ€“ Surveys conducted before and after SAFe show a 27% increase in employee satisfaction.

While CMS canโ€™t yet measure customer satisfaction gains directly, they know that fewer quality issues and more innovation contribute to that goal.

โ€œSAFe provided a map that enabled us to shift to modernizing versus just maintaining the status quo,โ€ Weaver said. โ€œBeneficiaries will ultimately benefit from more user-friendly, human-centered design systems, which will allow us to reduce the burden on our providers.โ€

The groupโ€™s success has caught the attention of others, with trains now starting in other CMS groups. โ€œOther programs within CMS have approached HQR asking us how to drive the same outcomes,โ€ Santos said. โ€œItโ€™s a testament to how far weโ€™ve come in the past year.โ€

  • Transformation starts with leadership โ€“ Ideally, you need two to three leaders who are fully committed to the change. If possible, send them through SPC training.
  • Coaches are a MUST โ€“ CMS found substantial value in them
  • Agile contracting is necessary โ€“ Rigid contracts that have highly specific deliverables can be an obstacle to agility and to embracing shifting priorities as new data emerges
  • Use contractors that understand Lean-Agile principles โ€“ Hire teams that truly understand what this means, not just those who can talk the talk
  • Find collaborative work space โ€“ From PI planning events to day-to-day work, collaborative work space enables teams to capture the value of face-to-face interaction
  • Just do it! โ€“ โ€œIf we could time-travel and do it again, we would emphasize a sense of urgency to get going,โ€ Weaver said. โ€œSet a near-term date and follow the roadmap.โ€
  • Engage employees โ€“ Any effort is only as strong as its people. Approach the change with empathy for what your team is undergoing and leverage the support of management and coaches to keep employees engaged and excited.
  • Start with Essential SAFeยฎ โ€“ CMS found it valuable to simplify as much as possible and started with a program that lent itself to Essential SAFe. The learnings they achieved will influence larger programs, which will require multiple Value Streams.

Back to: All Case Studies

Suggested Case Study:

NHS Blood and Transplant