Scrum is popular as a jumping-off point for agile software development because, experts agree, it’s lightweight and easy to grasp. Because it’s a framework and not a methodology, it’s more empirical in nature than prescriptive, and it looks more at the process of creating software than at the engineering required to create it.
Despite its relative simplicity, agile development experts agree that certain bridges must first be crossed before an organization can implement Scrum (and other agile methods) successfully.
Scrum is best utilized for developing software products that would otherwise fail using a more traditional approach to project management, according to Victor Szalvay, CTO of the Scrum business unit at CollabNet. Among the factors is the complexity of the project.
“Are requirements changing? Is the technology squishy? Where you’re injecting uncertainty, the project becomes non-deterministic, and that’s where Scrum is the best fit,” he said. Projects that are simple, highly repeatable, and with little risk and uncertainty, can be done using more traditional “waterfall” methods, he added.
After an organization makes the decision to move to Scrum and agile, it must first have an understanding of what it’s getting into with the framework before trying to implement it. Moving to agile development, and particularly Scrum, involves a major change in the way development shops work. As the old saying goes, “People don’t resist change, they resist being changed.”
One of the first keys to gaining the productivity benefits that Scrum promises is to get total buy-in, whether from a small team doing a pilot project or an organization-wide shift. The experts interviewed for this story agreed that it is important to have both bottom-up and top-down support before beginning to work this way.
“You have to understand when you move to agile methodologies, there will be a learning phase that should be accounted for in the planning,” said David Vidoni, director of product management at BPM software maker Pegasystems, which transitioned to Scrum and agile development about a year ago. “In situations where you don’t have buy-in, that’s where you run into problems.”
Ideally, an organization will have a team of developers that wants to work in a Scrum and agile environment, along with a higher-level executive who understands the benefits to the business and will champion the effort. “Bottom-up alone will get squashed because the higher-level individuals will feel threatened, while top-down doesn’t work because there’s too much organizational change involved,” said Szalvay. “It’s best if it spreads organically; people will want to do it.”
A top-down approach, according to Eli Lopian, CEO of Typemock, tells people what the organization wants, but usually lacks guidance as to how they should behave in their new roles. A bottom-up approach is better, he said, because the developers want to learn the practices, get the tooling and become a more responsive, professional team.
But for this to succeed, “You need a very strong champion,” he said. Typemock is a provider of commercial unit-testing tools that has adopted Scrum and agile development.
Once the decision has been made to use Scrum (either on the team level or throughout the organization), it is important to set a baseline as to what it means and what everyone’s roles are. Because of the changes in how people work, Scrum is as much a human resource issue as it is a technical one.
“You’re talking about changing where people will sit, who they will be working with and how they work, and you need top-level support for that. But you need developer support as well. They might have had a bad prior experience with agile, or a fear of the unknown,” said Szalvay.
Training is important for success, the experts said. Vidoni said that when the company transitioned to Scrum a year ago, it brought in Scrum creator Jeff Sutherland and put the development team through two days of training to get a baseline. This way, “at the outset, everyone had a level understanding,” he said.
Getting the team up to speed at once is important, agreed Richard Cheng, managing consultant at Excella. “Bring in someone—either coaches, or with hiring—who knows what they’re doing. People who really get agile generally come to agreement as to how things should be done.”
Szalvay recommends starting with a pilot team to win support of the higher-ups in the organization by demonstrating success. “Pick a high-visibility project, where you can allocate people to a cross-functional project team,” he said.
“There can be no silos. You need all the roles represented [in the project] in real time. Then, you try to demonstrate the same level of success that’s visible up and down the organization. A lot of benefits can come from this kind of realistic approach.”
Pat Guariglia, founder of ElegantAgile.com and a certified ScrumMaster, suggested in his blog choosing a project with “medium criticality” as a pilot. Too little criticality means upper management will blow off the project or not give it the resources it needs; too high criticality leaves little room for failure, placing too much pressure on the pilot team.
Vidoni cautioned against setting expectations too high too soon. “You need to get into the rhythm of the format. You should give yourself months to adjust to it before you look for the productivity gains.”
Joe Little, Certified ScrumMaster, agile coach and managing director of Kitty Hawk Consulting, recommends that teams beginning down the Scrum path be co-located for a while—about six sprints, he said. Further, he would not use Scrum tools to get started.
“I would learn it with a [whiteboard] in the team rooms,” he said. “Most people later on, with distributed activities, would need some kind of Scrum tool. If you’re widely distributed—which I wouldn’t be—you need a tool. But I don’t think it’s the key to success.”
One aspect, however, that is seen as a key to success is doing Scrum by the book at first. “Everything is there for a very good reason,” Szalvay said. “You set up responsibilities for the product owner and team. When people say they don’t want to name a product owner, that leads to all kinds of issues. A product owner makes priority decisions, and you need one person to be responsible to the organization and its customers for that priority. Or, they say they don’t want to co-locate teams. Then they won’t gel, and it leads to bad ramifications.”
Yet TypeMock’s Lopian admitted his company uses a hybrid mix. “We don’t have the team and project owner and ScrumMaster definitions,” he said. “We use managers to think about what needs to be done.”
There are two schools of thought on Scrum, according to Jez Humble of ThoughtWorks Studios, which make development tools for use in agile shops. First is the school that says you have to do all of it, or you’re not doing it properly. The other is doing what he called “cowboy agile” due to customizations.
Szalvay suggested that people wait until they have a firm understanding of Scrum and the process before they start to tailor it. “This way, they’re doing things intentionally and not unwittingly.” He also cautioned against doing “Scrum, but…” which he explained as “doing Scrum, but not this part of it or that part of it.”
Excella’s Cheng said Scrum can be tweaked to an organization’s needs, “but don’t lose the heart of it.”
“It’s like cooking,” he explained. “When you learn to cook, you follow a recipe. If it’s not to your liking, you tweak it so it comes out good. But it could also turn out inedible.”
But, like cooking, once you have the ingredients in place, it’s important to know how long to keep them on the stove. Time-boxed sprints are an essential part of Scrum, yet estimating how long a task will take can be problematic.
“Sizing is one of the bigger learning curves,” said Pegasystems’ Vidoni. “Our approach to estimation was to take a set of baseline stories of various complexities and to size new tasks to that reference point.”
Pegasystems has merged business processes with the development process, which Vidoni said provides the benefit of “really assessing what our business priorities are and not over-commit. We’re able to balance our tactical needs with the more strategic business needs.”
TypeMock’s Lopian agreed that project estimation was an early problem. Scrum advises that sprints take no longer than four weeks. At first, he said, “you don’t know how to estimate how long a task will be. It could take half the time you allot for it, or it could be three times as long.”
Trial and error got them down from four-week iterations. “We went to one week about a year ago, but the development team didn’t feel they could deliver value in that time, so we went back to two weeks. Now we’re more mature, and we’re back to one week.”
ThoughtWorks’ Humble took it even one step further, with the advocacy of what the company is calling continuous delivery. “It’s all about keeping software production-ready. Every four weeks isn’t enough. Continuous delivery means having software production-ready all the time,” he said.
Yet Scrum doesn’t prescribe any technical behaviors for achieving agile success, and that’s either a flaw or a compliment, depending on point of view.
“Scrum talks a lot about process and management of a project, but speaks very little about the engineering side,” said Excella’s Cheng. “XP talks more about that, with continuous integration, test-driven development and pair programming.”
To address this, Cheng pointed out that the Scrum Alliance is offering Certified Scrum Developer classes. “They’re trying to penetrate that area more,” he said.
But Humble, who said that “Scrum isn’t sufficient to regularly deliver high-quality software” because it doesn’t look at engineering practices, said the danger of Scrum expanding into technical practices is that it could lose its focus. “Scrum has a business focus, and it’s easy to get going on,” he said.
The fact that Scrum does not prescribe engineering practices is “not a bug, it’s a feature,” said Kitty Hawk’s Little. “It’s not clear which will be the right ones” for any given organization.
One of the key points of Scrum, said Cheng, is that it lets teams do what they do best, without heavy requirements. “You create partnerships as to why things are being built. You don’t manage people, but give them goals and let them figure out how to get there.”
