Agile development is now well entrenched, and most think they understand what agile means. As with just about everything, the definition varies depending upon whom you ask.
Most developers who are doing agile are using Scrum methodologies, but that is not the only path available. Today, agile development in the Microsoft world involves both processes and, despite its being de-emphasized in the Agile Manifesto, tooling. Microsoft offers a broad set of project-management and developer tools in its Team Foundation Service, and other software providers extend that platform with plug-ins that help organizations become agile in a more efficient, effective way. This report looks at both sides of the equation.
What agile means
Back in 2001, a group of very experienced software developers put forth the Agile Manifesto, which states a series of priorities and objectives that are meant to guide developers and development teams through the shark-infested waters of professional software development. The development world is filled with horror stories of projects gone wrong and fortunes lost on dead-end systems that never make it into production. I even testified as an expert witness a few years ago in a dispute arising from one of these less-than-successful endeavors.
The root causes are often a lack of transparency, failure to communicate (or outright miscommunications), and—perhaps the deadliest of sins in this space—wishful thinking. These were the factors that affected the aforementioned doomed application as well.
The Agile Manifesto espouses a philosophy rather than providing a detailed instruction guide. At its core are four statements about important things, such as “Individuals and interactions over processes and tools” and “Working software over comprehensive documentation,” along with “Customer collaboration over contract negotiation” and finally “Responding to change over following a plan.”
Notice that methodologies are not included in any of this, even though the founding fathers of Scrum are among the signees of the Manifesto. At its heart, agile just is a developer-style rehash of the age old Army motto of “Lead, follow or get out of the way!” The goal is to achieve something, to get to producing value and moving away from assigning blame or making excuses. All great rules to live by, but mere mortals are left to ask how, and that is where the methodologies and the tools become critically important.
Where agile fits
Agile development has many fans—for good reason—but the first thing to understand is that it is not necessarily for everyone and is not a fit for every project under the sun. There are many ways to characterize a project, but the primary differentiator for any development project is always whether the details of the final result are known in advance or not.
When a project is started with an exact blueprint already defined, then a competent consultancy could quote a fixed bid proposal. But if there are any unanswered questions, it is not ready for a waterfall development process. This is the kind of development project that is planned from the start, and also the kind that runs according to a project manager’s Gantt chart rather than according to a Scrum Master’s cadence of sprints. These kinds of projects are still out there, but they are fewer and farther between since entrepreneurs, website owners and even department heads are increasingly compelled to get coding before the idea or requirements are fully fleshed out.
That is the beauty of agile: It not only tolerates change, it actually welcomes it. Waterfall projects are not nimble, but they are predictable (or at least they should be) and that makes them really handy if you require a fixed price for the development or even just a reasonable estimate. A project that does not define all requirements at the outset must be undertaken using a Spiral development model in which the product owner states a vision, and then the coders mock up or develop to that vision such that the owner can see a demo and decide where it should go from there. Then that process is iterated until the project is deemed usable or good enough. After that, more iterations can be used to evolve the system or to complete it depending on whether it has a perpetual life or a definable end state.
The closer the original vision is to the ultimate end goal, the less of the early work will be wasted. Sometimes people who have been burned by Spiral development will refer to them as “Death Spirals” since a great deal of effort can be wasted before useable results are achieved. Almost all startups that are founded by developers use a Spiral since the developer does not have to pay out of pocket to get coding started.#!The evolution of Team Foundation Server
For years Microsoft offered Visual SourceSafe (VSS) as its sole solution to managing code in a project. Eventually, Team Foundation Server (TFS) evolved to replace VSS and greatly expanded features such that, rather than just protecting and versioning source code, allowed for the whole application life cycle to be managed.
Over a number of versions, TFS added features that have made it indispensable to organizations that have come to rely on it. With the popularity of TFS, the concept of a Work Item entered into the vocabulary of all developers who work with Microsoft technology, even if they did not use TFS. At the end of 2012 Microsoft launched an online version of TFS that is sometimes referred to as Team Foundation Service.
However, Microsoft seems to always have a naming mishap that ends up clouding the message around any major product. In the case of the Software-as-a-Service version of TFS, this came with the URL and associated name (visualstudioonline.com) coming into use for Team Foundation Service. In fact, Visual Studio Online includes Team Foundation Service, but also folds into that Visual Studio subscriptions and other extras. It makes sense if you already know that this URL is what TFS is using for the online version, and that it has added the Visual Studio subscriptions at the Pro level and above. Otherwise it looks like it’s supposed to be about Visual Studio as a product rather than Team Foundation Service.
The online nature of Visual Studio Online has helped accelerate the DevOps trend as the lines between developer and admin are blurred. Some organizations are resistant to moving from hosting their own TFS to using Visual Studio Online due to a desire to maintain control, or because they have need of something that is not possible yet with the online offering. Often there is fear that the network team will become obsolete if the cloud gets too much of a foothold, and objections flow from there. The CTO usually pushes for the cloud while the CIO sees a loss of influence if the data centers shrink.
There are still a number of valid reasons to choose on-premises TFS over the cloud-based version, but those differences are disappearing with each new iteration. Currently the main items missing from the online offering are the ability to do customizations or extensions, as well as CodeLens, SharePoint integration, SQL Reporting, and business intelligence features.
The greatest disappointment is the absence of CodeLens support, but this is likely to be added since there does not appear to be anything that would block this except time to implementation. Eventually we can expect all of these disparities to disappear, leaving a desire to maintain control as the defining reason for sticking with on-premises implementations.
To better understand how DevOps fits into the picture, Joel Semeniuk, cofounder of Imaginet and longtime proponent of ALM, talked about this in the context of how his company helps its customers manage projects from inception to end of life with TFS and a few other tools. One unique thing about his company’s process and experience is that it sticks around the application over the long term.
This means that DevOps is a big part of its process. “DevOps is all about bringing agility to the process of releasing and operating software,” said Semeniuk. “Traditionally, development and operations have been two separate worlds: developers using one methodology and set of tools, and IT using an entirely different set of processes and tools.”
He added, “DevOps is about bringing both worlds together from a process and tooling perspective. DevOps not only better facilitates the build and release cycle, but also provides critical insight back to the development team, creating a tight feedback loop that can dramatically impact the underlying software.”
Methodologies as a way of life
Among Microsoft development shops, Scrum is definitely the dominant agile methodology, and it might be the dominant methodology overall. Rather than talk about agile, however, most diehard Scrum practitioners in the Microsoft space talk about Application Lifecycle Management (ALM). This is possibly because that is how Microsoft talks about the features of Team Foundation Server and Team Foundation Service, but also because ALM is more encompassing.
This is an example of how Microsoft has embraced agile without forsaking the tools; instead, it makes the tools that enable agile to work with its own technology. SSW is a consultancy based in Sydney, Australia, that specializes in software development projects using Microsoft technology. Adam Cogan, chief architect at SSW, Microsoft Regional Director and Microsoft MVP in Visual Studio ALM; and Adam Stephensen, solution architect at SSW and one of its FireBootCamp Mentors, were enthusiastic about discussing how SSW uses both Scrum and TFS to deliver success to their customers.
Cogan is constantly open to talk about Scrum and the tools that make it work for him and his teams, and he famously posts rules for various endeavors on the SSW website. There is even a page for Rules to Better Scrum Using TFS.
When asked about whether SSW uses agile methodologies other than Scrum, and what tools they value for their agile development, Cogan said, “At SSW we run all projects with Scrum. It’s one of the important reasons we haven’t had a project fail in six years.” When asked how he defines no project failures, he said that all of the company’s projects have “gone into production.”
Cogan also said, “I think TFS is almost the complete ALM solution… Backlogs, source control, build, testing…it’s all great. The only areas that are weak are dealing with e-mails and deploying.” More on how SSW handles these perceived shortcomings a bit later.#!Scrum is not the only game in town for teams in search of an agile methodology. Kanban appears to be the second most popular, and Visual Studio Online supports it by providing a view of work items in the backlog as Kanban cards (see Figure 1). Kanban comes to us from Toyota and is applied to manufacturing as well as development.
In Kanban cards, tasks are the focus of the system, and the cards move through states of completion. The card concept overlays well with the Work Items of TFS, making TFS well-suited to organizations following Kanban as well as Scrum.
The primary goal of Scrum, Kanban and most other methodologies (when done well) is to expose issues and force the team to run toward problems rather than sweeping them under the rug. The techniques focus attention on the tasks at hand so that even teams with less experience can accomplish steady progress.
SSW has a program it runs called FireBootCamps, which teach development in a crash course environment with Scrum. Stephensen said, “I have just spent nine weeks living and breathing the Microsoft technology stack running FireBootCamp.com. We took a bunch of guys, and in a few weeks went from specs to release on two enterprise projects using the latest technologies and best practices. This wouldn’t have been possible a few years ago. The combination of Team Foundation Service on VisualStudio.com, Scrum, Windows Azure and continuous deployment makes it possible for awesome developers to efficiently gather requirements, add value, easily ship to staging and production, gather feedback, and delight the customer.” This seems like the ultimate torture test for development tools and methodologies.
Agile tools of TFS
Organizations that have adopted Scrum will often create their own tools, but among Microsoft development shops Team Foundation Server and selected add-on packages are the preferred tooling for keeping teams on track.
While Team Foundation Service may be a fairly common denominator among Microsoft development shops that are doing agile development, some of the features that the experts feel require supplemental tools are telling. When asked about tools used beyond TFS, Semeniuk said, “We typically augment TFS with the traditional whiteboard and sticky notes, which are really critically used in sprint planning. We also use the Imaginet TimeSheet for TFS to track our work against work items to be used in customer billing and effort analysis.”
This is a product that Imaginet built for its own use and has turned into a product for others to use. The bit about whiteboard and sticky notes recalls the emphasis in the Agile Manifesto for human interaction, though with widely dispersed teams that more often must be worked around with collaboration tools when not available.
In TFS, the focus is on the Work Items and the various views of them that shed light on how things are progressing on a project. The philosophy is very utilitarian; do you want a product with 10 features all of which are 80% complete, or would you prefer a product with 10 features with eight of them complete?
Work Items are atomic, which makes them achievable inside a sprint duration. There are a number of interfaces that present the Work Item data in revealing ways. According to Semeniuk, “Perhaps the most critical component of TFS that we use on our agile project is the agile boards. Visualization of flow is absolutely one of the biggest aspects of agility, and TFS agile boards are an information radiator of this flow that the team can use to gauge the rate that they are producing value and get early warning signals if expectations may not be met.”
Another interface in TFS is the backlog (see Figure 2). The concept of a backlog in Scrum is the list of things to be done on the project (the backlog is the list of these Work Items that are selected for each sprint). The items that make up the backlog are Work Items that are not currently assigned to a sprint, and they can represent features or bug fixes that are ready to be assigned to sprints (a sprint is a set period of time in which the project is meant to make progress). With each of these iterations, the project moves toward completion or to higher states of functionality. The Work Item charting is a more graphical view (see Figure 3) and shows how varied the views into the data can be.
There is a great deal to TFS and many features are important, but as Semeniuk explained, “My must-have feature of TFS is absolutely AppAnalytics. Some would argue that source code control, automated builds or work item tracking are more important; however, I find that deeply understanding how your software is being used by your customers can give you opportunities to innovate in ways you didn’t have in more traditional methods of developing software. AppAnalytics give you the ability to create a quantifiable feedback loop on your requirements and allow you to more incrementally experiment on requirements to rapidly fine-tune your ability to deliver real value to customers.”
The consensus appears to be that tools are critical to getting the job done in agile development, and the de-emphasis of tooling in the Agile Manifesto is not meant to say that tools are not important. It is meant to ensure the tools support the process rather than subordinating it. Semeniuk agreed by saying, “The tools should be there to facilitate and support that way of thinking and not get in the way.“
Staying out of the way means not limiting access, and that is where Team Explorer Everywhere comes in. It provides a client for the TFS data that is accessible from non-Microsoft clients in recognition that even Microsoft shops will use resources that use different platforms. By providing the TFS plug-in for Eclipse, Microsoft has lowered barriers that have traditionally meant that TFS was only realistically usable by pure Microsoft shops. Eclipse has made great progress as a development interface, and the attitudes so prevalent in the moves seen by the Azure group at Microsoft are being replicated by the TFS group.#!Tools that support agile
What are the must-have tools to support agile development? For most, Team Foundation Server or Team Foundation Service provide a complete solution, with one being hosted your own servers and the other being the online version done in a Software-as-a-Service model.
For some, TFS is just a baseline, and additional tools are added to support weaknesses. As we have seen, TFS does quite a lot, but not everything. Cogan called out e-mail management and continuous deployment as areas that required some extra help. When asked about what they use specifically, he said, “We use Team Companion for e-mail management, and Octopus Deploy to simplify and automate our continuous deployment.” Continuous deployment allows the project to be constantly up to date with all working code that has been checked in.
Cogan added, “Brian Harry’s ALM vision for Microsoft developers is pretty awesome, and I know our customers would back us up.”
Harry is a Microsoft Technical Fellow who joined the company when Microsoft acquired his product, which became Visual Source Safe. Just as Scott Guthrie has been serving as the person most identified with the technical leadership of Azure, Harry serves that role for TFS. Today he is the Product Unit Manager for Team Foundation Server, and he drives a great deal of the innovation for Microsoft in the ALM space.
Telerik offers a product named TeamPulse, which allows you to create custom boards and view multiple projects at the same time. According to the TeamPulse page the product allows you to “put customers at the center of your projects with a centralized and transparent feedback management system.” It goes on to assert, “With TeamPulse Ideas and Feedback Portal, you can effectively capture and prioritize feedback from your customers, partners and internal stakeholders and keep them up to date on the status of their requests.”
Semeniuk worked with Telerik to bring TeamPulse to market, but now Telerik has assumed sole development of the product. “There are a plethora of third-party tools that fit well with TFS, including TeamCompanion (an Outlook add-in for TFS) and other products such as SmartOffice4TFS that has very comprehensive Microsoft Office (Word, Excel, Visio) integrations with TFS that impact how you work with and represent requirements in TFS,” he said.
Echoing Semeniuk on the benefits of TeamCompanion, Cogan said, “We use Team Companion for e-mail management, and Octopus Deploy to simplify and automate our continuous deployment.”
Teams, not heroes
Many marketing campaigns aimed at developers promise to make them a “Super Hero” with promises that they can save the day. This pitch has a solid basis in the conventional wisdom that a single top-tier developer is worth a whole team of average developers in terms of productivity. But this also flies in the face of a central tenant of agile in general. Scrum and other agile methodologies require self-organizing teams to share tasks.
Recently I was able to discuss this conflict with Stuart Celarier, principal consultant at Neudesic. He takes exception to these kinds of campaigns and views them as sending the wrong message.
“Software developers should be like Peter Parker and Clark Kent, not Spidey and the Man of Steel,” he said. “Not only are superpowers not required, they are detrimental to teams and sustained, repeatable delivery of product to the customer.”
Celarier dislikes the dark side that the developer-as-superhero tends to engender. It is not unheard of to have some would-be hero developer gallantly step in to solve a problem that other, less visible team members realize was caused by the problem-solver in the first place. This does not mean that strong performers and even exceptional performers need not apply; it simply means that with agile, our progress no longer depends solely on their prowess. To be of use in the modern age, they have to be able to work as a team. Think Avengers or Justice League rather than lone wolf.
It seems that this change in attitude is the surest sign that agile is working and the tools are helping us win the war so that the average project can succeed and the average developer can take the place of the superhero.
