Once out of their initial planning phases, software projects have a tendency to lurch from immediate issue to immediate issue, dealing with each new requirement as it arises. Considering requirements in isolation invariably means the burden of large-scale redesign to satisfy any one requirement will seem onerous; a smaller-scale, more imperfect but less impactful alternative will always seem the better option. Over time, many such small, imperfect design decisions inevitably degrade the quality of the software.

Reflection, on the other hand, allows you to consider many weeks’ worth of problems in a holistic light: You can see all the birds at once, and an approach that once seemed over-engineered now appears justified. Surfacing all the issues at the same time clears a path forward that otherwise would have seemed prohibitive.

Another benefit of choosing Action Research is it intrinsically defines a structure for your thesis. You can do three or four Action Research cycles over the course of your Ph.D., and arrange them as chapters. In turn, each chapter can be broken down into the “planning,” “acting,” “observing” and “reflecting” phases of each cycle. This immediately helps focus your research.

I did, however, find this structure to be a double-edged sword. A common problem a lot of Ph.D. candidates face is doing all their research—and little of their thesis writing—up front. Then after several years, they face the stressful task of trying to document all their work as a coherent narrative. Action Research can help here, because it permits you to document each cycle as a self-contained chapter as you go.

However, it can also hurt. I discovered this because, while I wrote up most of my research each year, I confess I left a few sections unfinished. Action Research makes it much more difficult to go back and complete those sections later, because so much of Action Research is concerned with reflections. Proper reflections are constrained by the context of their time. Attempting to cast one’s mind back not just to the particular thoughts, but also to the overall mindset of two or three years prior, is a formidable task. It requires you to remember what you knew, what you didn’t know, and what you thought you knew but have since learned differently. And you must disregard that more recent knowledge, lest you end up writing a revisionist history. It’s very difficult, and I would warn against trying to document Action Research retroactively.

Action Research was a hit with my examiners. One wrote, “I was very moved by the candidate’s choice of methodology… So many [candidates] in computing disciplines propose new architectural approaches and prototype their approach, but [ultimately] readers are [only] left with ideas, which might work if only someone in industry ever adopted them. [This candidate’s] approach… meant that his system is proven both to academic and to industry developer communities. As an examiner, on realising how widely used the system is and then reading the adoption study results, I breathed a sigh of relief. What a delight to see good ideas actually used!”

Having broken your thesis into a series of Action Research cycles, it remains to discuss what you should do within each of these cycles. Much of this comes down to simply sound software engineering: planning, developing, testing, documenting. An important part is the holistic reflection discussed earlier. Another important part is how to gather feedback and observations from industry.
Grounded Theory: If you’re going to be developing your thesis in close collaboration with industry developers, that collaboration needs to be framed in a research context. This can take the form of interviews, case studies, adoption studies, or other academically recognized approaches.

A useful methodology here is Grounded Theory. It’s well suited to interviewing because it helps you gather observations without biasing your interviewees. Grounded Theory proceeds by first gathering the data, then codifying the data into categories and themes, and finally constructing hypotheses. It stresses that outcomes should be explicitly emergent, rather than simply verification of existing hypotheses.

You compose lists of directed but unbiased, open-ended questions that give people room to talk openly, while at the same time guiding them into the gap defined in your literature review. You then document, compare and categorize what emerges. Ideally the resultant categories will include your own understanding of the problem, but should also expose perspectives you hadn’t considered.