Since we implemented and went through some rounds of this new design phase in our development cycle, we have managed to communicate clearly all aspects of the design to all stakeholders, reduce the overall development time of features, and deliver features with higher quality.
An important part of any quality process is collecting quality attributes data and making them visible. So first the team suggested finding a way to collect all PHP errors from all Web servers, daemon servers, etc. to a single DB table, where all application errors were already reported. After we had these results, we exposed them as part of R&D metrics so everyone would be aware of the situation.
Then we decided to add a mechanism for logging all long DB queries in the system so we have all information regarding them. Exposing this information not only helped us improve the response time of the website, but also enabled us to track serious contention issues on the DB that were spotted easily.
Later on we decided to expose all important R&D metrics such as average home page load time and others—issues that we act on regularly—to monitor and help us improve our performance by looking at “small measurable” changes.
Automatic monitoring tools
We were satisfied with the progress we had made so far, and we aimed to achieve more. Thus, the next issue we decided to focus on was the manual quality operations we have performed. Because they were manual and took precious time away from other activities, we performed them only once a week. But MyHeritage could not afford to wait that long to find important quality issues.
So next we gave priority to monitoring our application and PHP errors. We decided to automate the error-monitoring process, run analyses twice per day, and send an e-mail that summarizes all new errors and all existing errors that grew significantly. Errors were spotted the same day they reached production.
Another quality measure that is under QA’s responsibility is going over all our statistics charts and looking for anomalies in the number. This might not sound like a big deal, but imagine going over 2,500 charts all by yourself. This is not fun, and definitely not efficient.
So once again, we came up with the idea of building an infrastructure (coming soon as open source) to analyze all our charts’ data automatically. Our mathematically inclined developer built our system to be highly flexible and to allow the user to add charts to white lists, define thresholds, replace the analysis algorithm and much more.
But what is a true quality change?
So everything we’ve talked about so far definitely contributes to quality, but is this a quality change? Not to me. A quality change to me is what the organization is experiencing right now as I write this article.