When software unexpectedly fails, the consequences for businesses are huge. For enterprises, the cost of downtime can run to thousands of dollars per minute, but the impact goes well beyond lost revenue. 

The knock-on effect of IT failures – tanking stock price, reputational damage and customer dissatisfaction – are well understood. What is less well known usually is the underlying cause: the inherent, unresolved issues in an organization’s system. The reason? Systems are now so complex and interconnected, it can be difficult for companies or stakeholders to get to the bottom of why an error has occurred. 

It is clear that businesses need to try to root out these failures before they become problems. It is no longer enough to conduct functional, unit or system testing alone because defects can creep in through the many integrations needed between not only the systems and subsystems but also a host of other means. That’s where end-to-end testing enters the picture.

End-to-end (E2E) testing is a technique that verifies that all the features of a system and its interconnected subsystems work as expected. And in today’s complex, distributed systems where software defects can be very hard to pinpoint, this approach is highly effective at detecting when apps are not performing as they should.

End-to-end testing: how it compares with other methods 

More traditional testing methods are limited to validating the performance of a single function, component or application. However, these processes do not account for how the whole system works together as one.

It should be noted that system or integration testing does address this challenge by verifying multiple components are working together as they should. This makes this type of testing useful for improving app performance and reliability but it does not give an accurate picture of how the app is working from the point of view of the end-user.

That is where end-to-end testing comes in.

By simulating the flows that a user would take, E2E testing assesses if end-user performance is functioning as expected. If we take an example of an online purchasing system, the E2E testing method would replicate a user logging into their account, adding products to the cart, proceeding towards checkout and filling in the required information to finalize the purchase.

When to conduct E2E testing

It goes without saying that E2E tests should be run in conjunction with other testing methods. In fact, they are especially valuable in cases where an output is not as expected, signalling a problem somewhere within the system.

As well as helping to check the overall “health” of an app or product, E2E tests identify errors in the system and its subsystems, validate app behavior over multi-tier architecture and help ensure a bug-free, seamless experience. For more complicated apps, it can be particularly useful in dividing them into multiple tiers for testing.

The basics of E2E testing

Most E2E testing follows this established lifecycle:

  • Planning: specify the tests to be run, organize a test schedule and allocate resources.
  • Design: take account of the different specifications, requirements and suitable testing frameworks to be used. As part of the design, risk and usage analysis should be considered.
  • Execution: run the test cases and document the results.
  • Results analysis: check what broke (and fix it), evaluate testing and re-run tests if needed.

There are two ways testing can be conducted – horizontal and vertical E2E testing.

  • Horizontal E2E testing is the method largely preferred by testers. In horizontal E2E testing, every workflow is tested through a discrete application from beginning to end to check if everything is working as it should.
  • Vertical E2E testing examines the system in layers and is used for critical modules of a larger, complex system. Testing is conducted in sequential, hierarchical order. In vertical E2E testing, each component of a system or product is tested from start to finish to ensure quality. This kind of testing is mainly used to test critical components of a complex computing system that does not typically involve users or interfaces.
Avoid E2E testing pitfalls

When embarking on E2E testing, the two main trip hazards are issues around orchestration and access.

In E2E testing, the test cases must be executed sequentially. However, it’s not easy to run multiple cases (sometimes thousands) in sequential order in a distributed workflow.

Ensuring continued access can be the other stumbling block. Ensuring continued access can be the other stumbling block. It’s comparatively easier to test custom applications in a virtual test environment. However, E2E testing the actual execution must occur in a production environment and needs a continuous active user session.

When an E2E automated testing bot sets out to test the workflow, it requires local agents to be installed and logged in to remote facilities. The test team will need to find a way to keep the machine awake for the duration of the test and prevent unwanted events interrupting the tests (a Windows update, for example).

Why perform E2E testing?

Distributed systems contain a lot of moving parts, many of which are interconnected, meaning that it is extremely hard to predict how one failure somewhere in the system will affect the whole. One seemingly small defect has the power to do a lot of damage. 

Addressing the root causes of such failures requires a holistic approach to testing and E2E testing is an important part of the mix. Not only does E2E testing root out the defects in complex, distributed systems, it also enables teams to improve how end-users experience an app.

Performing E2E testing helps software teams have greater confidence in the quality of the product and, in the case of new launches, it speeds up time-to-market. Ultimately, whether the product or service already exists or is being designed, E2E testing is a useful technique that reduces repetitive testing, improves software quality and optimizes the operational efficiency of the SDLC.