Unit tests, integration tests, and functional tests are all types of automated tests which form essential cornerstones of continuous delivery, a development methodology that allows you to safely ship changes to production in days or hours rather than months or years.
Automated tests enhance software stability by catching more errors before software reaches the end user. They provide a safety net that allows developers to make changes without fear that they will unknowingly break something in the process.
The Cost of Neglecting Tests
[author_more]
Contrary to popular intuition, maintaining a quality test suite can dramatically enhance developer productivity by catching errors immediately. Without them, end users encounter more bugs, which can lead to increased reliance on customer service, quality assurance teams, and bug reports.
Test Driven Development takes a little more time up front, but bugs that reach customers cost more in many ways:
- They interrupt the user experience, which can cost you in sales, usage metrics, they can even drive customers away permanently.
- Every bug report must be validated by QA or developers.
- Bug fixes are interruptions which cause a costly context switch. Each interruption can waste up to 20 minutes per bug, not counting the actual fix.
- Bug diagnosis happens outside the normal context of feature development, sometimes by different developers who are unfamiliar with the code and the surrounding implications of it.
- Opportunity cost: The development team must wait for bug fixes before they can continue working on the planned development roadmap.
The cost of a bug that makes it into production is many times larger than the cost of a bug caught by an automated test suite. In other words, TDD has an overwhelmingly positive ROI.
Different Types of Tests
The first thing you need to understand about different types of tests is that they all have a job to do. They play important roles in continuous delivery.
A while back, I was consulting on an ambitious project where the team was having a hard time building a reliable test suite. Because it was hard to use and understand, it rarely got used or maintained.
One of the problems I observed with the existing test suite is that it confused unit tests, functional tests, and integration tests. It made absolutely no distinction between any of them.
The result was a test suite that was not particularly well suited for anything.
Roles Tests Play in Continuous Delivery
Each type of test has a unique role to play. You don’t choose between unit tests, functional tests, and integration tests. Use all of them, and make sure you can run each type of test suite in isolation from the others.
Most apps will require both unit tests and functional tests, and many complex apps will also require integration tests.
- Unit tests ensure that individual components of the app work as expected. Assertions test the component API.
- Integration tests ensure that component collaborations work as expected. Assertions may test component API, UI, or side-effects (such as database I/O, logging, etc…)
- Functional tests ensure that the app works as expected from the user’s perspective. Assertions primarily test the user interface.
You should isolate unit tests, integration tests, and functional tests from each other so that you can easily run them separately during different phases of development. During continuous integration, tests are frequently used in three ways:
- During development, for developer feedback. Unit tests are particularly helpful here.
- In the staging environment, to detect problems and stop the deploy process if something goes wrong. Typically the full suite of all test types are run at this stage.
- In the production environment, a subset of production-safe functional tests known as smoke tests are run to ensure that none of the critical functionality was broken during the deploy process.
Which Test Types Should You Use? All of Them.
In order to understand how different tests fit in your software development process, you need to understand that each kind of test has a job to do, and those tests roughly fall into three broad categories:
- User experience tests (end user experience)
- Developer API tests (developer experience)
- Infrastructure tests (load tests, network integration tests, etc...)
User experience tests examine the system from the perspective of the user, using the actual user interface, typically using the target platforms or devices.
Developer API tests examine the system from the perspective of a developer. When I say API, I don't mean HTTP APIs. I mean the surface area API of a unit: the interface used by developers to interact with the module, function, class, etc...
Unit Tests: Realtime Developer Feedback
Unit tests ensure that individual components work in isolation from each other. Units are typically modules, functions, etc...
For example, your app may need to route URLs to route handlers. A unit test may be written against the URL parser to ensure that the relevant components of the URL are parsed correctly. Another unit test might ensure that the router calls the correct handler for a given URL.
However, if you want to test that when a specific URL is posted to, a corresponding record gets added to the database, that would be an integration test, not a unit test.
Unit tests are frequently used as a developer feedback mechanism during development. For example, I run lint and unit tests on every file change and monitor the results in a development console which gives me real-time feedback as I’m working.
For this to work well, unit tests must run very quickly, which means that asynchronous operations such as network and file I/O should be avoided in unit tests.
Since integration tests and functional tests very frequently rely on network connections and file I/O, they tend to significantly slow down the test run when there are lots of tests, which can stretch the run time from milliseconds into minutes. In the case of very large apps, a complete functional test run can take more than an hour.
Unit tests should be:
- Dead simple.
- Lightning fast.
- A good bug report.
What do I mean by "a good bug report?"
I mean that whatever test runner and assertion library you use, a failing unit test should tell you at a glance:
- Which component is under test?
- What is the expected behavior?
- What was the actual result?
- What is the expected result?
- How is the behavior reproduced?
The first four questions should be visible in the failure report. The last question should be clear from the test’s implementation. Some assertion types are not capable of answering all those questions in a failure report, but most equal, same, or deepEqual assertions should. In fact, if those were the only assertions in any assertion library, most test suites would probably be better off. Simplify.
Continue reading %JavaScript Testing: Unit vs Functional vs Integration Tests%
by Eric Elliott via SitePoint
No comments:
Post a Comment