Continuous Integration
Continuous integration is concerned with making sure that each completed task is always fully integrated into the current version of the product at an ongoing basis.
It is an extremely strange, but common, peculiarity among many software projects where for long periods of time during development process the application is not in a working state. The reason for this is simple: nobody is actually trying to run the whole application in a production-like environment until it is finished.
In such projects, releasing the software involves complicated stabilization phase where things need to be debugged, fixed and cleaned up before the software can be released.
On the other hand, there are projects that can make a release at any time of the day and it will always result in a working application. Not necessarily complete application but a working one. The defining factor is the use of continuous integration.
This introduces a different way of looking at your software: without continuous integration, your software is broken until somebody proves that it works (during a testing or integration phase). With continuous integration, your software is proven to work (using automated tests) with every new change and when it breaks you fix it immediately.
With continuous integration you tend to catch bugs much earlier and the cost of having to fix each bug is greatly reduced.
What you need
To successfully implement continuous integration you need at the very least the following parts of the devops process in place:
-
Version control: everything in your project must be versioned and reproducible from historical changes.
-
Automated builds: a fully automated build is essential because continuous integration requires you to run integration and end to end tests on actual production software. Thus your build must be fully automated.
-
CI Automation: even though it's possible to run all tasks using commit hooks, a fully automated CI system like GitLab will make your continuous integration process very smooth and robust.
Integration tests
Write integration tests to verify that your software components work together. Unless you are testing integration with hardware, make your integration tests run as native executables on the host system.
If you are testing integration with hardware then you will need to build out
physical hardware that can run your tests. The way to connect this hardware to
CI is through for example a gitlab-runner
. This service can run on the machine
with hardware connected to it and run pipeline jobs in docker environment with
full access to your hardware.
Do not include too much complexity into your integration tests unless your goal is to test a larger integration.
Virtual integration
One way to verify integration between components on target platform but without requiring hardware is to use a simulator.
-
QEMU: this simulator is useful for running your code on target CPU but does not provide extensive support for peripherals. QEMU is useful for running tests on an embedded target and correct architecture and testing integrations between components on that platform.
-
Renode: this simulator provides much more comprehensive hardware support and supports multidevice simulation as well. However, the drawback of a complex simulator is that it becomes more time consuming to configure and use and the benefits are marginal compared to testing on actual hardware directly.
It may be tempting to try to integration test everything on a simulator, however the big disadvantage of simulators is that they do not test integration with real hardware.
One big advantage of simulators for testing is that they are easy to scale. If you have physical hardware, you need to manage the physical wiring and scaling is hard. If you have a simulation environment, scaling to many runners in the cloud is easy.
Best practices
The following are most important practices for making continuous integration work.
- Checkin regularly: make it a habit to run full set of tests on your work even if you are not fully done with your merge request. This can bring your attention to problems you can fix right away. Regular checkins also ensure that if you mess up your local files, you will always have backup on the server that you can revert to.
- Comprehensive tests: create a comprehensive and automated test suite. Use unit tests to verify individual files in isolation, use integration tests to verify integration between software modules and use end to end tests to verify that the deliverable works. You should be building a fully deployable deliverable in each of your pipelines. Don't neglect this step.
- Quick feedback: structure your process so that it runs quickly. Some of the checks like formatting and spelling can even run before every commit where automated fixes can be applied as well. Your whole production pipeline should not take more than 20 minutes. The aim should be on 5 minutes at most but larger projects may require longer.
- Extreme programming: study this subject and apply XP practices to your work. XP contains all the practices that are relevant for scrum-based workflows but in much more pure form. Test driven development in particular is very good practice to adopt.
- Fail the build: since your CI pipeline is the main quality gateway for all code that is added to your software, it should fail the build whenever the code does not meet the required standards. Instead of relaxing these rules, try to shorten the feedback cycle for developers. Pre-commit hooks are a good way for example to avoid spelling errors from failing on the CI server by failing locally when developer tries to commit. This prompts the developer to fix such issues quickly and before they push to the repository.