What is balena?
Balena is a complete set of tools for building, deploying, and managing fleets of connected Linux devices. We provide infrastructure for fleet owners so they can focus on developing their applications and growing their fleets with as little friction as possible.
Our tools are designed to work well together as a platform, but you can also pick and choose the components you need for your project, and adapt them to your particular use case. We know that no two IoT projects are the same and there is no one-size-fits-all solution.
If you're eager to learn more about the inner workings of balena, you're in luck! We're eager to share. This guide covers the components and workflows involved in a typical balena deployment.
Our universe
The core balena platform, or what we call balenaCloud, encompasses device, server, and client-side software, all designed to get your code securely deployed to a fleet of devices. The broad strokes are easy to grasp: once your device is set up with our host OS (balenaOS), you can push code to the balena build servers, where it will be packaged into containers and delivered to your fleet.
All your devices and their running services can then be managed, monitored, and updated through the web dashboard, or through our API via the CLI and SDK.
On your device
Devices in the balena ecosystem run balenaOS, a bare-bones, Yocto Linux based host OS, which comes packaged with balenaEngine, our lightweight, Docker -compatible container engine.
The host OS is responsible for kicking off the device supervisor, balena’s agent on your device, as well as your containerized services. Within each service's container you can specify a base OS, which can come from any existing Docker base image that is compatible with your device architecture.
The base OS shares a kernel with the host OS, but otherwise works independently. If you choose, your containers can be configured to run as privileged, access hardware directly, and even inject modules into the kernel.
The balena device supervisor runs in its own container, which allows us to continue running and pulling new code even if your application crashes.
Host and kernel updates
Balena is built with the goal of 100% updatability. While the balena device supervisor and your application containers are easy to update without losing connection to the device, the process for updating the host OS involves a few more steps.
To mitigate problems while updating, balena creates an additional partition of identical size to the boot partition. The supervisor downloads a new OS version and boots the device from the alternative boot partition. This way, we can ensure that the new version of the OS is downloaded and installed correctly before rebooting the device to the new version.
Even if the new version fails to boot for some reason, our system is built in such a way that the next boot will bring the device back to the original working version of the host OS, operating and ready to download a correctly installed new version.
It is important to note that all balena devices, both those in production and development today, have the ability to have their host OS updated. For most devices, this can even be done directly through the dashboard.
Device provisioning
We recommend balenaEtcher, an open source tool we’ve built to make device provisioning safe and easy, to flash balenaOS images on your devices.
So how are devices then added to your balenaCloud fleet? A key feature of balenaCloud is that a provisioning key for your application is embedded in the balenaOS image download. When the device boots up for the first time, it uses the provisioning API to register itself with balenaCloud. A new device entry on the balenaCloud backend is created, and a device API key for this device is generated. Once the provisioning is successful, the provisioning API key is deleted from the device.
Unless someone downloads the OS from your dashboard (or via the CLI), a device cannot enter your fleet. While the details of provisioning differ depending on the device type (does it have an SD card slot? Does it boot from on-board flash?), the following things always happen at first boot:
First, the device connects to the network and performs its early provisioning, which registers it on your dashboard. Then, the container partition is decompressed, and the device supervisor starts. This is the part that takes the most time.
As soon as the supervisor starts, it registers onto the VPN and receives its unique balenaCloud API key. At that point, you will see the device as online in your dashboard and can use the device as normal.
If the fleet that the device provisions into has already had code pushed to it, the new device downloads the latest version and begins operating as expected.
Code deployment
Code deployment begins when you type balena push in your command line, moving your application code from the development machine to the balenaCloud platform. Our builder receives your code and starts building a release for your application.
This remote repo serves as the source of truth for all devices in your fleet. Any code pushed to the master branch is passed to our builders, which generate Docker images to be sent to your devices.
Building containers
Your code is then built in an environment that matches the devices in your fleet. So if you’re pushing an app for BeagleBone Black devices, we’ll build your code in an ARMv7 environment. For Raspberry Pi 1, it's ARMv6. In fact, we provide native ARM builders for ARM images, just as we use x86 servers to build images for x86 devices.
For applications with multiple containers, a docker-compose.yml file will need to be included at the root of your project. This configuration file specifies the services that make up your application, as well as the system resources each service has access to. Applications with a single container will have a default docker-compose.yml file generated if none is included.
Most services will need to include a Dockerfile, which contains a list of commands for the builders. For each service, the builders will pull a base OS, install packages and dependencies, clone git repositories, and run any other steps you define for the setup and initialization of that service.
For Node.js services, you can use a package.json file without a Dockerfile. In this case, the builders create an implicit Dockerfile, which simulates the build process a Node.js/npm project expects. In this way, we are able to transparently run Node.js services on balena, while also taking advantage of some of Docker’s caching features. A Dockerfile will always give you more power to fine-tune, but you can start fast without and shift to a Dockerfile whenever you like.
Getting to the devices
Once your Docker images are built, they are stored in our container registry, and the balena device supervisor is alerted that a new version of your application is ready. If a device is offline at the time, it will be made aware of the new containers when it comes back online. The communication between balena and your device is encrypted at all times, either through HTTPS or a VPN that we set up for the devices in your fleet.
The device supervisor then downloads the changed layers of your container images, stops the old services, and starts the new ones. You can control the exact sequence by configuring the supervisor to use different update strategies. To reduce download time and bandwidth consumption, you can also enable delta updates. This tells the supervisor to only download the binary differences between the old and the new images. The services themselves can also make use of update locking to block updates from happening during critical times (e.g. a drone that is flying, or an industrial machine that is in the middle of an operation).
As the downloads proceed, you can watch the progress in the balena dashboard. You can click on any device to see more detailed information about the services being downloaded.
Device management
Once your services are up and running, you can use the dashboard to monitor and interact with them. Messages from the device supervisor, as well as anything written by your services to stdout and stderr, will appear in the Logs window, which can be filtered by service. Our built-in web terminal allows you to SSH into any running services, as well as the underlying host OS.
Much of the device, service, and application information provided by the dashboard is managed through the balena API, and can also be viewed and modified using the CLI or the Node.js and Python SDKs. Balena has been designed so users can build rich experiences, combining device-level data provided by balena with higher-level application-specific data that lives in other data domains.
Hardware
We introduced a hardware component to our platform after seeing the pain our users experience trying to select the right board for production. The balenaFin is a carrier board for the Raspberry Pi compute module, hardened for deployment in the field. It comes with reliable storage, industrial power, flexible networking, real-time and low-power capabilities, and multiple interfaces, all while preserving the developer experience of the Raspberry Pi. The Fin is, of course, supported on balenaCloud, but you don’t need to use balenaCloud to use the Fin, and vice versa. We still provide first class support for many device types including the Raspberry Pi family, the Intel NUC, the Nvidia Jetson TX2, and more.
Final thoughts
Balena has been built to bring a cloud microservices workflow to the world of edge devices and remove friction for fleet owners wherever possible. Our aim is to make IoT applications as easy to deploy as web applications (or even easier!). Along the way, we've discovered and assimilated interesting ideas from both the cloud and embedded worlds, and even invented ideas that only apply to the new paradigm balena represents.
We hope this has given you a taste of how things work behind the scenes—for more detailed information, please take a look at our documentation. And if you have any questions, we'd love to speak to you in our forums!
Note: balena was formerly known as resin.io - read more about the name change on our blog.