Getting Started Developing on ev3dev
The standard LEGO Mindstorms EV3 Controller (better known as the LEGO Mindstorms EV3 Programmable Brick) is a Linux-based computing platform (referred to in the rest of this page as a Robot Controller) for interfacing with the various motors and sensors for the Mindstorms environment.
LEGO provides the EV3 Programmer software which uses a graphical programming paradigm, originating from National Instruments' LabView platform, to develop programs to run on the EV3 Brick. However, these executables stored in *.rbf files are actually virtual machine bytecode files (conceptually similar to Java Bytecodes) that run on top of the LMS Virtual Machine.
Alternatively, others are interested in writing programs that execute directly on the EV3 Brick, bypassing the LMS Virtual Machine. Commercial solutions include RobotC which supports C-like programming for the EV3 Brick to create the *.rbf virtual machine bytecode files for their custom firmware.
Nonetheless, since the EV3 Brick Operating System is based on Linux, we have various alternatives to commercially available tools. ev3dev provides a Debian-based Linux distribution to support people interested in writing code directly for the Linux environment using open source tools. This also opens up the possibility of using other Robot Controllers such as the Raspberry Pi, Beaglebone boards (with appropriate motor and sensor expansion capes), and others which have higher processing capabilities compared to the EV3 Brick to run software written for the ev3dev distribution.
The basic process to start developing on ev3dev is as follows:
- Choose a Language
- Choose a Workflow and Toolchain
- Build the Libraries
- Write your Programs
- Download Program to Target
- Run/Debug the Program
While it is a bit daunting to start by choosing a Programming Language if you're new to programming, the list of languages are more or less sorted from easiest for beginners such as Python, to more advanced options such as C++ and C.
This mostly applies to more advanced language options.
For Python-based development, almost everything is already built-in to the ev3dev distribution; all you need is an Editor or Integrated Development Environment (IDE) to write your programs in, and then execute it on the Robot Controller (after downloading if necessary).
The pre-built ev3dev image has everything you need, unless you're interested in installing non-standard library packages into the ev3dev image (this is an advanced topic which is not covered in this guide).
For the other languages, there is a need to compile the source files into executable files that will be run on the Robot Controller. Technically, it is possible to write and compile programs on the Robot Controller platform (this is also known as Target (Native) Compilation), but that would be an exercise in frustration except for compiling trivial programs such as Hello World!.
Typically we would want to use our Host PC to write, compile and debug the program, since it is much more powerful than the Robot Controller platform. However, the Robot Controller uses a different CPU or Instruction Set compared with the ones typically used on your PC.
More specifically, the available Robot Controllers are
almostall based on the ARM CPU Architecture, while the PC are (almost) all based on the x86 or x64 (Intel-Architecture compatible) CPU Architecture.
Bytecode based languages such as the LEGO Programmer and Java will have Compilers that compile the source programs into Virtual Machine bytecodes, which is able to run on any platform having the appropriate Virtual Machine software. However, the default C and C++ compilers for the PC platform targets the x86 or x64 CPU Architecture, and cannot be run on the Robot Controller.
Consequently, when compiling C and C++ programs on the PC to run on the Robot Controller, we need to perform Cross Compilation. The C and C++ cross compilers used by ev3dev are based on the GNU Compiler Collection (GCC).
It is important to select the correct Target environment for the cross compiler; otherwise the generated programs will not be able to run on the targeted Robot controller platform. Generally we should use a generic Cross-Compiler which can generate executables for multiple target architectures.
However, if you're building on the Target platform natively, or else intend to run the architecture specific cross-compiler under emulation on the Host to build with custom libraries which are only available for the given target architecture, GCC provides architecture specific compiler and cross-compiler toolchains that takes less disk space compared with the multi-target cross-compilers.
When cross-compiling using a GCC Cross-Toolchain, the selected target architecture MUST be compliant with the chosen Robot Controller platform distribution (EV3 Brick, RPi, etc.), otherwise unexpected problems may occur. There are three Target Architectures for ARM-based Debian distributions used by ev3dev depending on the Controller hardware platform:
- armel (for EV3 Programmable Brick)
- armhf for ARMv6 (for the original Raspberry Pi and Pi Zero, Docker Cross-compiler Image not provided)
- armhf for ARMv7 (for Raspberry Pi 2, 3, and Beaglebone)
You can use the following Toolchain Selection Guide for your chosen programming languages as a reference:
- Python
- C/C++
Four options are available for C/C++:
- Target-based (Native) Compilation (compiler running on Robot Controller): This is not a recommended configuration for large projects due to the limited storage, processing power and RAM on the Robot Controller. You'll need the build-essential apt package.
- Host-based Emulated Cross-Compilation: If custom libraries were only available for the Target platform; the cross-compilers are actually Target-based compilers running in an emulator on the Host (PC). Look for
debian-<dist>-<arch>-crossdocker iamges. - Host-based Native Cross-Compilation: If multi-architecture custom libraries were available on the Host (PC) platform; this is the most efficient configuration. Look for
debian-<dist>-crossdocker images. - Vendor supplied Host-based Cross-Compilation Toolchain: The latest versions of the vendor toolchains may not support the Robot Controller platform. See C++ Language Bindings Project for a link to vendor supplied packages.
- ???
- TBD
It is important to keep in mind that you're dealing with two separate Operating Systems environments when developing software for the EV3, the Host (PC) environment and the Target (EV3) environment. The Host environment typically would be based on Windows, macOS, or Linux Operating Systems, while the Target will be based on the Debian Linux-derived ev3dev Distribution. Nonetheless, the following discussion assumes the use of a POSIX compliant environment for the Host.
Packages maintained by ev3dev are highlighted in turquiose to indicate that they are used for building applications for the e3vdev platform. In addition, some examples of vendor-supplied Cross compilers are listed for reference. Standard packages (default packages provided by third parties for the Host OS) are highlighted in plum.
The essential components for developing on the Host are:
- Integrated Development Environment (IDE) or Editor
While it is not the intention of this guide to recommend any particular IDE, since it is highly subjective regarding which is the best (and this is often a contentious debate); you may want to take a look at Eclipse, which provides an open source, cross-platform everything-including-the-kitchen-sink IDE that is highly extensible and has support for various plugins relevant for cross-platform development.
- Cross-compiler Toolchain
ev3dev has packaged the relevant GCC Cross-compiler Toolchain in Docker containers to simplify the installation of a POSIX-compliant development environment for the Host.
- Project Build Tools (we assume the use of Makefiles for managing compilation)
- Remote Access program (we assume use of OpenSSH client to login to the EV3 Platform remotely)
- Executable Program Downloader from PC to Target (e.g., OpenSSH provides
scp(Secure Copy) for transferring files) - Cross-Debugger (usually comes with the Cross-compiler Toolchain, but can also be provided as part of the IDE)
In addition, several build tools are provided for building ev3dev Distribution Boot images or GCC Cross-compiler toolchains for Docker.
These build tools are not needed unless you plan to create custom toolchains or custom ev3dev Distribution images.
This is not meant as an exhaustive list of all packages used, but to highlight those essential or relevant for the operation of the ev3dev environment.
The description of the packages starts from the bottom layer, since the lower layers of the OS are common to the upper layers. The ev3dev specific packages are highlighted in turquiose to indicate that they are specifically developed for the e3vdev platform. Debian-derived packages (default packages provided by the Debian Linux Distribution) are highlighted in plum. Additionally, packages highlighted in dark gray indicate third-party packages ported for use with the e3vdev distribution.
- Platform specific Kernel (part of ev3dev Boot Image (only ev3dev-kernel based kernel provided currently)
- Linux Daemons (part of ev3dev Boot Image)
- Low-level Library Packages (part of ev3dev Boot Image)
- Mid-level Library Packages
Micropython is included as part of the ev3dev Boot Image.
The other ev3dev specific libraries listed here are for reference only. They are not included in the Boot Image but but will be statically linked in with the program where necessary
- Target Applications
Native Compiler Tools are not included in the ev3dev Boot Image, since users are expected to perform cross-compilation.
- Brickman is the GUI interface for ev3dev (included as part of the ev3dev Boot Image)
- brickrun is the command line program launcher for ev3dev (also called by Brickman internally)
- System Information and Configuration scripts: ev3dev-sysinfo and ev3dev-config from the ev3dev-tools repository
- Others (FIXME)
The use of the various libraries in ev3dev for development requires installing the relevant header (include) files for the Cross-compilation platform on the Host. This is documented in the Building the Libraries section.
Take a look at Organization of ev3dev Repositories for more information regarding the packages maintained by the ev3dev project, including available Language Bindings and support libraries.
Based on the chosen Programming Language and toolchain, the specific repository should be cloned to your Host (PC) platform.
The Getting Started guide for a programming language may suggest doing it on the Robot Controller, but if you plan on working with a large project with multiple (> 10) source files, it is advisable to choose a proper cross-compilation workflow due to the large storage requirements for compilation, as well as the issue of it being much slower due to the limited processing capabilities and RAM on the Robot Controller.
Typically cross-platform development uses statically linked libraries to create the final program.
While dynamically loaded libraries may save some space on the boot image, the compilation and linking step is more complicated, and it is up to the developer to make sure that the version of the libraries on the boot image is the correct version for the given application.
In this step, you need to take note of the directory locations for both the Library archive files (*.a), as well as the include header files. They would typically be different from the paths to the Host (PC) compiler libraries and include files.
On Linux, the standard library path is in
/usr/lib, while the standard include path is in/usr/include, they are used to keep all system-wide library archives and include files.Cross-compilation is considered an example of using custom libraries. The standard libraries for the cross-compiler would also usually be stored in a well known location such as
/usr/local/or/opt/local, depending on the vendor or packager of the cross-compilation toolchain. Normally, you won't need to specify the path to the standard libraries and header files for the cross-compilerOn top of that, the libraries for supporting EV3 cross-compilation is another set of custom libraries that we need to access from our programs. There may not be a standard location for accessing those files, except for the location where the libraries were cloned to, though well designed custom libraries usually allow us to install it to the same directories as the cross-compiler libraries and headers for easier access.
Therefore, it is important when compiling with custom libraries to specify the correct path to the header directory and the library archive correctly to the compiler and linker, so that it will know how to access the needed files unless they could be installed to the standard locations.
It is not the intention of this guide to go into detailed description of every step, but the things to be aware of are:
- Program Lifecycle
- Creating project Makefiles to build the program automatically
- Setting up correct paths to custom or ev3dev specific libraries and include files
- Use of the Editor or IDE features for syntax checking and API completion
- Adoption of common style-guidelines for consistency with existing projects/code
- Using source revision management software to track changes made to the program
Every program follows the basic life cycle for applications:
- Genesis of an idea (spark)
- Breaking down the idea into manageable components
- Sketch out essential logic of specific components
- Test out the basic functionality of a component (prototyping, also include running and debugging the program)
- Refine the functionality based on prototype
- Iterate until you're happy with the outcome
- Archive/freeze program development and/or share with others
There are many ways (formally termed development paradigms) to carry out these steps, some may not even consciously adopt a given paradigm but it came out naturally as part of the exploration. Nonetheless, having some semblance of a structured sequence would help reduce the frustrations and confusions involved with program development as the project scales up in complexity.
The default building process adopted by ev3dev and many other projects uses make to manage the compilation of files after they have been edited or modified. There are many other build management tools available, each with different strengths and weaknesses. Nonetheless, the default starting point is to use your IDE to create the necessary project build files, and modify it if necessary for advanced build features. For example, Eclipse CDT with the GNU MCU Eclipse plugin would create standard project folders with the necessary Makefiles for you automatically when creating a New Project.
FIXME: Need some additional HOWTOs for Make
This involves specifying the paths passed to the cross-compiler and linker. For GCC, the compiler command line switches are:
-L <path-to-library>-I <path-to-include-directory>
In addition, for the generation of the complete program, all custom libraries used by the program would need to be linked with the object files for the program using:
-l <library-archive>
This assumes static linking of libraries used by the program
One of the major advantages of using an IDE as opposed to a normal text editor is the capability to highlight the programming language syntax to ease comprehension and bug tracking. While purists may insist on using no-frills text editor for speed in entering the program code, a large portion of the programmer's time is actually spent understanding the program logic and how to convert it into the appropriate syntax used by the chosen programming language. Hence, IDEs with good syntax highlighting is a great help in reducing logical errors introduced during the course of programming.
If the IDE was configured well, it is even possible for it to provide prompts for function parameters and reference to the appropriate header files when looking up the definition of a particular function call. This is another useful feature especially when dealing with custom libraries and APIs (not the standard language supplied library APIs)
Very often we start from example projects or code taken from other open source projects. This is the basis for ev3dev after all. While it is tempting to just add new code to the project willy-nilly, it is better to try and be consistent with the existing project style guidelines, to ensure readability as well as future sharing of the code with others.
Large projects frequently have strict style guidelines and will not accept code contribution from others which do not meet those guidelines.
Source code control, revision management, or code control, refer to the task of managing changes to the program code over time. The ability to roll back a change which introduced an unexpected bug, or to test out new features while the already working code is being used actively, is a powerful feature of Source Revision Management Software. Beyond these typical features that benefit an individual programmer tremendously, almost all source revision management software also allows collaboration with other programmers to improve the code and work on larger projects in a team.
Since this is a huge topic in itself, this guide will not get into further details, except to note that knowledge of commonly-used source revision management software is a useful skill for any programmer to know, and is well worth the time investment involved. If you're new to this whole concept of source code control, Git is the go-to software for source revision management today.
For reference, all ev3dev projects are using Git for source code control; Git is the basis for all projects hosted on GitHub as well.
TBD
TBD