Code overview¶

This document describes the overall code layout and major code flow of Kalico.

Directory Layout¶

The src/ directory contains the C source for the micro-controller code. The src/atsam/, src/atsamd/, src/avr/, src/linux/, src/lpc176x/, src/pru/, and src/stm32/ directories contain architecture specific micro-controller code. The src/simulator/ contains code stubs that allow the micro-controller to be test compiled on other architectures. The src/generic/ directory contains helper code that may be useful across different architectures. The build arranges for includes of "board/somefile.h" to first look in the current architecture directory (eg, src/avr/somefile.h) and then in the generic directory (eg, src/generic/somefile.h).

The klippy/ directory contains the host software. Most of the host software is written in Python, however the klippy/chelper/ directory contains some C code helpers. The klippy/kinematics/ directory contains the robot kinematics code. The klippy/extras/ directory contains the host code extensible "modules".

The lib/ directory contains external 3rd-party library code that is necessary to build some targets.

The config/ directory contains example printer configuration files.

The scripts/ directory contains build-time scripts useful for compiling the micro-controller code.

The test/ directory contains automated test cases.

During compilation, the build may create an out/ directory. This contains temporary build time objects. The final micro-controller object that is built is out/klipper.elf.hex on AVR and out/klipper.bin on ARM.

Micro-controller code flow¶

Execution of the micro-controller code starts in architecture specific code (eg, src/avr/main.c) which ultimately calls sched_main() located in src/sched.c. The sched_main() code starts by running all functions that have been tagged with the DECL_INIT() macro. It then goes on to repeatedly run all functions tagged with the DECL_TASK() macro.

One of the main task functions is command_dispatch() located in src/command.c. This function is called from the board specific input/output code (eg, src/avr/serial.c, src/generic/serial_irq.c) and it runs the command functions associated with the commands found in the input stream. Command functions are declared using the DECL_COMMAND() macro (see the protocol document for more information).

Task, init, and command functions always run with interrupts enabled (however, they can temporarily disable interrupts if needed). These functions should avoid long pauses, delays, or do work that lasts a significant time. (Long delays in these "task" functions result in scheduling jitter for other "tasks" - delays over 100us may become noticeable, delays over 500us may result in command retransmissions, delays over 100ms may result in watchdog reboots.) These functions schedule work at specific times by scheduling timers.

Timer functions are scheduled by calling sched_add_timer() (located in src/sched.c). The scheduler code will arrange for the given function to be called at the requested clock time. Timer interrupts are initially handled in an architecture specific interrupt handler (eg, src/avr/timer.c) which calls sched_timer_dispatch() located in src/sched.c. The timer interrupt leads to execution of schedule timer functions. Timer functions always run with interrupts disabled. The timer functions should always complete within a few micro-seconds. At completion of the timer event, the function may choose to reschedule itself.

In the event an error is detected the code can invoke shutdown() (a macro which calls sched_shutdown() located in src/sched.c). Invoking shutdown() causes all functions tagged with the DECL_SHUTDOWN() macro to be run. Shutdown functions always run with interrupts disabled.

Much of the functionality of the micro-controller involves working with General-Purpose Input/Output pins (GPIO). In order to abstract the low-level architecture specific code from the high-level task code, all GPIO events are implemented in architecture specific wrappers (eg, src/avr/gpio.c). The code is compiled with gcc's "-flto -fwhole-program" optimization which does an excellent job of inlining functions across compilation units, so most of these tiny gpio functions are inlined into their callers, and there is no run-time cost to using them.

Klippy code overview¶

The host code (Klippy) is intended to run on a low-cost computer (such as a Raspberry Pi) paired with the micro-controller. The code is primarily written in Python, however it does use CFFI to implement some functionality in C code.

Initial execution starts in klippy/klippy.py. This reads the command-line arguments, opens the printer config file, instantiates the main printer objects, and starts the serial connection. The main execution of G-code commands is in the process_commands() method in klippy/gcode.py. This code translates the G-code commands into printer object calls, which frequently translate the actions to commands to be executed on the micro-controller (as declared via the DECL_COMMAND macro in the micro-controller code).

There are four threads in the Klippy host code. The main thread handles incoming gcode commands. A second thread (which resides entirely in the klippy/chelper/serialqueue.c C code) handles low-level IO with the serial port. The third thread is used to process response messages from the micro-controller in the Python code (see klippy/serialhdl.py). The fourth thread writes debug messages to the log (see klippy/queuelogger.py) so that the other threads never block on log writes.

Code flow of a move command¶

A typical printer movement starts when a "G1" command is sent to the Klippy host and it completes when the corresponding step pulses are produced on the micro-controller. This section outlines the code flow of a typical move command. The kinematics document provides further information on the mechanics of moves.

Processing for a move command starts in gcode.py. The goal of gcode.py is to translate G-code into internal calls. A G1 command will invoke cmd_G1() in klippy/extras/gcode_move.py. The gcode_move.py code handles changes in origin (eg, G92), changes in relative vs absolute positions (eg, G90), and unit changes (eg, F6000=100mm/s). The code path for a move is: _process_data() -> _process_commands() -> cmd_G1(). Ultimately the ToolHead class is invoked to execute the actual request: cmd_G1() -> ToolHead.move()

The ToolHead class (in toolhead.py) handles "look-ahead" and tracks the timing of printing actions. The main codepath for a move is: ToolHead.move() -> LookAheadQueue.add_move() -> LookAheadQueue.flush() -> Move.set_junction() -> ToolHead._process_moves().
- ToolHead.move() creates a Move() object with the parameters of the move (in cartesian space and in units of seconds and millimeters).
- The kinematics class is given the opportunity to audit each move (ToolHead.move() -> kin.check_move()). The kinematics classes are located in the klippy/kinematics/ directory. The check_move() code may raise an error if the move is not valid. If check_move() completes successfully then the underlying kinematics must be able to handle the move.
- LookAheadQueue.add_move() places the move object on the "look-ahead" queue.
- LookAheadQueue.flush() determines the start and end velocities of each move.
- Move.set_junction() implements the "trapezoid generator" on a move. The "trapezoid generator" breaks every move into three parts: a constant acceleration phase, followed by a constant velocity phase, followed by a constant deceleration phase. Every move contains these three phases in this order, but some phases may be of zero duration.
- When ToolHead._process_moves() is called, everything about the move is known - its start location, its end location, its acceleration, its start/cruising/end velocity, and distance traveled during acceleration/cruising/deceleration. All the information is stored in the Move() class and is in cartesian space in units of millimeters and seconds.

Kalico uses an iterative solver to generate the step times for each stepper. For efficiency reasons, the stepper pulse times are generated in C code. The moves are first placed on a "trapezoid motion queue": ToolHead._process_moves() -> trapq_append() (in klippy/chelper/trapq.c). The step times are then generated: ToolHead._process_moves() -> ToolHead._advance_move_time() -> ToolHead._advance_flush_time() -> MCU_Stepper.generate_steps() -> itersolve_generate_steps() -> itersolve_gen_steps_range() (in klippy/chelper/itersolve.c). The goal of the iterative solver is to find step times given a function that calculates a stepper position from a time. This is done by repeatedly "guessing" various times until the stepper position formula returns the desired position of the next step on the stepper. The feedback produced from each guess is used to improve future guesses so that the process rapidly converges to the desired time. The kinematic stepper position formulas are located in the klippy/chelper/ directory (eg, kin_cart.c, kin_corexy.c, kin_delta.c, kin_extruder.c).

Note that the extruder is handled in its own kinematic class: ToolHead._process_moves() -> PrinterExtruder.move(). Since the Move() class specifies the exact movement time and since step pulses are sent to the micro-controller with specific timing, stepper movements produced by the extruder class will be in sync with head movement even though the code is kept separate.

After the iterative solver calculates the step times they are added to an array: itersolve_gen_steps_range() -> stepcompress_append() (in klippy/chelper/stepcompress.c). The array (struct stepcompress.queue) stores the corresponding micro-controller clock counter times for every step. Here the "micro-controller clock counter" value directly corresponds to the micro-controller's hardware counter - it is relative to when the micro-controller was last powered up.

The next major step is to compress the steps: stepcompress_flush() -> compress_bisect_add() (in klippy/chelper/stepcompress.c). This code generates and encodes a series of micro-controller "queue_step" commands that correspond to the list of stepper step times built in the previous stage. These "queue_step" commands are then queued, prioritized, and sent to the micro-controller (via stepcompress.c:steppersync and serialqueue.c:serialqueue).

Processing of the queue_step commands on the micro-controller starts in src/command.c which parses the command and calls command_queue_step(). The command_queue_step() code (in src/stepper.c) just appends the parameters of each queue_step command to a per stepper queue. Under normal operation the queue_step command is parsed and queued at least 100ms before the time of its first step. Finally, the generation of stepper events is done in stepper_event(). It's called from the hardware timer interrupt at the scheduled time of the first step. The stepper_event() code generates a step pulse and then reschedules itself to run at the time of the next step pulse for the given queue_step parameters. The parameters for each queue_step command are "interval", "count", and "add". At a high-level, stepper_event() runs the following, 'count' times: do_step(); next_wake_time = last_wake_time + interval; interval += add;

The above may seem like a lot of complexity to execute a movement. However, the only really interesting parts are in the ToolHead and kinematic classes. It's this part of the code which specifies the movements and their timings. The remaining parts of the processing is mostly just communication and plumbing.

Adding a host module¶

The Klippy host code has a dynamic module loading capability. If a config section named "[my_module]" is found in the printer config file then the software will automatically attempt to load the python module klippy/extras/my_module.py . This module system is the preferred method for adding new functionality to Kalico.

The easiest way to add a new module is to use an existing module as a reference - see klippy/extras/servo.py as an example.

The following may also be useful:

Execution of the module starts in the module level load_config() function (for config sections of the form [my_module]) or in load_config_prefix() (for config sections of the form [my_module my_name]). This function is passed a "config" object and it must return a new "printer object" associated with the given config section.
During the process of instantiating a new printer object, the config object can be used to read parameters from the given config section. This is done using config.get(), config.getfloat(), config.getint(), etc. methods. Be sure to read all values from the config during the construction of the printer object - if the user specifies a config parameter that is not read during this phase then it will be assumed it is a typo in the config and an error will be raised.
Use the config.get_printer() method to obtain a reference to the main "printer" class. This "printer" class stores references to all the "printer objects" that have been instantiated. Use the printer.lookup_object() method to find references to other printer objects. Almost all functionality (even core kinematic modules) are encapsulated in one of these printer objects. Note, though, that when a new module is instantiated, not all other printer objects will have been instantiated. The "gcode" and "pins" modules will always be available, but for other modules it is a good idea to defer the lookup.
Register event handlers using the printer.register_event_handler() method if the code needs to be called during "events" raised by other printer objects. Each event name is a string, and by convention it is the name of the main source module that raises the event along with a short name for the action that is occurring (eg, "klippy:connect"). The parameters passed to each event handler are specific to the given event (as are exception handling and execution context). Two common startup events are:
- klippy:connect - This event is generated after all printer objects are instantiated. It is commonly used to lookup other printer objects, to verify config settings, and to perform an initial "handshake" with printer hardware.
- klippy:ready - This event is generated after all connect handlers have completed successfully. It indicates the printer is transitioning to a state ready to handle normal operations. Do not raise an error in this callback.
If there is an error in the user's config, be sure to raise it during the load_config() or "connect event" phases. Use either raise config.error("my error") or raise printer.config_error("my error") to report the error.
Use the "pins" module to configure a pin on a micro-controller. This is typically done with something similar to printer.lookup_object("pins").setup_pin("pwm", config.get("my_pin")). The returned object can then be commanded at run-time.
If the printer object defines a get_status() method then the module can export status information via macros and via the API Server. The get_status() method must return a Python dictionary with keys that are strings and values that are integers, floats, strings, lists, dictionaries, True, False, or None. Tuples (and named tuples) may also be used (these appear as lists when accessed via the API Server). Lists and dictionaries that are exported must be treated as "immutable" - if their contents change then a new object must be returned from get_status(), otherwise the API Server will not detect those changes.
If the module needs access to system timing or external file descriptors then use printer.get_reactor() to obtain access to the global "event reactor" class. This reactor class allows one to schedule timers, wait for input on file descriptors, and to "sleep" the host code.
Do not use global variables. All state should be stored in the printer object returned from the load_config() function. This is important as otherwise the RESTART command may not perform as expected. Also, for similar reasons, if any external files (or sockets) are opened then be sure to register a "klippy:disconnect" event handler and close them from that callback.
Avoid accessing the internal member variables (or calling methods that start with an underscore) of other printer objects. Observing this convention makes it easier to manage future changes.
It is recommended to assign a value to all member variables in the Python constructor of Python classes. (And therefore avoid utilizing Python's ability to dynamically create new member variables.)
If a Python variable is to store a floating point value then it is recommended to always assign and manipulate that variable with floating point constants (and never use integer constants). For example, prefer self.speed = 1. over self.speed = 1, and prefer self.speed = 2. * x over self.speed = 2 * x. Consistent use of floating point values can avoid hard to debug quirks in Python type conversions.
If submitting the module for inclusion in the main Kalico code, be sure to place a copyright notice at the top of the module. See the existing modules for the preferred format.

Adding new kinematics¶

This section provides some tips on adding support to Kalico for additional types of printer kinematics. This type of activity requires excellent understanding of the math formulas for the target kinematics. It also requires software development skills - though one should only need to update the host software.

Useful steps:

Start by studying the "code flow of a move" section and the Kinematics document.
Review the existing kinematic classes in the klippy/kinematics/ directory. The kinematic classes are tasked with converting a move in cartesian coordinates to the movement on each stepper. One should be able to copy one of these files as a starting point.
Implement the C stepper kinematic position functions for each stepper if they are not already available (see kin_cart.c, kin_corexy.c, and kin_delta.c in klippy/chelper/). The function should call move_get_coord() to convert a given move time (in seconds) to a cartesian coordinate (in millimeters), and then calculate the desired stepper position (in millimeters) from that cartesian coordinate.
Implement the calc_position() method in the new kinematics class. This method calculates the position of the toolhead in cartesian coordinates from the position of each stepper. It does not need to be efficient as it is typically only called during homing and probing operations.
Other methods. Implement the check_move(), get_status(), get_steppers(), home(), and set_position() methods. These functions are typically used to provide kinematic specific checks. However, at the start of development one can use boiler-plate code here.
Implement test cases. Create a g-code file with a series of moves that can test important cases for the given kinematics. Follow the debugging documentation to convert this g-code file to micro-controller commands. This is useful to exercise corner cases and to check for regressions.

Porting to a new micro-controller¶

This section provides some tips on porting Kalico's micro-controller code to a new architecture. This type of activity requires good knowledge of embedded development and hands-on access to the target micro-controller.

Useful steps:

Start by identifying any 3rd party libraries that will be used during the port. Common examples include "CMSIS" wrappers and manufacturer "HAL" libraries. All 3rd party code needs to be GNU GPLv3 compatible. The 3rd party code should be committed to the Kalico lib/ directory. Update the lib/README file with information on where and when the library was obtained. It is preferable to copy the code into the Kalico repository unchanged, but if any changes are required then those changes should be listed explicitly in the lib/README file.
Create a new architecture sub-directory in the src/ directory and add initial Kconfig and Makefile support. Use the existing architectures as a guide. The src/simulator provides a basic example of a minimum starting point.
The first main coding task is to bring up communication support to the target board. This is the most difficult step in a new port. Once basic communication is working, the remaining steps tend to be much easier. It is typical to use a UART type serial device during initial development as these types of hardware devices are generally easier to enable and control. During this phase, make liberal use of helper code from the src/generic/ directory (check how src/simulator/Makefile includes the generic C code into the build). It is also necessary to define timer_read_time() (which returns the current system clock) in this phase, but it is not necessary to fully support timer irq handling.
Get familiar with the the console.py tool (as described in the debugging document) and verify connectivity to the micro-controller with it. This tool translates the low-level micro-controller communication protocol to a human readable form.
Add support for timer dispatch from hardware interrupts. See Kalico commit 970831ee as an example of steps 1-5 done for the LPC176x architecture.
Bring up basic GPIO input and output support. See Kalico commit c78b9076 as an example of this.
Bring up additional peripherals - for example see Kalico commit 65613aed, c812a40a, and c381d03a.
Create a sample Kalico config file in the config/ directory. Test the micro-controller with the main klippy.py program.
Consider adding build test cases in the test/ directory.

Additional coding tips:

Avoid using "C bitfields" to access IO registers; prefer direct read and write operations of 32bit, 16bit, or 8bit integers. The C language specifications don't clearly specify how the compiler must implement C bitfields (eg, endianness, and bit layout), and it's difficult to determine what IO operations will occur on a C bitfield read or write.
Prefer writing explicit values to IO registers instead of using read-modify-write operations. That is, if updating a field in an IO register where the other fields have known values, then it is preferable to explicitly write the full contents of the register. Explicit writes produce code that is smaller, faster, and easier to debug.

Coordinate Systems¶

Internally, Kalico primarily tracks the position of the toolhead in cartesian coordinates that are relative to the coordinate system specified in the config file. That is, most of the Kalico code will never experience a change in coordinate systems. If the user makes a request to change the origin (eg, a G92 command) then that effect is obtained by translating future commands to the primary coordinate system.

However, in some cases it is useful to obtain the toolhead position in some other coordinate system and Kalico has several tools to facilitate that. This can be seen by running the GET_POSITION command. For example:

Send: GET_POSITION
Recv: // mcu: stepper_a:-2060 stepper_b:-1169 stepper_c:-1613
Recv: // stepper: stepper_a:457.254159 stepper_b:466.085669 stepper_c:465.382132
Recv: // kinematic: X:8.339144 Y:-3.131558 Z:233.347121
Recv: // toolhead: X:8.338078 Y:-3.123175 Z:233.347878 E:0.000000
Recv: // gcode: X:8.338078 Y:-3.123175 Z:233.347878 E:0.000000
Recv: // gcode base: X:0.000000 Y:0.000000 Z:0.000000 E:0.000000
Recv: // gcode homing: X:0.000000 Y:0.000000 Z:0.000000

The "mcu" position (stepper.get_mcu_position() in the code) is the total number of steps the micro-controller has issued in a positive direction minus the number of steps issued in a negative direction since the micro-controller was last reset. If the robot is in motion when the query is issued then the reported value includes moves buffered on the micro-controller, but does not include moves on the look-ahead queue.

The "stepper" position (stepper.get_commanded_position()) is the position of the given stepper as tracked by the kinematics code. This generally corresponds to the position (in mm) of the carriage along its rail, relative to the position_endstop specified in the config file. (Some kinematics track stepper positions in radians instead of millimeters.) If the robot is in motion when the query is issued then the reported value includes moves buffered on the micro-controller, but does not include moves on the look-ahead queue. One may use the toolhead.flush_step_generation() or toolhead.wait_moves() calls to fully flush the look-ahead and step generation code.

The "kinematic" position (kin.calc_position()) is the cartesian position of the toolhead as derived from "stepper" positions and is relative to the coordinate system specified in the config file. This may differ from the requested cartesian position due to the granularity of the stepper motors. If the robot is in motion when the "stepper" positions are taken then the reported value includes moves buffered on the micro-controller, but does not include moves on the look-ahead queue. One may use the toolhead.flush_step_generation() or toolhead.wait_moves() calls to fully flush the look-ahead and step generation code.

The "toolhead" position (toolhead.get_position()) is the last requested position of the toolhead in cartesian coordinates relative to the coordinate system specified in the config file. If the robot is in motion when the query is issued then the reported value includes all requested moves (even those in buffers waiting to be issued to the stepper motor drivers).

The "gcode" position is the last requested position from a G1 (or G0) command in cartesian coordinates relative to the coordinate system specified in the config file. This may differ from the "toolhead" position if a g-code transformation (eg, bed_mesh, bed_tilt, skew_correction) is in effect. This may differ from the actual coordinates specified in the last G1 command if the g-code origin has been changed (eg, G92, SET_GCODE_OFFSET, M221). The M114 command (gcode_move.get_status()['gcode_position']) will report the last g-code position relative to the current g-code coordinate system.

The "gcode base" is the location of the g-code origin in cartesian coordinates relative to the coordinate system specified in the config file. Commands such as G92, SET_GCODE_OFFSET, and M221 alter this value.

The "gcode homing" is the location to use for the g-code origin (in cartesian coordinates relative to the coordinate system specified in the config file) after a G28 home command. The SET_GCODE_OFFSET command can alter this value.

Time¶

Fundamental to the operation of Kalico is the handling of clocks, times, and timestamps. Kalico executes actions on the printer by scheduling events to occur in the near future. For example, to turn on a fan, the code might schedule a change to a GPIO pin in a 100ms. It is rare for the code to attempt to take an instantaneous action. Thus, the handling of time within Kalico is critical to correct operation.

There are three types of times tracked internally in the Kalico host software:

System time. The system time uses the system's monotonic clock - it is a floating point number stored as seconds and it is (generally) relative to when the host computer was last started. System times have limited use in the software - they are primarily used when interacting with the operating system. Within the host code, system times are frequently stored in variables named eventtime or curtime.
Print time. The print time is synchronized to the main micro-controller clock (the micro-controller defined in the "[mcu]" config section). It is a floating point number stored as seconds and is relative to when the main mcu was last restarted. It is possible to convert from a "print time" to the main micro-controller's hardware clock by multiplying the print time by the mcu's statically configured frequency rate. The high-level host code uses print times to calculate almost all physical actions (eg, head movement, heater changes, etc.). Within the host code, print times are generally stored in variables named print_time or move_time.
MCU clock. This is the hardware clock counter on each micro-controller. It is stored as an integer and its update rate is relative to the frequency of the given micro-controller. The host software translates its internal times to clocks before transmission to the mcu. The mcu code only ever tracks time in clock ticks. Within the host code, clock values are tracked as 64bit integers, while the mcu code uses 32bit integers. Within the host code, clocks are generally stored in variables with names containing clock or ticks.

Conversion between the different time formats is primarily implemented in the klippy/clocksync.py code.

Some things to be aware of when reviewing the code:

32bit and 64bit clocks: To reduce bandwidth and to improve micro-controller efficiency, clocks on the micro-controller are tracked as 32bit integers. When comparing two clocks in the mcu code, the timer_is_before() function must always be used to ensure integer rollovers are handled properly. The host software converts 32bit clocks to 64bit clocks by appending the high-order bits from the last mcu timestamp it has received - no message from the mcu is ever more than 2^31 clock ticks in the future or past so this conversion is never ambiguous. The host converts from 64bit clocks to 32bit clocks by simply truncating the high-order bits. To ensure there is no ambiguity in this conversion, the klippy/chelper/serialqueue.c code will buffer messages until they are within 2^31 clock ticks of their target time.
Multiple micro-controllers: The host software supports using multiple micro-controllers on a single printer. In this case, the "MCU clock" of each micro-controller is tracked separately. The clocksync.py code handles clock drift between micro-controllers by modifying the way it converts from "print time" to "MCU clock". On secondary mcus, the mcu frequency that is used in this conversion is regularly updated to account for measured drift.