Refactor

From iagent
Revision as of 12:02, 28 January 2022 by Elias (talk | contribs) (→‎Running Klipper)
Jump to: navigation, search

Refactor header.png

Refactor is the next generation of controller board software stack installation.

It is meant to simplify the lives of new users when they begin using a Thing-Printer controller board. Pre-built images of Refactor are available for users who want to get up and running quickly with Replicape or Recore.

Refactor is available for Replicape and Recore, with updates being propagated simultaneously for both platforms.

These instructions are for version v3.0.3, for v3.0.2 and older see:

Latest images

The latest images for Recore and Replicape are available from github: https://github.com/intelligent-agent/Refactor/releases

Installed software

Recore

The Recore version is based on Armbian, Debian Buster. On top of that, Refactor comes preinstalled with a stack of programs to simplify setting up a new controller board.

Replicape

The Replicape version of Refactor is based on the BeagleBoard official Debian Buster console image. with a stack of programs to simplify setting up a new controller board.

Getting started

It can be a good idea to test the board on the bench before installing it in the printer. Use the USB-C connector to power the board.

Once the board is powered up, use a computer to access the web interface provided by OctoPrint. In order to get access to OctoPrint, a network cable must be connected to the board and to a network switch. It is also possible to use a wifi dongle. If a network cable is used, the board should respond to the address: http://recore.local

Once the OctoPrint interface is visible, run through the wizard to set up a new printer. After the wizard is done, go to the Klipper tab in OctoPrint and edit the Klipper config file. This is where the bulk of the work will happen.

Klipper configuration

There is a standard klipper configuration file that comes with Refactor. The configuration file is set up to work out of the box, but it assumes that there is a 100 K ntc thermistor on analog inputs T0 and T3 (bed). If these are not present, Klipper will halt with a warning.

The default klipper config can be used as a starting point for building a custom config file for the printer at hand.

After the Klipper configuration is complete, do a test print to make sure everything is working.

Recore booting

Recore comes with Refactor pre-installed.

The Recore images will need to be flashed to a USB flash drive, as there is no SD slot on the board. Recore's uboot is designed to attempt to boot from a USB drive if there is no OS on the eMMC.

There are several stages to booting Recore, and depending on which media is installed, the boot process will try and boot either from eMMC, USB host or USB device.

Booting Recore from eMMC (the normal way)

  1. Turn on power
  2. The BROM will load u-boot SPL from eMMC
  3. The u-boot SPL will load u-boot
  4. u-boot will load Linux
  5. Linux will load the rootfs from eMMC.

Booting Recore from eMMC, but load rootfs from USB

Note

Using a root file system on a different drive work well if the kernel version is the same on both media. The kernel version has changed between Refactor v3.0.2-RC3 and v3.0.2-RC4. The board will still boot, but will not load kernel modules as expected, including nft that maps OctoPrint to the HTTP port (80).

If the board is working normally and you are able to log in through ssh, but want to try out a new Refactor version you can let the rootfs be loaded from a USB drive (mass storage device):

  1. Download Refactor and flash it to a USB drive
  2. Insert the USB drive in the board.
  3. Boot the board normally
    1. Open the Refactor tab in OctoPrint.
    2. Select run from USB
    or
    1. Use SSH to log into the board: ssh root@recore.local
    2. Run the command select-boot-media usb
  4. Reboot

The board should now be have the rootfs from the USB drive.

Booting Recore from USB, load Linux kernel from USB drive

This method requires a UART to USB adapter that can plug into a host computer. There is a 4 pin header on Recore marked DBG, where UART 0 is routed. All u-boot and kernel messages will appear here. If the eMMC is partitioned and has an OS, u-boot will launch right into that. In order to override that and load the boot script from a USB drive, stop u-boot by pressing a key and then write:

  1. Download Refactor and flash it to a USB drive
  2. Insert the USB drive in the board and apply power
  3. When the boot process gets to u-boot, you will see: Hit any key to stop autoboot:
  4. Hit the Any key.
  5. Then write run bootcmd_usb0

This will load the U-boot script from the USB drive and use the USB drive as the rootfs.

Boot Recore from a USB stick using the FEL button

If u-boot should be loaded from a host computer and kernel and rootfs is loaded from a USB drive:

  1. Flash the Refactor Linux distro to a USB drive.
  2. Insert the USB drive in the Recore board.
  3. Insert the Recore board in a host computer with the USB-C connector.
  4. If the "FEL" button is pressed, the board will go into bootloader mode.
  5. Upload SPL, TF-A and u-boot through the USB port:
  6. Download the Recore repository https://github.com/intelligent-agent/Recore
  7. Install the sunxi-tools program
  8. Run make fel

You can read more about the steps needed to boot aboard in FEL mode on the Sunxi wiki: https://linux-sunxi.org/FEL

Flashing Refactor from a USB drive

Get the latest Refactor image from here: https://github.com/intelligent-agent/Refactor/releases for example: 

wget https://github.com/intelligent-agent/Refactor/releases/download/v3.0.0/Refactor-recore-v3.0.0-2021-08-06.img.xz

Assuming you have managed to boot Recore from a USB flash drive and you have the ReFactor image available in the same folder, you can use dd to do a direct transfer of the image from the USB drive to eMMC. On Replicape / BeagleBone this will be /dev/mmcblk1 

xz -v -d -c Refactor-recore-v3.0.0-2021-08-06.img.xz  | dd of=/dev/mmcblk0

Upgrading Refactor

The only way to upgrade Refactor is by flashing a new version. In OctoPrint there is a "Refactor" plugin that can aid with this. If Refactor is run from a USB stick, a new version of can be downloaded and flashed on the eMMC.

Refactor-tab.png

Update u-boot on Recore

Place u-boot-sunxi-with-spl.bin in sector 15 (8KB offset) Flash the binary 

dd if=u-boot-sunxi-with-spl.bin of=/dev/mmcblk0 bs=1024 seek=8 conv=notrunc

Note: U-boot needs to be patched in order to use the eMMC as the first (and only) boot device. The changes can be done in "spl_mmc_get_device_index" where dev_num should be 0 instead of 1.

AR100 firmware

The ar100 has a binary file that is loaded on startup by klipper.service The binary file comes pre-compiled in Refactor, but can be recompiled on Recore (or on a host computer).

Cross compilation toolchain for ARM64

The AR100 needs a cross compilation toolchain. For ARM64, it can be downloaded from feeds.iagent.no:

http://feeds.iagent.no/toolchains/

The toolchain has been built with

https://github.com/richfelker/musl-cross-make

GCC version: 9.2.0

Cross compilation toolchain for x86_64

The toolchain used during automatic testing and is the same used for the CRUST firmware project:

https://github.com/crust-firmware/crust#building-the-firmware

Compiling the ar100 binary

cd /opt
wget http://feeds.iagent.no/toolchains/or1k-linux-musl.tar.xz
tar xf or1k-linux-musl.tar.xz
export PATH=$PATH:/opt/output/bin

To configure klipper for AR100 and build the binary: 

cd /home/debian/klipper/
cp test/configs/ar100.config .config
make olddefconfig
make

This will produce a bin file located in out/ar100.bin, so it should be moved to /op/firmware 

mv out/ar100.bin /opt/firmware/

This is the same operation that happens from the klipper.service script, so it can also be done by running 

systemctl restart klipper

STM32 firmware

Note

The latest Refactor now does automatic flashing of the STM32 firmware on first boot after a fresh install. This was merged with PR#199 in Refactor on Github.

The firmware for the stm32f031 mcu that is responsible for ADC conversions and PWM for heaters and fans comes pre-installed in the flash of the MCU. It can be recreated on the board using the Klipper build system. 

cd /home/debian/klipper
cp test/configs/stm32f031.config .config
make menuconfig
-> Enable extra low-level configuration options
-> Clock Reference internal
-> Communication interface (Serial (on USART2 PA15/PA14))
-> Baud rate for serial port: 38400
make

Flashing STM32 firmware

To flash the STM32 firmware: 

systemctl stop klipper
make recoreflash FLASH_DEVICE=/dev/ttyS4
systemctl start klipper

Replicape

The Replicape images will be meant to run from the micro SD you flash it to. You cannot boot the Replicape from a USB key at this time. By default, Refactor images do not flash to the eMMC the way Umikaze used to. Part of this is historical - the early betas of Refactor did not fit on the eMMC, but there's another advantage as well: it allows you to test out Refactor on an SD card while keeping your working Redeem config on the eMMC.

Booting Refactor from USB

If you want to boot Refactor from a USB stick, you need to stop u-boot by pressing space during boot. You must run these commands: 

setenv bootpart 0:1; setenv oldroot /dev/sda1; setenv devnum 0; run usb_boot

load ${devtype} ${bootpart} ${loadaddr} /boot/uEnv.txt; env import -t ${loadaddr} ${filesize}

run uname_boot

Running Klipper

Klipper is run as a systemd service. It should not normally need to be restarted from the command line, but during development it can be necessary. 

systemctl restart klipper

The log for Klipper is located in /tmp. It can be viewed as: 

tail -f /tmk/klippy.log

Klipper can also be restarted from the OctoPrint interface. On the "Klipper" tab, press "Firmware" to restart the Klipper firmware.

Klipper-firmware-press.png

Running OctoPrint

OctoPrint is stated on boot. If there is a need to restart the daemon, use this command 

systemctl restart octoprint

Refactor build system

Refactor uses Ansible as the underlying framework to describe the desired system state. There are many reasons for this:

  1. Ansible describes the system's end state, not individual actions to perform, so it is relatively easy to port across platforms (i.e. debian, arch, BSD...)
  2. The Refactor repository can be updated from the git repository and Ansible will only change what has been updated from the previous version, not needing a complete reinstallation of the system for upgrades.
  3. Description of components is modular, making it easy to pick and choose different components for the end goal desired, while keeping each component's setup independent and reliable, and independently updatable.

Why?

The ultimate goal is to have multiple decliations of Refactor, providing the user the ability to choose which firmware/software stack to run on their hardware. The following pre-build declinations are planned:

  • OctoPrint, Klipper, Replicape
  • OctoPrint, Klipper, Recore
  • OctoPrint, Klipper, Toggle, Replicape
  • OctoPrint, Klipper, Toggle, Recore
  • Duet Web Control, Klipper, Replicape
  • Duet Web Control, Klipper, Recore
  • OctoPrint, Klipper, Raspberry Pi [1-3]
  • OctoPrint, Klipper, Raspberry Pi 4 (arm64 edition)


Only the options in bold have been implemented so far, while the others are undergoing various steps in development in the order shown in this list. The Refactor images with OctoPrint are meant to provide drop-in replacements for the previous Umikaze images, including all the extra goodies that were present in Umikaze, like mjpg-streamer, SMB server, and usbreset for the main ones.

The default login is root with password kamikaze. The system will prompt for an immediate password change, to prevent a massive IoT exploitation through the web. Wouldn't want your printer sending out spam or starting to print unauthorized objects, now would we?

Building a runnable image for Replicape

The Replicape runs on a Beaglebone black (or green, without video output), which has a single core ARM processor running at 1GHz with 512MB of RAM. While impressive for an embedded application, it's rather weak when trying to build a system, which can take a lot of time.

There's a significant time gain when building the image from a more powerful host system, then flashing it to memory from an SD card.

The host system can be any ARM v7 or above architecture machine. In this example, the Raspberry Pi 4 will be used as an example, as it has a quad-core arm64 1.5GHz processor with 1 to 4GB of RAM. Any standard linux platform will do, but you will need a build storage space of at least 5GB for the build to succeed. Using a fast SD or a USB3 flash drive can provide higher disk speeds, reducing build time significantly.

To build an image from Refactor for the Replicape, follow these steps:

  1. Browse to a folder on the pi where you wish to run the build
  2. Clone the git repository for Refactor
  3. In the Refactor folder, run the build-image-in-chroot-end-to-end.sh script. It may take some time, so you may want to consider using a screen or byobu session to protect against network drops if you're doing so through SSH.
  4. If the build completes successfully, you should be left with a file 'Umikaze-console.img.xz'
  5. Use etcher to write the .img.xz file to a micro SD card, insert it into the Beaglebone, power it up and wait for it to power down.
  6. Start using the Refactor image on your printer

Installing a custom software stack with Refactor from a base Debian image

At the moment Refactor only supports using a debian-based distribution as a basis. Further down the line it will be refactored so that many more packaging systems are supported - this is not an Ansible limitation, but rather simply a developer time limitation.

The goal for Refactor is to allow the end-user to choose a custom software stack, while making installation of those packages easier. In this case, the user will want to learn how to write a simple Ansible playbook - there are multiple examples available in the Refactor base folder (build_full_klipper_octoprint.yml, install_klipper.yml, any yml file really).

The files starting with "build_full_" are meant to be used for the image building described in the previous section. The files starting with "install_" are designed to be run on a printer platform exclusively - not on a host system.

Each install_ file will install a specific component with its required dependencies. For instance, if you with install toggle, it will also install the Beaglebone's graphics drivers, required for video output, along with a number of libraries that Toggle is reliant on (such as a custom-built libclutter using the framebuffer output instead of the X11 driver).

To install a specific component, make sure that you have installed ansible. Alternatively you can just run the prep_ubuntu.sh script - the only thing it does is make sure that Ansible is ready to execute any further tasks required.

Once Ansible is installed, just run "ansible-playbook install_klipper.yml" for example. Let it run and it will go through to make sure everything is available upon the next reboot.

After all packages have been installed as desired, reboot the system. A reboot is necessary, as depending on the software installed, kernel modules may be enabled / disabled, kernel versions may have changed etc. These are unfortunately not changes that can be done without a reboot.

Retrieved from "https://wiki.iagent.no/mediawiki/index.php?title=Refactor&oldid=571"