Tag Archives: linux

Quick-and-dirty partition resize with Acronis True Image

A little embedded Linux magic comes in unexpectedly handy. Years ago, I started using Acronis True Image to back up my office PC. Other backup packages are available, as they say, but it’s worked well enough for me. This week I had reason to use it in earnest.

img_20180619_112420

The PC is set up with two main partitions on the disk. There’s one for the operating system (Windows 10) and applications, and another with data on it, analogous to the /home partition on a Linux system. Inevitably, it turned out that the boundary between these partitions was in the wrong place. The operating system partition was full, and things were starting to fail. Time to make sure the backup is up-to-date and then to restore to differently-sized partitions.

The operation fell at the first hurdle. Acronis True Image will create a ‘recovery’ USB stick from which the PC boots. This contains a small Linux distribution (Acronis Linux) and a GUI for managing the recovery operation. My backups are on a network drive, which happens to be an Apple Time Capsule. Would the Acronis recovery system connect to the Time Capsule? Would it ever. I couldn’t even persuade it to ask for the username and password, though it could see the Time Capsule on the network.

In Acronis Linux, pressing ctrl-alt-F2 in time-honoured fashion brings up a command line. Yay! Power! A bit of digging reveals a command called asamba which can connect to SMB shares. Using that, I could connect to the Time Capsule absolutely fine and mount it to the local filesystem. So why couldn’t the GUI do it? Annoying.

My next trick was to copy the backup from the Time Capsule to a USB hard drive using another Linux PC. It took a while for the several hundred gigabytes, but it got there. Acronis recovery will read USB drives, so the next step should be easy, right? No. My USB drive happened to be formatted ext4. Though Acronis Linux is perfectly capable of mounting ext4 filesystems, the recovery GUI only acknowledges the presence of FAT and NTFS discs. Aargh!

The solution was to mount the USB drive on my Linux PC, then share it using Samba. Acronis recovery found and let me log on to the Samba share, and could thus see the drive and backups.

Changing the partition sizes was the next problem. The GUI only allows you to shrink partitions or to grow them in to free space. My disk didn’t have any free space. It was all partitioned and in use. I thought that shrinking the data partition would create free space that the operating system partition could grow in to, but I was caught out by the presence of a 500MB ‘recovery’ partition which Windows had sneaked in between the two. I don’t know what it’s for but I didn’t feel like taking the risk of destroying it. There are no tools in the Acronis GUI for deleting partitions and starting again, so I was stuck.

img_20180619_113338

Command line to the rescue! Ctrl-alt-F2 again, and there’s a perfectly good copy of fdisk available. It was a moment’s work to delete the existing recovery and data partitions, then ctrl-alt-F1 to the GUI. Restart the restore process and there’s free space available for the operating system partition to grow in to, and plenty of room to create new recovery and data partitions.

img_20180619_112939

The restore took a few hours, as expected, but was successful. The only wrinkle was that Acronis recovery had marked the recovery partition as active rather than the boot partition, so the PC complained that there was no operating system. Another trip in to fdisk to mark the boot partition active sorted that out, and now I’m typing this very text on the PC restored to working order.

Footnote: there is another Acronis restore tool intended for restoring backups to dissimilar hardware. This may have handled the partition resizing more elegantly, but I didn’t try it.

Advertisements

Cracking a password-protected PDF file

Suppose you asked an insurance company for a letter. The insurance company kindly sent it as a PDF attached to an email. Sensibly, they protected that PDF with a password which they told you over the phone. You wrote it in a notebook and then left the notebook at work over the weekend.

pdf-password

How could you read the letter in the password-protected file at home, then? Remembering that the password was definitely an English word, and all in lower case, a dictionary attack has got to be worth a try.

Linux provides some handy tools for this. There’s a list of English words in /usr/share/dict/words, and a suite of PDF tools which can attempt to open the file using a password, indicating success or failure. A few minutes with Python and:

#!/usr/bin/python
import os,sys

wf=open('/usr/share/dict/words','r')
while True:
  word = wf.readline().strip().lower()
  if word == '':
    print "No solution found"
    break
  print word
  cmdline = 'pdftotext -upw "'+word+'" '+sys.argv[1]
  result = os.system(cmdline)
  if result == 0:
    break

The same thing must be possible in a more hipsterly fashion using awk, but I couldn’t be bothered to figure out a sufficiently baroque command line.

By the way, the password was ‘orange’. Don’t tell anybody.

Systemd for Embedded Linux

Over the last few years, there has been a lot of controversy in the Linux world about systemd. As I understand it, systemd is intended to be a better-engineered, more powerful version of the motley collection of little programs and scripts which keeps the essential services on a Linux system running.

systemctl

The controversy arises because the original 1970s Unix way of doing things was to rely on a motley collection of little programs and scripts for everything, each of which was simple but well understood, and to knit them together to form a complete operating system. Systemd takes a different approach, using larger and more sophisticated components which are more dedicated to particular tasks, such as managing services or network connections. This is supposed to make it more efficient and easier to manage in the twenty-first century.

I’ve been doing some work recently on an embedded Linux system which runs on the latest version of Debian Linux, version 8 (‘Jessie’). Debian Jessie fully supports systemd to the extent that it seems to be the default way of doing things. I thought I’d experiment with it a bit.

When working on an embedded Linux system, I very frequently want to have a piece of my software run reliably at startup, get restarted if it fails, and be able to output logging information to an easily-managed place. In this case, my software provides a D-Bus interface to a piece of industrial electronics.

In the past I’ve relied on copying and pasting scripts from other pieces of software, and managing log files has always been a bit of a mess. It’s hard to do these things right, so re-inventing the wheel is too risky, which means that the best strategy is to copy somebody else’s scripts. I have never counted the hours of my time which have been wasted by dealing with awkward corner cases and peculiar bugs due to recycled scripts behaving in ways I hadn’t anticipated.

What does it look like with systemd? There are some helpful tutorials out there, including this one from Alexander Patrakov, so it didn’t take me too long to put together a service file which looks like this:

[Unit]
Description=My D-Bus Gateway
[Service]
Type=dbus
BusName=com.martin-jones.gateway
ExecStart=/usr/bin/my_dbus_gateway
Restart=always
[Install]
WantedBy=multi-user.target

I’ve changed the names to protect the innocent, but the contents of the file are pretty self-explanatory. The [Unit] section just includes a description which is readable to a human being. The [Service] section describes the service itself. In this case it’s of type  dbus, which means that systemd will check that the service name (com.martin-jones.gateway in this case) gets correctly published on to D-Bus. The Restart=always setting means that my software gets restarted if it exits. The [Install] section just indicates that this service should run when the system comes up in multi-user mode (like the old runlevel 5).

Having created this file, I simply copied it into /etc/systemd/system/my_dbus_gateway.service and, lo and behold, my new service worked. It was immediately possible to manage the service using commands like

systemctl start my_dbus_gateway.service
systemctl stop my_dbus_gateway.service
systemctl status my_dbus_gateway.service

Great! That’s exactly what I wanted.

Now for logging. I’d heard that systemd would log the stdout and stderr outputs of services into its own journal, and forward that to syslog as required. It does, but there’s a subtlety. Output from stderr appears in /var/log/syslog immediately line-by-line, but output from stdout gets aggressively buffered. This means that it gives the appearance of not working at all unless you explicitly flush the stdout buffer in your code using something like

fflush(stdout)

That’s the only wrinkle I came across, though.

In summary, using systemd’s facilities has made my life as an embedded Linux developer much, much easier and hopefully more reliable. That’s a good thing. My top tips for getting your software working under systemd are these:

  • Create your .service file using the recipe above and the documentation
  • Don’t forget to flush stdout if you want to see it in syslog.

Dealing with Shellshock on Debian Squeeze for ARM

Today’s announcement of the Shellshock Bash vulnerability had me worried. I run lots of Debian Linux systems, and they’re not all the latest version. Many are still Debian Squeeze (version 6) which no longer gets security updates as standard. That’s my fault, of course, and I should have upgraded, but I haven’t. Yet. Now I’m more motivated to do it. However, upgrading to Debian Wheezy (version 7) isn’t something I wanted to do in a hurry, especially on remote machines.

Debian have thought of people like me, and there is a ‘Long Term Support‘ option for Debian Squeeze, which is great, and includes the necessary security update to Bash. The trouble is, it only supports i386 and amd64 processors, and the machines I’m worried about are ARM (specifically armel) ones.

I was left with one option: build the new Bash from source. Fortunately, Debian Squeeze LTS has the source available, so I was able to do this. Here’s how. This might be useful to other Debian ARM users who are none too fastidious about keeping up to date.

I added the line

deb-src http://http.debian.net/debian squeeze-lts main contrib non-free

to /etc/apt/sources.list, and did

apt-get update
apt-get source bash

which fetched the source code. Then I had to build it.

cd bash-4.1
dpkg-buildpackage -b -us -uc

This complained bitterly about a load of missing dependencies, which I dealt with:

sudo apt-get install autoconf autotools-dev bison libncurses5-dev debhelper texi2html gettext sharutils texlive-latex-base ghostscript

which was a royal pain due to lack of disc space. Beware, these packages want about 180MB of disc space (plus about 80MB for the package downloads) so might need some care on a small system. I started by installing packages individually, doing ‘apt-get clean’ after each one, but texlive-latex-base is an absolute monster and I had to do some filesystem reshuffling to get it to install. I hope you don’t have to.

During the build (repeating the dpkg-buildpackage command above) the patch for ‘CVE-2014-6271‘ was mentioned, which was reassuring. The actual build process took a while – about half an hour on a 1GHz-ish ARM chip (a SheevaPlug).

The build completed successfully, so I was able to install the new package:

cd ..
sudo dpkg -i bash_4.1-3+deb6u1_armel.deb

and then start a new shell and try the test:

env X="() { :;} ; echo busted" `which bash` -c "echo completed"

on a ‘broken’ version of Bash, this will print

busted
completed

but on a fixed one, it prints

/bin/bash: warning: X: ignoring function definition attempt
/bin/bash: error importing function definition for `X'
completed

which is the right answer, and means that the vulnerability is patched. It worked!

I hear that the fix isn’t complete, though, so more work may be required later.

Orange Livebox 2.0, port forwarding and NAT loopback

For several years, I’ve been running a small server (a Sheevaplug, as it happens) connected to my home broadband connection. It’s not running anything public, but it’s simply handy to have somewhere I can log in to and do things with from anywhere on the internet. It used to be connected via Virgin Media’s cable broadband service, using an Apple Time Capsule as a router, and a dynamic DNS service to handle the name lookup, and it all worked just fine. I could access my server from computers both inside and outside the house.

Recently I moved house, and the new place has broadband provided by Telekomunikacja Polska masquerading as Orange. Their broadband deal comes with its own router which also provides the VoIP phone service as well as having something to do with the TV. It’s a Livebox 2.0. It would probably be possible to set up a different router to do the same job, but the thought of getting the VoIP stuff to work is too awful to contemplate.

livebox-2

The Livebox 2.0 is a fairly flexible box, and I managed to set it up to forward traffic on one port to my server. The server has a little script which updates its dynamic DNS entry, and that did exactly what it said on the tin. I could log in to the server from elsewhere on the internet. But could I connect to the server’s public IP address from inside the house? No way. The router just reported ‘no route to host’. This was annoying. I have various things set up on my laptop which assume they’ll be able to connect to my server, and to have those not work when I’m at home would be a real pain.

The problem seemed to be that the Livebox didn’t want to route traffic from the internal network, where my laptop was, to its external IP address. Reading Wikipedia on NAT, it turns out that this is a feature called NAT loopback. I didn’t even know it was possible for routers not to support it, but I learn something every day. The Orange Livebox 2.0 won’t do it (and yes, I played exhaustively with all the security/firewalling/port forwarding options).

This called for a workaround. My server has a DNS entry with a dynamic DNS provider. Let’s call it server.dyn.com (it’s not, actually). This resolves to whatever the external IP address of my router happens to be – let’s say 203.0.113.1. From inside my home network, the router makes a mess of handling traffic to 203.0.113.1 so I can’t contact it. However, I have control of DNS on the home network: it’s one of the services on my Sheevaplug. I know that ‘spoofing’ DNS entries is bad practice, but I’m going to do it anyway. I can add entries to my local DNS server which make server.dyn.com look up to 192.168.1.10, the local address of the server. The laptop won’t know the difference, but it’ll be able to contact the server. And outside the home network, there’s no chance of anyone, including me, accidentally contacting my DNS server because there’s no way for traffic to get to it. Here’s a picture.

Img_6117s

I use bind9 as my internal DNS server. I added a section to /etc/bind/named.conf.local:

zone "dyn.com" {
  type master;
  file "/etc/bind/db.dyn.com";
};

and then created another file /etc/bind/db.dyn.com:

$ORIGIN .
$TTL 604800 ; 1 week
dyn.com IN SOA localhost. root.localhost. (
  2009060801 ; serial
  604800 ; refresh (1 week)
  86400 ; retry (1 day)
  2419200 ; expire (4 weeks)
  604800 ; minimum (1 week)
  )
  NS sheevaplug
$ORIGIN dyn.com
example A 192.168.1.10

It worked! At home, example.dyn.com resolves to 192.168.1.10, and out on the internet, it resolves to 203.0.113.1. I think it’s far from perfect – the timing settings on my dyn.com record probably need to be made shorter, so that the laptop doesn’t hang on to them when I leave home, and this setup means that all other subdomains of dyn.com don’t work, but it’s good enough for me for now. Maybe it’ll be helpful for you too.

Installing balloonboard.org Debian Linux build on a Raspberry Pi

I’ve just successfully got Debian Linux running on a Raspberry Pi, having built it entirely from the balloonboard.org distribution. Why might you want to do this? Well, I did it because it gives me a small, clean Debian Linux installation which I can then customise. Here’s how I did it.

Get the software:

svn checkout svn://balloonboard.org/balloon/branches/menuconfig2
make menuconfig
  • Mode Expert mode
  • Balloon Board Raspberry Pi board
  • Choose which buildroot version to build -> Feb 2013
  • Select kernel version 3.8 (rpi)
  • Select Build boot image
  • Select Build kernel modules
  • Select Build initrd kernel
  • Select Build Raspberry Pi boot patition image
  • Select Build Debian Root Filesystem

Now type make and it should all build.

Create yourself an SD card with two partitions on it: one smallish FAT partition for the boot files, and a big ext4 one for the root filesystem. Don’t forget to format them.

Into the FAT partition copy the contents of build/kernel/rpi-initrd-boot. Into the ext4 partition untar the file build/rootfs/debianrootstrap.modules.tgz.

Now put the SD card into your Pi and switch it on. This should boot into a ‘recovery kernel’ which has a minimal root filesystem in its initial ramdisk, just enough to sort out the ‘proper’ root filesystem. I used the console serial port to work with it. The HDMI and USB ports might also work but I haven’t tried them. It should come up with a login prompt. Log in as root, password rootme.

Now to configure the root filesystem:

mount /dev/mmcblk0p2 /mnt/root
chroot /mnt/root

Finish the Debian installation (this has to be done now because some aspects of it need to run on the ARM processor, so it’s not easy to have your PC do it). It will ask you questions about time zones and things:

/var/lib/dpkg/info/dash.preinst install
dpkg --configure -a

Set a password for the root account so you can log in

passwd root

and add the serial port to the list of secure ports which are considered safe for root logins:

echo /dev/ttyAMA0 >> /etc/securetty

And shut things down

halt

That last step will probably produce an error message, but it doesn’t matter as long as it’s written everything to the SD card.

Now put the SD card back in your PC, and copy the contents of build/kernel/rpi-boot into the FAT partition. That contains the real kernel which will mount the newly-minted root filesystem. Put it back into the Pi and boot. It will ask you to change the root password at first login.

It worked for me, though I had to ‘ifdown eth0’ and ‘ifup eth0’ again to get Ethernet to work. From that point on I was able to install Debian packages normally.

How to get the second Raspberry Pi I2C bus to work

One of the useful interfaces on the Raspberry Pi is the I2C bus. Originally invented by Philips in the 1970s for controlling functions inside consumer electronics, especially TVs, it’s still very handy for connecting up lowish-speed peripherals to computers. Some of the most popular are real time clocks like the MCP7940, amongst many others, and general purpose input-output chips like the venerable PCF8574. One of its convenient features is that it only involves two wires: SCL (clock) and SDA (data).

pi2c

The Raspberry Pi originally exposed one I2C bus on its GPIO connector, P1. It had another I2C bus dedicated to the camera connector, S5. However, with revision 2 of the Raspberry Pi, another connector was added. This was P5, squeezed in next to P1, and it also carried the second I2C bus, making it easier to get at and use. However, for some reason the two I2C buses got swapped over between revision 1 and revision 2. And to add a further layer of complication, the camera connector and P5 are wired to different GPIO pins on the processor, even though they are both logically the same bus. I’ve tried to summarise the situation in the table below.

Revision 1 Revision 2
GPIO0 SDA0 P1 pin 3 SDA0 S5 pin 13
GPIO1 SCL0 P1 pin 5 SCL0 S5 pin 14
GPIO2 SDA1 S5 pin 13 SDA1 P5 pin 3
GPIO3 SCL1 S5 pin 14 SCL1 P5 pin 4
GPIO28 SDA0 P1 pin 3
GPIO29 SCL0 P1 pin 5

Working on the CUED MDP project recently, I had a need to get the both I2C buses working on a revision 2 Raspberry Pi. There was no problem with the one on P5: typing

i2cdetect -y 1

showed the devices I had connected to it. But

i2cdetect -y 0

showed nothing at all, and examining pins 3 and 5 of P1 showed no activity. Disappointing. After a bit of digging, it seemed to me that the standard Raspberry Pi Linux kernel configures the processor to use GPIO 0 and 1 as I2C bus 0, and GPIO 2 and 3 as I2C bus 1. I wanted the bus on P1 to work, so I needed GPIO 28 and 29 to be my bus 0.

The BCM2835 processor on the Raspberry Pi, like most modern integrated processors, can have its pins programmed to do various different functions. In this case I needed to disable I2C on GPIO 0 and 1 and enable it on GPIO 28 and 29. To do this I enlisted the help of Mike McCauley’s BCM2835 library. It makes reprogramming the GPIOs fairly straightforward.

To install the library on your Raspberry Pi, make sure it’s connected to the internet, then:

wget http://www.airspayce.com/mikem/bcm2835/bcm2835-1.26.tar.gz
tar xzvf bcm2835-1.26.tar.gz
cd bcm2835-1.26
./configure
make
sudo make check
sudo make install

By the time you read this, there might be a new version of the library, so check on Mike’s site to see what you’re getting.

The C code to set up the bus looks like this:

#include <bcm2835.h> 

#define BCM2835_GPIO_FSEL_INPT 0 
#define BCM2835_GPIO_FSEL_ALT0 4 

main() {
    bcm2835_init(); 
    bcm2835_gpio_fsel(0, BCM2835_GPIO_FSEL_INPT); 
    bcm2835_gpio_fsel(1, BCM2835_GPIO_FSEL_INPT); 
    bcm2835_gpio_fsel(28, BCM2835_GPIO_FSEL_INPT); 
    bcm2835_gpio_fsel(29, BCM2835_GPIO_FSEL_INPT); 

    bcm2835_gpio_fsel(28, BCM2835_GPIO_FSEL_ALT0); 
    bcm2835_gpio_set_pud(28, BCM2835_GPIO_PUD_UP); 
    bcm2835_gpio_fsel(29, BCM2835_GPIO_FSEL_ALT0); 
    bcm2835_gpio_set_pud(29, BCM2835_GPIO_PUD_UP); 
}

It initialises the library, sets GPIO 0 and 1 as normal inputs (thus disabling their I2C function) and enables GPIO 28 and 29 as alternate function 0 (I2C bus), with pullup enabled. To build it, cut and paste the code into a file. I called it i2c0.c. Then compile it:

cc i2c0.c -o i2c0 -lbcm2835

and run it:

sudo ./i2c0

Now I2C bus 0 will be active on P1. Well, it worked for me. Once the code is compiled, of course, you can just run ‘i2c0’ after booting the Pi.

I realise that in future this will probably be made obsolete by changing the device tree sent to the Linux kernel at boot time, but for now it’s useful!