Tektronix 549 Storage Oscilloscope, Restored

A couple of years ago, I restored this exquisite brute of an oscilloscope to working order. You can read the story starting with part 1. I never posted any pictures of the finished article, so here they are.

You can read more about the machinery at the TekWiki.

549_34a

549_front_off

549_34c

549_34b

549_rear

1a4

manuals

Switched on and working, with all four traces showing.

549_front_light

549_front_dark

I made a short movie of the storage functions working.

Cracking a password-protected PDF file

Suppose you asked an insurance company for a letter. The insurance company kindly sent it as a PDF attached to an email. Sensibly, they protected that PDF with a password which they told you over the phone. You wrote it in a notebook and then left the notebook at work over the weekend.

pdf-password

How could you read the letter in the password-protected file at home, then? Remembering that the password was definitely an English word, and all in lower case, a dictionary attack has got to be worth a try.

Linux provides some handy tools for this. There’s a list of English words in /usr/share/dict/words, and a suite of PDF tools which can attempt to open the file using a password, indicating success or failure. A few minutes with Python and:

#!/usr/bin/python
import os,sys

wf=open('/usr/share/dict/words','r')
while True:
  word = wf.readline().strip().lower()
  if word == '':
    print "No solution found"
    break
  print word
  cmdline = 'pdftotext -upw "'+word+'" '+sys.argv[1]
  result = os.system(cmdline)
  if result == 0:
    break

The same thing must be possible in a more hipsterly fashion using awk, but I couldn’t be bothered to figure out a sufficiently baroque command line.

By the way, the password was ‘orange’. Don’t tell anybody.

Reliable I2C with a Raspberry Pi and Arduino

There are many ways of connecting sensors and devices to a Raspberry Pi. One of the most popular is the I2C bus. It’s great for devices which don’t need to transfer too much data, like simple sensors and motor controllers, and it’s handy because lots of devices (up to 127, or even more) can be connected to the same pair of wires, which makes life really simple for the experimenter. I’ve mentioned using the I2C bus in another blog post, because sometimes a bit of software fiddling is needed to get it to work.

Recently I’ve been working on a project involving various devices connected to a Raspberry Pi. Some of them use I2C. The project is based around a breakout board I designed for the Multidisciplinary Design Project at Cambridge University Department of Engineering, in which students collaborate in teams to put together a robot. The breakout board is shown next to the Raspberry Pi in the photo below.

IMG_1381

It fits on top of the Pi, and has lots of useful features including a student-proof power supply, real time clock, accelerometer, space for a Zigbee module, analogue inputs, diagnostic LEDs and four motor driving outputs, all wired to convenient connectors.

The analogue inputs and motor outputs are implemented by a PIC microcontroller connected to the I2C bus. The software for the PIC was written by an undergraduate several years ago. It works well, but seems to have some odd habits. I found that it would apparently work, but sometimes an attempt to read data from the PIC would just fail, or return wrong data, and sometimes data would get written to the wrong register. At first I suspected a wiring problem, but examining the SDA and SCL signals with a scope showed nothing wrong. I tested another device on the same bus – a Philips PCF8575 I/O expander – and it worked perfectly every time. That narrowed the problem down to the PIC. Since there was nothing I could do about the PIC’s software, I had to find a workaround.

I spent some time experimenting with where the communications seemed to go wrong. Reading from an I2C device usually involves two separate operations on the bus. The first one tells the I2C device which register address we want to read, and the second does the actual read. The diagram below shows the sequence. The ‘control byte’ in each case sends the address of the I2C device (0x30 in this case) plus a bit indicating read or write.

smbus-transaction

I found a pattern in the failures. From time to time, the write operation which sets the register address would fail, reporting ‘I/O error’. After that, reading the data would return the wrong value. I modified my code so that if the write operation failed, it would retry a couple of times before giving up. It turned out that retrying was always successful, if not on the first attempt then on the second. However, the data read would still return the wrong value. The value returned was always the address of the register I wanted! It seemed as if something was getting stuck somewhere in the I2C system. Whether it was in the Linux drivers, or the PIC software, I don’t know, and I didn’t spend long enough to find out. My assumption is that the PIC software is sometimes just too busy to respond to the I2C operations correctly.

I tried the retry strategy again, and it turned out that the second attempt to read the data byte always got the right value. The algorithm to read reliably looks like this, in pseudo-code:

  if (write_register_address() fails)
    retry up to 3 times;

  read_data();
  if (we had to retry writing register address)
    read_data();

In practice I was using the Linux I2C dev interface to implement this. Yes, it’s a bit of a nasty hacky workaround, but it did get the communications working reliably.

There was another device I wanted to talk to: an Arduino Mini running a very simple sketch to return some sensor data. This also used the I2C bus. There are handy tutorials about how to get an Arduino to behave as an I2C slave device, like this one. The I2C interface is implemented nicely by the Wire library. Implementing a slave involves responding to two events: onReceive and onRequest.

The onReceive event is called when data, like the register address, is written to the slave, and the onRequest event is called when the master wants to read data. My initial code looked like this:

Wire.begin(I2C_ADDRESS)
Wire.onReceive(receiveEvent)
Wire.onRequest(requestEvent)

void receiveEvent(int bytes) {
  registerNumber = Wire.read();
}
void requestEvent() {
  Wire.write(registers[registerNumber];
}

This worked most of the time, but after a few thousand transactions, it would appear to ‘lock up’ and ignore any attempt to change registers – it would always return the same register, and in fact no more onReceive events were ever generated. Of course, it turned out to be my fault. When reading data in the onReceive event code, it turns out to be important to make sure that data is actually available, like this:

void receiveEvent(int bytes) {
  while(Wire.available())
    registerNumber = Wire.read();
}

That solved the problem. It’s annoying that reading non-existent data can lock up the whole I2C interface, so watch out for this one if you’re using an Arduino as an I2C slave.

Systemd for Embedded Linux

Over the last few years, there has been a lot of controversy in the Linux world about systemd. As I understand it, systemd is intended to be a better-engineered, more powerful version of the motley collection of little programs and scripts which keeps the essential services on a Linux system running.

systemctl

The controversy arises because the original 1970s Unix way of doing things was to rely on a motley collection of little programs and scripts for everything, each of which was simple but well understood, and to knit them together to form a complete operating system. Systemd takes a different approach, using larger and more sophisticated components which are more dedicated to particular tasks, such as managing services or network connections. This is supposed to make it more efficient and easier to manage in the twenty-first century.

I’ve been doing some work recently on an embedded Linux system which runs on the latest version of Debian Linux, version 8 (‘Jessie’). Debian Jessie fully supports systemd to the extent that it seems to be the default way of doing things. I thought I’d experiment with it a bit.

When working on an embedded Linux system, I very frequently want to have a piece of my software run reliably at startup, get restarted if it fails, and be able to output logging information to an easily-managed place. In this case, my software provides a D-Bus interface to a piece of industrial electronics.

In the past I’ve relied on copying and pasting scripts from other pieces of software, and managing log files has always been a bit of a mess. It’s hard to do these things right, so re-inventing the wheel is too risky, which means that the best strategy is to copy somebody else’s scripts. I have never counted the hours of my time which have been wasted by dealing with awkward corner cases and peculiar bugs due to recycled scripts behaving in ways I hadn’t anticipated.

What does it look like with systemd? There are some helpful tutorials out there, including this one from Alexander Patrakov, so it didn’t take me too long to put together a service file which looks like this:

[Unit]
Description=My D-Bus Gateway
[Service]
Type=dbus
BusName=com.martin-jones.gateway
ExecStart=/usr/bin/my_dbus_gateway
Restart=always
[Install]
WantedBy=multi-user.target

I’ve changed the names to protect the innocent, but the contents of the file are pretty self-explanatory. The [Unit] section just includes a description which is readable to a human being. The [Service] section describes the service itself. In this case it’s of type  dbus, which means that systemd will check that the service name (com.martin-jones.gateway in this case) gets correctly published on to D-Bus. The Restart=always setting means that my software gets restarted if it exits. The [Install] section just indicates that this service should run when the system comes up in multi-user mode (like the old runlevel 5).

Having created this file, I simply copied it into /etc/systemd/system/my_dbus_gateway.service and, lo and behold, my new service worked. It was immediately possible to manage the service using commands like

systemctl start my_dbus_gateway.service
systemctl stop my_dbus_gateway.service
systemctl status my_dbus_gateway.service

Great! That’s exactly what I wanted.

Now for logging. I’d heard that systemd would log the stdout and stderr outputs of services into its own journal, and forward that to syslog as required. It does, but there’s a subtlety. Output from stderr appears in /var/log/syslog immediately line-by-line, but output from stdout gets aggressively buffered. This means that it gives the appearance of not working at all unless you explicitly flush the stdout buffer in your code using something like

fflush(stdout)

That’s the only wrinkle I came across, though.

In summary, using systemd’s facilities has made my life as an embedded Linux developer much, much easier and hopefully more reliable. That’s a good thing. My top tips for getting your software working under systemd are these:

  • Create your .service file using the recipe above and the documentation
  • Don’t forget to flush stdout if you want to see it in syslog.

Lenovo Thinkpad T61 GPU fix. Or not.

My mobile workhorse is a trusty Lenovo T61, a close cousin of the ones they use on the International Space Station. How cool is that? It’s built like a tank, and weighs about as much, but the feature I appreciate most is its screen: a lovely 1680 x 1050 resolution, which is actually enough pixels to get some work done. Most laptops have rubbish screens which were clearly only designed for watching DVDs and reading Facebook. It’s a pet hate of mine.

IMG_1276

Sadly the Thinkpad has blotted its copybook. The Nvidia graphics chip has a reputation for expiring earlier than it should do, and this one has. It started with the screen going blank a couple of months ago. I replaced the screen, and then it worked but only with every other column of pixels. Then it spontaneously started working almost-properly, but with a peculiar green shimmering effect on bright colours which was just about possible to avoid by fiddling with the display settings. Then it died altogether, and just gave the ominous beep-bip-bip code at startup which means ‘my graphics hardware isn’t working’.

In a last ditch attempt to revive it, I thought I’d try reflowing the solder on the graphics chip. It helps on some laptops. In theory you need very specialised equipment for this, but I’ve had success in my professional life doing it with a more, shall we say, agricultural approach. With nothing to lose, I had a go. Here’s what I did.

Reflowing the solder on the graphics chip involves removing the motherboard, which requires completely disassembling the laptop.

Remove all the bits the come out: the battery, DVD drive, hard drive, and any PC card and SD card.

IMG_1237

Remove the marked screws  on the bottom to remove the keyboard, touch pad and palm rest.

IMG_1235

Unplug the three wires to the Wifi module. Plug 1 is grey, 3 is white, 2 is black.

IMG_1238

Remove the keyboard surround/speaker grills. Two short screws on the top, one long one from the bottom rear right corner, one long one from the outer rear bottom left corner, four little flat ones from the metalwork near the CPU.

Unplug the screen connector, the grey cloth one near the fan.

IMG_1242

Remove the two tiny screws holding the left speaker and move it to one side.

IMG_1243

Remove 2 medium-length screws from the bottom rear edge and two short screws from the screen brackets on the top side. Take the screen off.

IMG_1245

Remove one tiny silver screw from the front right hand side, 9 short screws from the bottom, three more long ones from the bottom, and four short ones with big flat heads near the docking connector.

From the top, remove the four medium-length screws holding the heatsink down.

IMG_1249

Remove the two short silver screws holding the two silver brackets on the right side of the heat sink.

IMG_1251

Unplug the fan connector and ease the heatsink/fan assembly free.

 

Img_1252

Unplug the multiway connector in the rear centre which feeds the USB sub-board. Tease the wire free from under its sellotape.IMG_1253

Remove the two medium screws holding the wifi module in and remove the wifi module.

IMG_1255.JPG

The motherboard and frame should now be free of the bottom case.

IMG_1256.JPG

Turn the motherboard over. Remove the 8 remaining screws with big flat heads. Remove the medium screw by the DVD connector. Remove the small silver screw by the SD socket. Remove the short screw holding the heatsink brace. Remove the heatsink brace.

Returning to the top of the motherboard, unplug the charger connector, the small connector by the phone socket, the speaker connector by the RAM and the little black connector at the back where the wifi module was. Leave the backup battery connected.

The magnesium frame should now be free of the motherboard. Take it off. Clean the heatsink compound from the tops of the large chips. U47, with the Nvidia logo on it, is the GPU.

To try and reflow it, I made a makeshift heatshield out of a doubled-up piece of kitchen foil. I’ve done some emergency BGA repairs in the past so I had some idea of what I was aiming at. My strategy is to heat the whole chip area while carefully poking some nearby easy-to-repair part to see when the solder has melted. I expected it to take a couple of minutes.

Img_1273.jpg

I did exactly that, but ran into a nasty problem: there is self-adhesive tape on the top of the GPU and nearby on the PCB, which shrinks in the heat. Unfortunately it takes components with it, so they end up on the tape instead of the PCB. It turns out that there are a couple of dozen parts – decoupling capacitors, I think – on the top of the GPU. Most of them had come off and stuck to the shrunken tape.

IMG_1274

In addition, the backup battery got itself in the way and overheated and burst during the reflow. There were also solder balls visible round the end of the GPU. I thought I’d pretty much wrecked it. But, having come this far, I decided to try and repair the damage and put it back together.

Reassembly is the reverse of disassembly, as they say in the Haynes manuals. Don’t forget to apply new heatsink compound to the tops of the chips in contact with the heatsink. I didn’t hold out much hope.

I got it all back together with no screws left over. I put the battery in, pressed the power switch, and apart from a brief flicker of the power light, nothing. Not a sausage, or even a beep. Game over. Time to go shopping for a new motherboard.

As luck would have it, the Thinkpad T61 was built in various versions. Only the really upmarket ones have the Nvidia graphics chip with the bad reputation. The cheaper versions use the graphics provided by the Intel motherboard chipset. I care more about reliability than I do about ultimate 3D graphics performance, so I decided to do a motherboard swap once I’d discovered that replacement motherboards were available from the US at about $30. It’s useful to know the part numbers. My original motherboard, an early one with Nvidia graphics, was a 42W7652. The ones with the Intel graphics are 42W7651 (early version, supports Intel Merom processors) and 42W7875 (later version, supports both Merom and Penryn processors). On the left the old motherboard, on the right the replacement one.

Img_1377

Swap the CPU and RAM to the new motherboard, and make sure you put the little foam block on top of the transistors by the VGA socket – it conducts heat to the heatsink. Also swap the PC Card cage, which is held by two screws underneath the motherboard. It pulls out vertically from the motherboard.

Getting the machine to boot after I’d swapped the motherboard took a bit of fiddling. It turns out that, because the backup battery had been removed, all the BIOS settings were lost and the SATA interface mode had changed from ‘Emulation’ to ‘AHCI’. The symptom was that the machine would start booting but Windows 7 would just bluescreen immediately. Changing that BIOS setting fixed the problem.

I didn’t have to re-validate Windows, but a couple of pieces of software got unhappy. The Visagesoft Expert PDF tools required me to re-enter the registration information I’d already paid for, which was no trouble, and the Xilinx and Lattice FPGA tools needed new host-locked licence keys generating, which was free.

The Thinkpad lives to fight another day, and to let me write this blog post.

 

Refurbishing a JVC 5 1/4″ floppy disc drive

A couple of months ago, I was given this splendid setup in return for some data recovery work. The original disc drives had stopped working, and my customer wanted the data from the discs. I used my office BBC B+ to retrieve the data successfully from the discs which were in ADFS format. This equipment – a BBC Master 128, Acorn AKF12 monitor and a pair of 40/80 track disc drives – was in such good condition that I decided to try and get it all working.

Img_0917

The monitor worked fine and the computer booted up quite happily, but the disc drives were definitely in trouble. The right hand one would just about read the first couple of tracks of a disc, but the left hand one wouldn’t read anything at all. I pulled the lid off to see what kind of drives were inside. The disc unit has a ‘Cumana’ label on the front, but they were a dealer and put together boxes with all sorts of drives inside, though usually of good quality.

Img_0918

There’s nothing complicated in there. A simple linear power supply at the back, the two drives, and the switches at the front with some distinctly dodgy-looking wiring. Was zip-tying the 40/80 track switch wiring directly to the 240V mains wiring really such a good idea? And I think I might have splashed out on insulated crimps for the mains connections on the back of the switch.

The drives themselves are ones I’d never seen before. They’re made in Japan by JVC, model number MDP-100. I associate JVC with 1980s consumer electronics rather than computer peripherals. They generally had a decent reputation amongst techno-savvy schoolboys. Their video recorders, especially, rebadged in the UK as Ferguson, were almost indestructible. The floppy drives should be worth saving.

Looking more closely at the drives revealed some interesting details. Underneath is a little brown add-on board with a bunch of TTL logic which does the 40/80 track switching. It looks like a factory modification, but the wiring to the main board (not visible in this photo) is a bit heath robinson.

Img_0920

The main board features a couple of 7438 TTL chips made by Tesla in Czechoslovakia. They were definitely a rarity on the western side of the iron curtain. Their date code (courtesy of tubes-store.com) is November 1983, which is the right sort of time. Why on earth would a Japanese disc drive contain Czechoslovak chips? It’s not as if there weren’t any Japanese ones. All the other TTL chips in the drive are made by Hitachi, and they’re 74LS series which was current at the time. Almost nobody was still using straight 74 series in 1983. My best guess is that there was some subtlety of the Shugart-standard floppy disc interface that needed standard TTL, and nobody else was still making it. Or they were just cheap.

Img_0921

The first problem I spotted was corrosion. The board has lots of horrible little electrolytic capacitors on it. The tiny ones like these are notorious for being badly sealed and oozing gunk which causes electrical faults, corrodes tracks and other components. Playing on the safe side, I whipped the lot off and replaced them with newer ones which were a bit bigger, and cleaned up the damage. No tracks seemed to have suffered, fortunately.

Img_0922

Next I inspected the heads. Squinting at them in position, I thought I could see some dirt, so I unscrewed the upper head to get a good look at both of them.

Img_1210

 

Yuk! I’ve never seen a floppy drive head so dirty. No wonder it wouldn’t read discs. The photo is a view of the upper head. The lower one was just as bad. A thorough scrub with isopropyl alcohol, being careful not to damage the rather delicate suspension, and they came up like new.

Img_1213

I put the upper head back on in roughly the right place and tried the drive again. It would now read discs, at least on the lower head (drive 0) but only intermittently, and wouldn’t work at all beyond track 2. Attempting to get to track 3 or beyond just resulted in a lot of rattling and a disc error.

I suspected a head stepper motor problem. There were two BA6208 chips visible on the PCB between the main JVC controller chip and the connector to the stepper motor, CN04.

Img_1307

I suspected they, or one of them, might be the culprit. This type chip and many similar ones were used for driving motors in lots of consumer electronics, and they did give trouble. I remember replacing them in VCRs and the like. The data sheet is easy enough to find on the internet. A quick check with the scope and sure enough, the right hand one wasn’t working properly. Each chip contains two drivers, and one of them wasn’t driving at all.

I found a replacement chip for about a pound on allegro.pl. Bargain! I fitted it and now had full head-stepping action. However, reading data still wasn’t reliable. Verifying a known-good disc would fail quite frequently, often in the same place. I had a look at the signal waveform coming from the disc. Fortunately the head amplifier chip, a Hitachi HA16631P, is also documented online. The read data signal is visible at pin 12. What I saw was this.

Img_1315

It was difficult to capture on the scope, but basically the waveform is of uneven size: as the disc rotates, it grows and shrinks, sometimes so much that errors result. I tried fiddling with the head alignment by adjusting the head stepper motor, but it didn’t help. Eventually I figured out that during my head-cleaning efforts, I’d managed to bend the thin steel tab which holds the upper head. Not much, but enough to prevent it exerting the right amount of pressure on the disc. This pressure also keeps the disc in contact with the lower head, which is fixed. I straightened it and tried again.

Img_1312

Much better! A nice fat waveform, with no odd variations. You can see the head mounting in the photo below. The two screws marked with a cyan dot mount the head assembly via its steel strip, and loosening them allows it to be aligned with the lower head. The screw between them marked in pink just attaches the copper-coloured bracket which holds a tiny coil spring against the back of the head assembly which adds a little more pressure, though it didn’t seem to make much difference in practice. Perhaps if the disc was wrinkly or something it might help.

Img_1308a

After all that I had a disc drive in perfect working order.

At some point I’ll do the second one and then the whole system should be OK.

Img_1316

Cracking the code: reverse engineering the AlcaTech BPM Studio controller

The recent demise of my old workshop PC has spurred me into action. Back in the halcyon days of the late 90s and early 2000s, I used to DJ a bit. I was an early adopter of computer-based DJ technology. In those days it was still fairly unusual to have music in MP3 format – the iPod wasn’t released until late 2001 – never mind being able to actually DJ with it. At the time I found a product from a German company which did exactly what I wanted: it was a combination of hardware and software called BPM Studio which meant I could use MP3 files as if they were a professional CD player: cueing them, pitch-shifting, mixing and so on. The hardware is a solidly-built control panel which connects to the PC, on which runs some software which does all the audio processing.

Img_1278

So why has the failure of my old workshop machine reminded me of this? Because, once upon a time, it was my media PC, and it ran the DJ software. The software and hardware is now 15 years old, and it shows: the controller connects using a serial port (when did you last see a PC with one of those?) and the software has…wait for it…a dongle! Yes, just like in the bad old days, it has a device which plugs into the parallel port on the PC, and if it’s not found, the software won’t run. Parallel ports, especially, are a dying breed today, so the chances of being able to use this controller and software in the future fade as PC technology moves on.

The BPM Studio package was expensive, and the controller is quite nice and robustly built, so I’d like to be able to preserve it and, if possible, use it with more modern software. The trouble is, its interface to the PC is proprietary, unsupported by any other software, and I couldn’t find any documentation on it. There was only one way forward to protect my investment: hack it.

The first step was to have a look at what was going on on the serial connection, and what baud rate was in use. I put a little breakout adapter in the serial cable (this one, in fact, modified a bit) so I could examine the data.

IMG_1283

First thing was to figure out the baud rate. Set the scope to trigger on the rising edge and play with the timebase a bit and soon we can see the start bit of each byte:

Img_1282

That’s definitely 19200 baud. Nice and standard. So I started my handy hdump2 software, which displays two streams of data side by side so it’s possible to see what came from where and when, and hoped to see something which made sense: a recognisable packet format, perhaps, or at least consistent data. What I got instead was this:

bpmstudio-startup

The left column is data from the PC, and the right column is data from the controller. It’s clear there’s a conversation going on, but it looks encrypted to me. There are no obvious packets, no start or end markers, nothing clearly related to what’s going on. I played around pressing buttons on both the controller and PC, and lots of data flowed but nothing made any sense. No readable track names for the displays, no recognisably similar data when I pressed the same button numerous times.

Why on earth would anyone encrypt the connection between the PC and a controller like this? Only the designers know, but I guess it’s part of the same mindset that required a hardware dongle to run the software. A fear of piracy, probably.

Interestingly, if the PC and controller are separated, they each send out  a burst of data once a second. The PC sends bursts of 4 bytes, and the controller sends bursts of 12 bytes. Each of them follows a fixed pattern from startup. The PC sends:

9c 94 dc 0e
56 1e 97 95
ad f8 87 4a
dc bc f0 37
32 44 bd a1

and so on. The controller sends:

f0 99 d0 af 3b 2f c8 5b 21 3c 4f d4
44 95 ac e1 d9 76 2a 58 bf 1e 52 52
34 e7 1a 93 ce b1 97 3e a4 f9 01 37
d3 f3 94 c1 32 57 31 a7 9a 6c 83 68
84 ae d1 f6 e7 c1 c8 5d e2 e4 46 36

and so on. I can’t see an obvious relationship between them. What I can see from the conversation dump above is that the controller seems to restart its sequence when it sees the data from the PC, but with some subtle differences.

If I was a proper mathematician, I’d spend more time trying to work out what the code was. Being an engineer, I thought I’d take it apart and have a look inside.

Img_1285

There’s more to it than I thought. This predates the days of powerful PIC and AVR microcontrollers, and actually has separate chips for its CPU, ROM and RAM. That’s good news for anyone interested in reverse engineering it. The CPU is a Siemens/Infineon 80C166:

Img_1287

and there’s a 29F010 (128 KByte) ROM connected to it, presumably holding the software:

Img_1288

There’s also a 32K RAM chip, which is more than I’d expect. I just hope the software isn’t doing something horrible like decrypting itself into RAM and running from there.

The good news is that documentation for the 80C166 is freely available, as is a disassembler, ADIS16X. And I’ve just ordered a PLCC32 adapter so that the ROM, once I’ve desoldered it, will fit my EPROM programmer. Watch this space.