Rigol oscilloscope binary file parser

Introduction

So, just a day before my vacation trip I got a brand new RIGOL DS1052E. I was very eager to get back from my travels to check out my new scope. Unfortunately, I soon discovered that there is no good program to parse the native wfm files generated by the scope.

There already exist several attempts in generating a parser for the file format. The first attempt seems to be a script for Matlab. But this script suffers from not parsing all the available information and doing errors in alternate trigger mode. There are several derivations of this code on the Internet – all suffering more or less from the same problems.

New code

The problem was finally solved by Dexter who successfully reverse-engineered most fields of the Rigol wfm file format and uses it in its awesome wfm_view program. Unfortunately, it does only work under Windows and closed-source. Fortunately, he opened up the source code regarding the parsing of the wfm file. While it is not usable in it’s own it can be used to extract a protocol definition from it.

Armed with this wealth of information, I’ve re-implemented the file parser in standard Python. The code has been tested with Python 2.7.3 and 3.3.0 and should work without any modification.

The code contains an example usage of the library, see wfmutil.py. Which can be used to

  • extract header information
  • export to CSV as if the CSV was generated on the scope
  • plot the waveform and its FFT. (This requires numpymatplotlib and scipy)

Example

Inspired by Dexter’s work, I’ve also put an antenna to one input of my Rigol DS1052E and could clearly see all the radio stations I have here:

Radio with a Rigol DS1052E

Get it

You can find the code on my Github page:

https://github.com/mabl/pyRigolWFM

Accessing an encrypted full disc image (LUKS;LVM)

So I typically use a full disc encryption with LVM over LUKS. So assume you have got an image from your harddisk via

dd if=/dev/sda of=image.img

You now want to access this data again – maybe you don’t even have the drive anymore. So here is a quick rundown how I did it just now.

First we look at the image:

% fdisk -l -u backup_x220_november_2012_sdb.img                        

Disk backup_x220_november_2012_sdb.img: 80.0 GB, 80026361856 bytes, 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000589c6

                            Device Boot      Start         End      Blocks   Id  System
backup_x220_november_2012_sdb.img1            2048      499711      248832   83  Linux
backup_x220_november_2012_sdb.img2          501758   156301311    77899777    5  Extended
backup_x220_november_2012_sdb.img5          501760   156301311    77899776   83  Linux

Great! So all partitions are in there! We will use kpartx to map this file to a block device which you can use just as your normal drives.

% sudo kpartx -a -v backup_x220_november_2012_sdb.img
add map loop0p1 (254:3): 0 497664 linear /dev/loop0 2048
add map loop0p2 (254:4): 0 2 linear /dev/loop0 501758
add map loop0p5 : 0 155799552 linear /dev/loop0 501760

Even better, now we have the partitions on the image mapped to /dev/mapper/loop0px. It’s now straightforward to mount the encrypted drive loop0p5:

% sudo cryptsetup luksOpen /dev/mapper/loop0p5 imgroot
Enter passphrase for /dev/mapper/loop0p5:

Now find and open the LVM drives inside:

% sudo vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "mikrocanonix" using metadata type lvm2

% sudo vgchange -a y mikrocanonix
  2 logical volume(s) in volume group "mikrocanonix" now active

% sudo mount /dev/mikrocanonix/root /mnt/

DONE!

Installing Debian on a Sheevaplug into Flash

Ok, so you are looking for a comprehensive guide on how to install a modern Debian system into the Flash memory of your Sheeva Plug? Your googling has paid of – look no further!

What we will do in this guide:

  • Update the bootloader to a recent version which supports direct booting from ubifs partions. So you can just use the stock Debian kernel without any hassle or SD card.
  • Install Debian to a ubifs filesystem on the Sheevaplug flash.

Continue reading

Fixing Authentication of ownCloud for webfaction

So these days I was playing with setting up ownCloud – a cool application to server your own content. I was especially looking into having an online calender which is not saved at Google.

Now, the installation is straight forward. But once you start pointing your Calender program or browser to http://DOMAIN.TLD/apps/calendar/caldav.php/principals/USERNAME the password dialog always popped up, no matter if the password was correct or not.

After some investigations it turned out that this is caused by the special setup of my webspace provider. Basically, ownCloud extracts the username and password from the HTTP header via mod_rewrite and provides it to PHP over environment variables. Unfortunately, this mechanism is disabled by webfaction.

Fortunately they also propose a solution: Instead of passing the header over environment variables I have to pass it over the URL via an internal redirect. To achieve this, I’ve inserted a new conditional rule into my .htaccess file:

RewriteCond %{HTTP:Authorization} ^Basic.*
RewriteRule ^(.*) $1?Authorization=%{HTTP:Authorization} [QSA,C]
RequestHeader unset Authorization

This rule is active once a authentication header is transmitted. In that case the Authorization header is inserted in the query string. Additionally the Authorization header is removed since we end up doing endless recursions otherwise.

The only thing left now is to also tell ownCloud how to parse this new part of the query string. For that just add the following code to  ./lib/base.php around line 350 inside the init() function:

//sett http auth headers for Webfaction workaround
if(isset($_GET['Authorization']) && preg_match('/Basic\s+(.*)$/i', $_GET['Authorization'], $matches))
{
	list($name, $password) = explode(':', base64_decode($matches[1]));
	$_SERVER['PHP_AUTH_USER'] = strip_tags($name);
	$_SERVER['PHP_AUTH_PW'] = strip_tags($password);
}

Congratulations! You have ownCloud up and running on webfaction.

HOWTO: Fixing a broken EDID EEPROM with a Bus Pirate v4

This post is about fixing a broken EDID ROM in a monitor. There are several ways to do this. Typically this involves writing the EEPROM with the Windows only tool PowerStrip (see here for a HOWTO) . Here I want to present an alternative solution using the Dangerous Prototypes Bus Pirate – a device which every hardware hacker with self-respect should have anyway. You should already be tech-savvy to attempt this procedure.

Introduction

So I own 3 Samsung Syncmaster 2343BW with a staggering 2048×1152 pixels of resolution on 23″. When I bought them 3 years ago – this was really amazing. (Nowadays, such monitors are not even for sale anymore… but there is the ipad… Hopefully Apple sets a trend here for higher resolution…

So one of these monitors often suffers from a broken EDID EEPROM. Since this ROM is used to store all important information for Plug&Pray, the monitor not being detected anymore at all. I had this happen to me 3 times during warranty and they always exchanged the whole motherboard.  But unfortunately continues to appear after warranty, so I set out to fix it myself.

Continue reading

ChibiOS based STM32 bootloader example

Hi there,

since I’ve finally but my last oral exam for this semester behind me there is more time for me to hack on various stuff again.

Some of you might now that I’m a big fan of of the ChibiOS paired with the awesome STMicrolectronics STM32F series of Cortex-M3 microprocessors. These days I’m more and more into developing products which will run and work i.e. in Belgium. So there is no easy way to just pop by and deliver a firmware update over JTAG or there must be at least one remote controllable computer with a JTAG probe – not an elegant solution.

But there is hope, the STM32 chips do include the possibility to update the Flash ROM from inside a boot loader. Now, there are examples for boot loaders but I found most of them to be too “hackish”. So I set out to write my own boot loader based on ChibiOS. The example is targeting the Olimex STM32P107 development board but should be easy to adapt to other boards. You can find the final result on my github page:

https://github.com/mabl/ARMCM3-STM32F107-BOOTLOADER

The example includes helper function abstracting the flashing process. The example implementation demos reading the firmware in Intel Hex format from a SD card.

 

 

Hacking Basler BIP2 cameras

In the previous post I’ve praised the Basler BIP2 IP cameras for their great hardware and software. But sometimes using the build in software is not enough and the hacker in me always wants to have administrator (root) access to all Linux systems I own, be it my Laptop, Router or just my ebook reader.

 

Finding a hole

The firmware of the camera is not intended to be accessed by the user. So there is no telnet/ssh interface.

Finding a serial port

First I hoped to find a serial port, which would allow me to communicate with the u-boot bootloader. Sadly, there is only a RS-485 interface in the back of the camera which I could not connect to my laptop. I also tried to probe the PCB which has a 4 pin connector, but there is no serial signal on it.

Probing the firmware update mode

My next step was to investigate firmware updates which can be uploaded via the web interface of the camera. Luckily Basler provides a firmware update for its BIP2 cameras on their webpage.

Cracking the ZIP file

The update file is called FW-3.5.0.bin and is a password encrypted ZIP archive.

$ unzip -l -v FW-3.5.0.bin
Archive:  FW-3.5.0.bin
3.5.0
 Length   Method     Date    Time   Name
--------  ------  ---------- -----  ----
     260  Defl:X  2011-10-24 17:19  compcheck.sh
       0  Stored  2011-10-24 17:19  data/
      16  Stored  2011-10-24 17:19  data/bf.version
   32768  Defl:X  2011-10-24 17:19  data/image.b_53.ubl.bin
     849  Defl:X  2011-10-24 17:19  data/bf.sh
  294912  Defl:X  2011-10-24 17:19  data/image.b_60.u-boot.bin
   32768  Defl:X  2011-10-24 17:19  data/image.b_51.ubl.bin
    1323  Defl:X  2011-10-24 17:19  data/check.awk
25436160  Defl:X  2011-10-24 17:19  data/usr.ubifs
   32768  Defl:X  2011-10-24 17:19  data/image.b_55.ubl.bin
  425984  Defl:X  2011-10-24 17:19  data/image.b_25.u-boot.bin
  393216  Defl:X  2011-10-24 17:19  data/image.b_1.ubl.bin
    1307  Defl:X  2011-10-24 17:19  data/info.txt
  294912  Defl:X  2011-10-24 17:19  data/image.b_78.u-boot.bin
   32768  Defl:X  2011-10-24 17:19  data/image.b_57.ubl.bin
  294912  Defl:X  2011-10-24 17:19  data/image.b_96.u-boot.bin
       9  Stored  2011-10-24 17:19  info.txt
     244  Defl:X  2011-10-24 17:19  Readme.txt
   63490  Defl:X  2011-10-24 17:19  ReleaseNotes.txt
    4957  Defl:X  2011-10-24 17:19  update.sh
--------                            -------
27343623                            20 files

 

The above investigation yields the structure of the encrypted zip archive. Unfortunately the password used to encrypt this file was vision01 which  was to long to break via Brute-Force (it would have taken 10 days to check up to 8 characters even with GPU support).

Building our own firmware

Knowing that it was impossible to crack the password in reasonable time, I resorted to a new strategy: If we cannot crack the firmware update, why don’t we write a new unencrypted one?

It turns out that this is indeed possible! After some experimentation I found the minimal requirements for a BIP2 firmware upgrade:

  • /info.txt    Holds information which is displayed on firmware update
  • /update.sh  Shell script which is invoked on uploading the firmware and upgrade. The output of the script is printed on the web page.
  • Some padding file which inflates the zip file size since the firmware upgrade web interface will complain if the file is too small.

Enable SSH via firmware update

This allows us to execute any command as root user. So after some experimentation I used this way to enable the Dropbear SSH client on camera. The required steps are:

  1. Resetting the root password to a known value to allow remote logins
  2. Enable dropbear in /etc/default/dropbear
  3. Start dropbear service
Unfortunately it is not trivial to reset the root password since the passwd command reads from stdin, so there is no simple way to replace the password. For now, I’ve chosen the simplest way: Replacing /etc/passwd  with a new file with known password.
If we incorporate all theses steps, update.sh looks like the following:
#!/bin/sh
echo "This is a hacked update.sh script to enable Dropbear SSH"
echo
echo
echo "Arguments given to update.sh are:"
for ARG in $*
do
    echo $ARG
done
if . "/etc/bip.model"; then
    :
else
    echo "Failed to read environment"
    exit 1
fi

# Reject non-root callers
if [ `id -u` != "0" -o `id -g` != "0" ]; then
    echo "Script must be called by root:root\n"
    exit 1
fi

# Save zip file name
[ -n "$2" ] && FWFILENAME="$2"

# Print out current ZIP file password
echo Firmware password: `cat /etc/fw_pwd`

echo "Old passwd content was:"
cat /etc/passwd

echo "Set root password to 'pass' by copying new password file..."
unzip -P "$FWP" -qo "$FWFILENAME" "passwd.new"
cp passwd.new /etc/passwd

echo "New passwd content is:"
cat /etc/passwd
echo

echo "Activate SSH in config file..."
echo "NO_START=0" > /etc/default/dropbear

echo "Run dropbear SSH server..."
/etc/init.d/dropbear stop
/etc/init.d/dropbear start

echo "Reenable camera application"
camera start

echo "Finished update script, close this window and reconnect to your camera now."

Generated firmware for SSH access

An example firmware can be found here.

Note that all changes made from this script are temporary because the root file system is mounted in RAM. So the camera goes back to normal after one power cycle.

Now you can ssh into your camera and play with it via

ssh root@172.16.0.180
The authenticity of host '172.16.0.180 (172.16.0.180)' can't be established.
RSA key fingerprint is 88:24:f0:60:a6:a8:5c:a0:b6:5a:98:03:66:d4:63:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.0.180' (RSA) to the list of known hosts.
root@172.16.0.180's password: pass

BusyBox v1.17.3 (2010-11-15 09:01:13 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

root@Geutebrueck-xxxxxx:~#

Now go forth and play with your new camera. You mostly don’t have to fear to corrupt anything because the root file system is loaded from NAND to RAM during bootup.

root@Geutebrueck-xxxxxxx:~# cat /proc/cpuinfo
Processor       : ARM926EJ-S rev 5 (v5l)
BogoMIPS        : 215.44
Features        : swp half fastmult edsp java
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant     : 0x0
CPU part        : 0x926
CPU revision    : 5

Hardware        : DaVinci DM368 BIP
Revision        : 0000
Serial          : 0000000000000000

 

Investigating the Basler BIP2-1600c (updated)

Marketing verbiage

I have long been looking for the optimal IP camera, not only from picture quality but also from an engineering point of view. BOSCH IP camera such as the NBN-921 have great dynamics but suffer under a lot of Software problem, such as garbled H.263 encoding under high bitrates as well as hardware problems with high heat dissipation. Where as most AXIS camera have adequate firmwares, but abysmal sensor performance.

But fear not for there is a new player on the IP camera market: Basler. Basler is already a big player on the ComputerVision market and extended its product range for security applications.

Now I have made quite extensive research in my demo BIP2-1600c which is branded by Geutebrück as TopBC-1188.

My demands

Most IP cameras are intended to be used for surveillance of public places with mostly static images and no real-time dependence. But of cause there are countless other applications where good IP cameras are needed — in my case it is outdoor optical character recognition on fast moving objects. Now this requires several unique features:

  • Absolute time synchronization in the millisecond regime for each frame
  • Good dynamic and fast adaptation to lighting conditions
  • Low noise sensor
  • External trigger for picture synchronization
  • High resolution
  • MJPEG stream with at least 10 fps for high-quality (>90) JPEGs

Most of these features cannot be found on typical IP cameras, so I was rather curious who Basler IP cameras handle these demands.

Investigation

Web interface

Basler Webinterface
Basler Webinterface - Stream configuration

The web interface is clean and structured and allows to configure all settings one is used to. Noteworthy exceptions:

  • There is no support for IPv6! Shame on you Basler.
  • Set Area of Interest (AOI) which allows for partial reading of the sensor
  • It is possible to change all settings via a simple HTTP-POST RPC interface – there is no XML-RPC
  • Can sustain one 12.5FPS high-quality JPEG stream at 19Mb/s
  • Each JPEG frame includes EXIF data holding
    •  I/O state infos
    • UNIX timestamp in microseconds!
    • Frame id

Hardware

The BIP2-1600c is a very small and light camera in a nice aluminium casing. Even under load, the camera only warms up slightly. The PCB design is beautiful, featuring a DaVinci DM368 (ARMv5TEJ) from Texas Instruments. The Sensor is interfaces to an Altera  Cyclone FPGA.

Trigger input

The hardware features a trigger input which allows to precisely time each frame which is guaranteed to be taken after up to 7 ms delay.

Real time trigger input and time synchronization

I thoroughly inspected the real time trigger input and time synchronization features of the BIP2-1600c camera. To test the trigger time delay, I used my Olimex STM32P107 which was generating an NTP synchronized second pulse. The pulse was connected to the trigger input of the camera which was itself also NTP synchronized to my host computer which displayed timestamp information on its display.

Using the above setup, the camera was always synchronized to within 8ms but showing a constant 90 ms offset in the JPEG timestamp which still exceeds my greatest expectations.

<update> I inquired about this problem with Basler Tech Support. They basically told me that the timestamp value also includes the exposure time and the sensor reading duration plus some jitter.

Now, instead of correcting for this time duration afterwards wouldn’t it be wiser to subtract the time difference inside the camera? Especially given that different models have different sensors.</update>

Licences

There is one small thing constantly bugging me with the camera though – licences. This camera uses Linux and a great deal of other GPL licensed software. Under the GPL you are required to supply the source code of the software on user demand as well as to notify the user about the GPL software.

Unfortunately, there is no source code downloadable from neither Basler nor Geutebrück. The only information regarding this issue is a small link to the license texts (but not code) on the web interface of the camera, but not in the user manual.

I have requested Basler to provide me with parts of the source code — especially the Linux and u-boot source code. Let’s see how they will respond.

Conclusion

The Basler BIP2-1600c is a great camera with potent hard- and software which has even more potential once the software matures further. I highly recommend looking at this camera.

Outlook

In the next post I will look into hacking this camera and gain root access, so stay tuned.