dm-cache in Slackware

Posted on 2015/03/12 by majek2015/04/07

bcache vs. dm-cache

I tested bcache about year ago on my spare/backup server and it worked quite fine. But since then every newer kernel had new issues, even on mailing list there were suggestions that bcache should be marked back as experimental. So, after some time I abandoned bcache and started to wait for something more stable and configuration friendly. In the same time dm-cache was also available, but it’s setup was quite challenging and dangerous (using dmsetup and raw calculated numbers – very easy way to make a horrible mistake). There was also quite friendly way to manage dm-cache using LVM2 tools, but only in plans 🙂

Right now, LVM2 have all promised featured for managing dm-cache, so I gave it a try 🙂

Just to sum up my experience:

bcache
- + quite good documentation about cache policies, monitoring interface, disk format, disaster recovery etc.
- + cache works on block device layer, so you can cache whole PV device regardless how many LV you have on top of it (one cache for all)
- + no need for resize as you are caching whole physical device
- + you can tweak cache parameters on the fly, including cache policy (writethrough/writeback)
- – you can’t convert backing device on the fly to cached with data on it
- – unstable
- ~ doesn’t require LVM support
dm-cache
- + stable
- + easy to setup
- + you can convert any LVM volume to use cache or not to use it, online, without any problem
- – one cache for one LV, so if you have multiple LVs and you want to cache it all, you’d need to divide cache device (ssd disk) to many small chunks for every LV – space wasted
- – very brief documentation – enough to set up things, but without any info about how to tune or monitor it
- – resizing LVs must be done by uncaching, resizing and caching again, so you loose all cache contents
- ~it’s part of LVM

Requirements

It’s time to get back to Slackware. Current version is 14.1, so everything described here is based on this version.

Recent kernel

Stock 3.10.17 kernel is little too old for good dm-cache work, so I recommend to get something at least from 3.14 series. You could compile it yourself or grab from slackware-current tree which have currently 3.14.33.

Recent LVM2

LVM2 version shipped with Slackware 14.1 is almost last version without dm-cache support, so there is a need to grab latest version and compile it. For complete support you need also thin-provisioning-tools with fsck-like tools for dm-cache.

Hard way (compile everything yourself)

Get sources of LVM2 package from ftp://ftp.osuosl.org/pub/slackware/slackware64-14.1/source/a/lvm2/
Get more recent LVM2 sources from ftp://sources.redhat.com/pub/lvm2/releases/LVM2.2.02.116.tgz
Get thin-provisioning-tools from https://github.com/jthornber/thin-provisioning-tools/archive/v0.4.1.tar.gz and save it as thin-provisioning-tools-0.4.1.tar.gz
Get my patch for SlackBuild from https://majek.sh/dm-cache/slackware-lvm2-dm-cache.patch and apply it in lvm2 directory:
patch -p0 < slackware-lvm2-dm-cache.patch
Make package:
sh lvm2.SlackBuild
Upgrade lvm2 package with new one found in /tmp
If you want cached root volume, you’d need also to patch mkinitrd and rebuild initrd.
Patch: slackware-mkinitrd-dm-cache.patch.
Apply:

cd /sbin
patch -p0 < /somewhere/slackware-mkinitrd-dm-cache.patch
Use it 🙂

Lazy way

Just get my lvm2 package and install it. You may also need mkinitrd patch (see above).

Configuration

Start is easy and well documented in man lvmcache.
Just quick howto assuming that you have volume group named vg_group, your physical cache device (ssd) is /dev/sdb1 and you want to cache volume named lv_home.

First, cache/ssd disk must be part of volume group:
vgextend vg_group /dev/sdb1
Create cache pool on fast device (default mode is writethrough – safer but slower, so for writeback, there is a need to add –-cachemode option):
lvcreate --type cache-pool --cachemode writeback -L 10G -n cacheX vg_group /dev/sdb1
Attach cache to logical volume:
lvconvert --type cache --cachepool vg_group/cacheX vg_group/lv_home
Enjoy 🙂

Christmas tree lamps with Arduino and WS2811 leds :-)

Posted on 2015/01/08 by majek2015/02/09

Somewhere at the beginning of November I discovered RGB LEDs with embedded controller: WS2812 and others. Idea was so cool for me that I had to own one to play with it 🙂
These chips became so common, that there are even put inside LEDs, like on this WS2812 diode:

Considering time of the year, it was obvious to me to make whole string of lamps for Christmas tree. Then I found on Aliexpress ready made chains of such LEDs. 🙂
So, my LEDs aren’t with chips inside, but with small PCBs sealed with some kind of hard silicone – maybe not so elegant as chip integrated inside LED, but cheaper – which counts if you buy 50 of them :-):

Day before Christmas I still haven’t done anything with these light, so it was last chance to do something useful with it.

Connecting everything was quite easy as there are only 3 components: 5V power supply, Arduino Pro Mini board and LED chain.

Programming protocol for WS2811 was more challenging – generated waveform must be very accurate, with some time intervals taking only 8 cpu Atmega328 clocks (at 16MHz). So, I chose assembler to do it (as usual 🙂 ).
To be more hardcore, it was 3am when I started debugging 🙂 Honestly, without my new oscilloscope, finding these 3 mistakes I’ve made, could’ve took hours. Especially that Chinese guy who sells these LEDs described them as ‘we sent WS2811. Or we may sent UCS1903’ – two different possible chips (!) with two different protocols and no one knows which version I’ve got.
Luckily for me. it was WS2811, so first guess and code worked.

I added few simple routines to move light across whole chain of LEDs and put everything on Christmas tree.

All code for this ‘one night project’ you could find on my Github page: https://github.com/majekw/wsled

This is how it looks like on my Christmas tree:

Initrd in Slackware

Posted on 2014/12/26 by majek2015/02/20

There is small simple howto with Slackware package mkinitrd, man page says a little bit more, more is kind of mystery 🙂
I switched from full kernel (in fact, my own compiled kernel) to initrd few years ago and since then I had learned a lot about it (sometimes in hard way) 🙂

So, this is my way (a little different than in man and readme) of doing initrd.

Installation

During Slackware installation you need to install:

kernel-generic (not kernel-huge!)
kernel-modules
mkinitd

and of course other packages you would like to have 🙂
Remember NOT to install kernel-huge package.

Install everything, setup lilo, timezones etc as usual. At the end DON’T reboot at the end of setup – system isn’t finished yet!

Now, it’s time to prepare new initrd.

Chroot to /mnt (where new system should be already mounted)
Make copy of default mkinitrd config:
cp /etc/mkinitrd.conf.sample /etc/mkinitrd.conf
Edit new /etc/mkinitrd.conf with your favourite editor and set few options:
- uncomment and set MODULE_LIST and ROOTFS to type of your root filesystem, for example
  ROOTFS="ext3"
  MODULE_LIST="ext3"
  
  You don’t need to worry about all related modules, they will be added automatically.
- if you use software raid, uncomment and set
  RAID="1"
- if you use lvm or encryption, uncomment and set
  LVM="1"
- I strongly recommend to set SOURCE_TREE and OUTPUT_IMAGE to some more unique names, for example
  SOURCE_TREE="/boot/initrd-tree-3.10.12"
  OUTPUT_IMAGE="/boot/initrd-3.10.12.gz"
  
  You don’t have to do this if you don’t want to play with custom kernels etc.
make new initrd with mkinitrd -F
add initrd image to lilo.conf adding it just after ‘image’ line, so it will look like this:
image = /boot/vmlinuz-generic-3.10.12
initrd = /boot/initrd-3.10.12.gz
get out of chroot and rebuild lilo with:
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
lilo -r /mnt

Right now you could reboot system and enjoy new Slackware with initrd:-)

What’s an advantage of this way instead of running mkinitrd with many bizarre options? See next part.

Normal usage

Sometimes you need to change kernel, upgrade or something like that.

There are 2 ways of maintaining Slackware with initrd:

Use one initrd for all kernels
Use separate initrd for every kernel or even every entry in lilo

One initrd for all

At start, make sure that you have commented out CLEAR_TREE or set it to 0 in mkinitrd.conf!

After installation of new kernel and running still from old one, you need to to set KERNEL_VERSION to version of your new kernel, for example

KERNEL_VERSION="3.17.4"

Then just run mkinitrd -F
That’s all about initrd – now image has required modules for both/all your kernels.
Of course you need new entry in lilo.conf for new kernel with exactly the same initrd as old one.
If you are making kernel upgrade (replace old kernel package) you don’t even need to touch lilo.conf. But in both cases, you must run lilo at the end.

From time to time, you could clean initrd tree from old modules from kernel you don’t use anymore. In this case just remove all directory with that modules from /boot/initrd-tree/lib/modules/old-kernel-version and regenerate initrd using as usual mkinitrd -F. And rerun lilo of course 🙂

Now, you probably see advantage of using mkinitrd.conf instead of remembering all those weird switches you used last time to generate this image.

Separate initrd for every kernel

This way have a little sense if you only use one root filesystem and only stock kernel which you sometimes only upgrade.

Separate initrd is even easier than ‘one for all’ attitude.
One drawback is that mkinitrd with -F flag only reads one default /etc/mkinitrd.conf file, so each time you have to copy saved configuration back to this file.
So, for first kernel you should customize at least SOURCE_TREE, OUTPUT_IMAGE and KERNEL_VERSION. Generate new initrd with mkinitrd -F, then make backup of mkinitrd.conf to for example mkinitrd.conf.3.10.17.
For next kernel with different options customize mkinitrd.conf again, create new initrd, make backup and so on.
That way you could have different initrds and saved all options to regenerate each of them. It’s safe attitude as playing with new kernel and initrd configuration you always have working old one.

Ok, that was easy part:-)
Next time I’ll cover how to combine initrd with encrypted root and even root on encrypted lvm 🙂

Update 2015.02.20

I found quite good article on the same topic at Slackware Documentation Project. So, it looks like i didn’t do so much research before writing this post 🙂

Picture thief

Posted on 2014/12/14 by majek2014/12/14

Recently I noticed in server’s logs that one of my blog pictures become very popular:

It’s from this post: RTL2832U spectrum analyzer – part 2

Sometimes people embed my schematic or pictures on forums to enhance discussion. But this time one seller from Italy used my picture in his eBay offer. Moreover, he embedded my (and not only my) full resolution picture there, so he is also using my server bandwidth for free. Everything without a word.

There are many ways of dealing with that type of abuse, but I decided to use it as free promotion 🙂

I prepared new edited version of this picture (also smaller one):

And set new rule in .htaccess to redirect all requests from eBay page to new file:

RewriteCond %{HTTP_REFERER} ebaydesc\.com/|ebay\.it/
RewriteRule ^wp-content/uploads/2013/06/sdrsharp\.png /wp-content/uploads/2014/12/sdrsharp-ebay.jpg [R=302,L]

Now, his offer shows my new picture 🙂

Soldering AWG10 wire to XT60 connector

Posted on 2014/08/16 by majek2021/04/18

If you need more flight time, more battery capacity helps (to some extent) 🙂
But most batteries are designed for very high current, wires and connectors are also choosen for this purpose.
It’s sometimes a problem. I use in my tricopter standard XT60 connectors as it’s amperage is good enought for this purpose. When I used 1800-2200mAh batteries it was ok, as there come also with XT60.

When I bought 4000mAh with HXT4mm connector, I also bought adapter from HXT to XT60 for it. Problem solved.

Soon 4000mAh was not enought for me 🙂
So, I bought 5000mAh, but this battery comes with 5.5mm bullet connectors. This time I decided to change connector to XT60 instead of messing with heavy adapters.
On Polish R/C forum there is a thread about such changes and most difficult thing for people is to solder XT60 to much thicker wires. My new 5Ah battery have 10AWG wire, so it’s quite thick. There are some ways for soldering these wires and I also have one.

Because ‘seeing is believing’ or ‘a picture is worth a thousand words’, I recorded how I solder such thick wire to XT60 🙂

What you need to make it:

tin ends of wire (mine were already tined as I desoldered old connectors instead of cutting it off)
turn pins in XT60 in opposite direction (you could use flat pilers to do it)
use ‘third hand’ – it’s very handy and also heat resistant
solder one wire at a time – it’s for your safety – don’t get a chance for short circuit
put tin into connector
put soldering iron into tin in connector and put wire there – this way area of heating is increased, so it’s easier and faster to do it
don’t hold wire in bare hands – use pilers, beacuse of heat
little patience depending on power of your soldering iron (mine have 65W) and experience

Good luck 🙂