Fedora 21 with bcache on a T430

Posted on January 22, 2015 by iainr

The linux partition on the work laptop was full and a power failure mid-rezizing partitions (top tip, never resize your partitions on a low battery near a child playing Disney infinity, It’ll end in tears) meant that it was time to think about upgrading from F20 to F21.

The First decision was whether or not to keep the copy of Windows that the laptop came with, given I’d only used it twice in the last six months…and then only to play Return to Castle Wolfenstein it seemed like a small sacrifice.

So to installing linux….Fedora kind of wins because work is rpm based and I like to have something fairly bleeding edge see what’s happening. Ubuntu or debian would I’m sure work equally as well, some of the tweaks I’m going to list will work equally well with those (in fact some are stolen from ubuntu support websites. At this point I should admit to making a small modification to the laptop. T430s are normally advertised as coming with an mSATA drive or a WWAN card in the mini PCIe slot (if you drop the battery out you can see where you’d fit a SIM card). My T430 was purchased off the back of a Scottish Government tender and they presumably see such things as foolish luxuries. I didn’t really want to add a third phone bill to the household accounts, that will come in about 10 years time no doubt, but a 32G mSATA SSD card at £26 seemed like a reasonable purchase(*).

Why a 32G SSD? Well admittedly it’s not quite big enough to comfortably fit the OS on, I did consider a 120G msata but at the time they were ~£120 thought I see they can be had for ~£60 now. 32G would allow two things.

  1. I would be able to do suspend to disk onto it so both suspend and recovery time would be improved
  2. from the 3.8 kernel onwards linux supports bcache which allows the SSD to act as a cache to a SATA hard disk.

So it’s a cheap cool toy that means I get to play with a new (well to me) linux kernel feature….what better way to spend £30.

Unfortuntately whilst F21 ships with both the kernel module and the supporting programs the F21 installer doesn’t so installing bcache is a little tricky. It may be possible to retroactively add bcache to a partition but I’m not sure and I managed to do the install fairly painlessly.

I mainly followed the instructions at this F20 walkthrough until starting the installer. my main differences were I decided to boot using Uefi so instead of having a 2M boot partition I created a 500M /boot partition (ext4) and a 200M /boot/efi(vfat with boot and esp flags) partition. Also as per most DICE machines I put in a 9G afs cache partition (ext4) finally I split the SSD into two 16G partitions, one for bcache and one to suspend to disk onto. Finally in my case I needed to run make-bcache –wipe-bcache -w512 –discard –writeback -B /dev/sda5 -C /dev/sdb1 on the make-bcache command

Start the install and then go into the installation destination menu option. Unlike the instruction on the link I had to manually configure the partitions. When adding whichever partiton you’ve added the SSDto make sure to use the bcache device and not the raw HDD. There’s currently a push for all faculty/University laptops to be encrypted so I took the opportunity to tick “encrypt” on all the partitions that would accept it, I think everything other then the /boot and /boot/efi partitions.

Continue the installation as normal BUT BE SURE TO ADD BCACHE SUPPORT to the installed system after you quit out of install to disk. If you don’t then the machine won’t get very far booting.

Once it’s all done then reboot and, if you’ve enctypted your disks type in the passphrase, Bobs your uncle.

More stuff about customising F21 coming later.

If you’re interested in bcache then you might be interested in some performance stats.

(*)School officials please note that since paid for upgrades at point of purchase were considered gifts to the school I am lending you this SSD whilst the laptop is in my possession, I shall retain ownership throughout.

Posted in Uncategorized | Tagged | Leave a comment

Fedora Workstation…your slip is showing

Posted on December 10, 2014 by iainr

So, having been busy with a procurement exercise this week and having failed in my battle to get into work in the face of a howling gale and flurries of snow I decided to sit down Coffee in hand and catch up on my email. Not the bori^h^h^h important work email but some of the mailing lists that I’m subscribed to.

Nothing very interesting or controversial until I get to this thread

https://lists.fedoraproject.org/pipermail/devel/2014-December/205010.html

It would appear that the newly minted Fedora 21 workstation ships with firewalling disabled above port 1024. If you’re not aware of the significance of port 1024, is the first port that can be used by non root processes. If you’re at home safely ensconced behind an ADSL type router with a sensible firewall this probably makes a lot of sense. If you’re going to use fedora in Informatics……not so much. This means that if you fire up a myslq database then there will be nothing between it and the rest of the network.
I’d advise changing to something more secure like

firewalld-cmd –set-default FedoraServer

or if you;re really paranoid

firewalld-cmd –set-default block

I’m not sure what the “work” zone allows but an Informatics zone might be an idea.

Posted in Uncategorized | Leave a comment

linuxcon Europe 2013 Edinburgh EICC – Day 2

Posted on November 7, 2013 by iainr

Small child was annoying this morning so thankfully I’d decided to park the bike just outside the EICC and change in the toilets because there was no time to walk from the office.

So keynotes mainly concentrating on the cloud.

Introduction from Mat Devine

A panel on cloud computing
including a question from Colin Higgs from engineering. Mostly is was all about cloud and agile development, getting products to service quickly and difficulties working with companies that are still working on bare metal.

Keynote: Fueling Samsung R&D Innovation with Collaborative Open Source Development – Yannick Pellet, Vice President – Advanced Software Platform, Samsung Research America
Samsung has gone from an open source consumer to producer and are engaging with the open source community. They have set up a development lab to become involved in the community all a number of levels from developing their own code, through developing upstream and evangalising within other units in the company.

An Introduction to OpenDaylight’s First Release, Chris Wright Red Hat
gave a good overview of the opendaylight architecture. opendaylght is a project to provide a SDN abstracting out a controller into a number of layers from shims controlling the bare metal devices (mostly via openflow but also via snmp and various cli)

OpenDaylight In Practice, Giovanni Meo
This is a more in depth description of the core controller, this is a distributed/clustered logical controller which uses an abstracted software layer to provide an api to service a number of protocols which describe the network. The code is written in JAVA or other jvm languages. sadly this was in too much depth so I bailed to get to:

HPC computing http://sched.co/18Fuf2j
A good overview ofthe current state of the art with some specifics about how higher end specialised equipment (NUMA.GPU,Xeon5…) has found an io bandwidth issue…for example gpu processing requires routing IO from filesystem to the GPU ususally through the kernel and back out where giving the GPU direct access to the block device would be much quicker.

Keynote: Kernel Developer Panel: Core & Embedded – Greg Kroah-Hartman, Will Deacon, Sebastian Hesselbarth, Peter Zijlstra, Jon Corbet (Moderator)
Interesting and varied discussion on what it’s like to be a kernel developer and various issues with integrating ARM stuff.

What Science Fiction Can Teach Us About Building Communities – Dawn Foster, Puppet Labs
So anoher fluffy talk in similar vein to the Game of thrones one. lots of pretty good advice linked to various scifi books. I’ve now got http://www.amazon.co.uk/Redshirts-John-Scalzi/dp/0575134305/ref=reg_hu-rd_dp_img on my amazon wish list.

SSD and HDD Performance Testing – Christoph Mitasch, Thomas-Krenn
Really interesting talk about a benchmarking tool that does hdd and ssd benchmark testing, there was a bunch of good stuff about ssds and performance . Sadly the slides don’t appear to be anywhere.

Posted in Uncategorized | Leave a comment

linuxcon Europe 2013 Edinburgh EICC – Day 1

Posted on October 21, 2013 by iainr

So after travelling to exotic locations like Philidelphia and….Newcastle I get to go to Linuxcon which is a brisk 15 minute walk from the office. I was late leaving the house, getting to the office to change and fortunately enough so were the Linux Foundation people in starting the conference. Huge registration queue on arrival led to the organisers taking the sensible approach of advising people to just go to the keynote and register later. Sensible thing to do and seems to have worked out quite well.

I’m taking a mix and match approach to the schedule apart from the fact I’m interested in the opendaylight stuff and will probably go for a bunch of those talks.

Keynote State of the Union address by Jim Zemlin started 30 minutes late and led to eh rest of the day running 30 mins behind schedule. Very upbeat and whilst I’d tend to agree with most of it I have a slightly queasy feeling about including android with linux. For me it lives in a strange cul-de-sac of not quite being proprietary and not quite being linux. This theme was continued in the next talk “We won. What’s next.” Which suggested that the open-source community not rest on it’s laurels having turned a hobby OS into a world leader and creating open source hardware but extend the same attitude of openness and cooperation to other fields such as prosthetics and drug development.

Next up Big Data for Good or Evil Lessons from the NSA PRISM scandal. TBH I was kind of hoping for an in depth security analysis starting from the revelations and suggesting where to go next. Instead Jason Bloomberg talked around Management issues in handing BIG data, it was all good stuff but missing something.
slides

Last one before lunch Software Defined Networking in CloudStack. Pretty good talk through of virtualised networks from the cloudstack point of view not really a subject I know a lot about (but a little bit more now).

then there was lunch…. huge lunch break, restarted at…..well not restared yet at 2.46…..maybe we should have used this time to try and grab back some of the 30 minutes delay introduced into the programme.

More to follow.

Recent Advances in Linux Tracing – Elena Zannoni, Oracle
fairly comprehensive coverage of the current state of linux kernel tracing tools again not an area of expertise for me but very interesting, Some points; oracle are porting Dtrace to linux (slowly), distribution kernels tend to lag so far behind that the kernel developers are using tools (and versions of tools) that are not available to the end user

Grand Unification of ACPI-Based Device Hot-Plug – Rafael J. Wysocki, Intel Corp.
Really was everything I didn’t want to know about ACPI and was afraid to ask. A very low level talk about the workings of ACPI hotplug at the kernel level.
slides (I dare you)

Everything I know about the cloud I learned from Game of Thrones.
I had three things I could have gone to, after the ACPI talk I needed something….lighter and this certainly delivered basically a tapestry of cloud wisdom stiched together with aphorisms from the programme…..and actually less spoilers than suggested at the start.

Posted in Uncategorized | Leave a comment

Dropbox drops the box?

Posted on April 13, 2011 by iainr

We regularly get requests for software to be installed on DICE and we are generally happy to put anything on provided it meets certain criteria (which is in itself another blog posting). Probably the most popular one recently has been Dropbox. I’ve been meaning to write something about Dropbox but a recent story has kind of pushed it to the front of my to do list.

At it’s most basic dropbox offers a way of seamlessly synchronising your files across multiple machines; the fact that it offers offsite backup, file sharing, version control and supports multiple OS’s and devices makes it very attractive. Install the software and anything put in a designated directory, on linux ~/Dropbox, gets copied to the dropbox server and then replicated across any of your devices that have the software installed and are registered to your account (If you’re interested the features are listed on the dropbox site but read on a bit before you rush off and install it).

In terms of installing it on DICE I have a couple of issues…the first is that phrase “copied to the dropbox server”. Dropbox is a San Fransisco based startup and they use Amazon S3 for their filestore. Unfortunately they don’t guarantee where data is stored and whilst Amazon are certified to the EU-US safe harbour initiative Dropbox is not. Put anything holding personal information in your Dropbox and you’re probably in breach of the Data Protection Act. Also while they cheerfully state that your agreement with Dropbox is covered by Californian law the files you put in your dropbox are presumably subject to whichever law is in place anywhere DropBox/Amazon copies them. There’s nothing wrong with that in itself but files which are legal in the UK may not be so legal elsewhere in the EU, in the US or indeed California. Equally in the event of some kind of civil action the other party may decide to act under Californian Law, or Belgian law, for no other reason than it’s a long way for you to go to fight a case but it’s just round the corner for them. This is presumably just part of the price of using the cloud.

My other problem is more technical. From a System Administrators point of view it’s a horrible program. On first sight of the linux download page it looks like we have an open source application which someone has helpfully packaged up for most of the large distributions. Unfortunately, what you’d downloading is not the daemon that does the heavy lifting with your files, what you’re getting is a plugin to nautilus and an installer for the actual daemon. Dropbox itself is a proprietary daemon which is downloaded when you register your installation and takes up 25M of space in your home directory. AFAICT this software gets updated automatically on the fly whenever there’s a new version, seemingly with no notification. So from an admin point of view we have a 25M application that is using very expensive replicated, tape backed afs filespace when it could easily be sitting on the cheap non-networked, unbacked up disk in your desktop. 25M doesn’t sound much, but multiply it up by the number of DICE accounts we have and you’re into the tens of gigabytes. Looking at the files themselves that are shipped it’s even more muddy, there are a number of local versions of site wide libraries which can really add to the fun if you’re trying to debug things. Finally there’s one other file that makes the heart sink, _sqlite3. Sqlite is “a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.” unfortunately if it’s used unsympathetically it can cause real problems for network filesystem response. You may remember we had issues shortly after we deployed firefox 3 well….. step forward sqlite.

So here is the problem, from the users point of view we have a useful application that they can use for work or personal use to safely copy files around and has all kinds of funky features like version control and built in cryptography. I can see all this but also that there are possibly horrible legal wrangles involved in it’s use, it uses a database library we’ve already had issues with, if there are problems it’s not really possible for us to control the version that people are using or seriously debug it. Oh and the command line tools are not really rational under linux

-bash-4.1$ dropbox status
Dropbox isn’t running!
-bash-4.1$ ps auxww|grep dropbox
iainr 21297 0.0 0.0 4420 716 pts/10 S+ 13:34 0:00 grep –color=auto dropbox
iainr 30765 0.0 1.5 227224 31904 ? Ssl Apr12 0:23 /afs/zathras.org/home/iainr/.dropbox-dist/dropbox
-bash-4.1$ dropbox stop
Dropbox isn’t running!
-bash-4.1$ dropbox start
Dropbox isn’t running!
Dropbox is already running!
-bash-4.1$

So it’s not really a no-brainer for installing on DICE.

Then came this article by Derek Newton. It appears that your dropbox installation identifies itself via an alphanumeric host-id stored in ~/.dropbox/config.db. By copying this hostid you can access the particular dropbox without any password. I hadn’t really given much thought to how it does authentication but had thought it would be a bit more sophisticated than an obfuscated hash. This isn’t a shock horror security bug but it does mean that you have to be very careful who has access to your ~/.dropbox directory or equivalent. You may want to run fs listacls on ~/.dropbox right now to check it’s not world readable. Equally Dropbox don’t really make it easy for you to be a bit more secure. there’s no option to password protect the hostid, no ability to lock it to an individual IP address or set of addresses, no logging if the address using it changes. In fact the logging is very limited and inexact.

If you must use dropbox then by all means do so, but guard the ~/.dropbox directory with your life, revoke any key that seems to be doing anything un-towards and encrypt anything you put in there that’s important.

Posted in DICE, Uncategorized | Leave a comment