LCFG Client: It lives!

Posted July 15, 2016 by squinney
Categories: Uncategorized

Tags: ,

Cue forked lightning and crashes of thunder…

After much effort I finally have the first functional installation of the v4 LCFG client. This sees all the XML parsing and profile handling moved over to the new LCFG::Profile Perl modules which are wrappers around the new lcfg-core suite of libraries. There is still a bit of work required to properly handle LCFG contexts but otherwise it can handle everything we need. There are probably lots of small bugs to be resolved, there is also an almost total lack of documentation and the tests needs lot of attention but hey, at least it runs!

LCFG Profile – Secure mode

Posted May 19, 2016 by squinney
Categories: Uncategorized

Tags: ,

The LCFG client has a, slightly weird, feature called “secure mode“. This makes the client hold off applying any resource changes until they have been manually reviewed. The manual checking is done by examining the contents of a “hold file” which shows the differences in values for each modified resource in a simple text form. The file also contains a “signature” which is the MD5 digest (in hex) of the changes. A change set is applied manually by passing that signature to the client which then regenerates the hold file and compares that signature with the one supplied. This is not a heavily used feature of the client but it is something we want to support in the new LCFG profile framework. The new framework has built-in support for diffing the data structures which represent LCFG profiles, components and resources. This makes it relatively straightforward to add a feature which generates the secure-mode hold file when required, the only awkward part was finding some code to do the MD5 digest in a nice way.

Here’s an example using the C API, error checking and suchlike has been dropped to keep it simple.


#include <lcfg/profile.h>
#include <lcfg/bdb.h>
#include <lcfg/differences.h>

int main(void) {

char * msg = NULL;

LCFGProfile * p1 = NULL;
lcfgprofile_from_status_dir( “/run/lcfg/status”,
&p1, NULL, &msg );

LCFGProfile * p2 = NULL;
lcfgprofile_from_bdb( “/var/lcfg/conf/profile/dbm/example.lcfg.org.DB2.db”,
&p2, NULL, 0, &msg );

LCFGDiffProfile * diff = NULL;
lcfgprofile_diff( p1, p2, &diff, &msg );

char * signature = NULL;
lcfgdiffprofile_to_holdfile( diff, “/tmp/holdfile”, &signature, &msg );

lcfgprofile_destroy(p1);
lcfgprofile_destroy(p2);
lcfgdiffprofile_destroy(diff);

free(msg);

return 0;
}

LCFG profile querying

Posted May 13, 2016 by squinney
Categories: Uncategorized

Tags: ,

The new LCFG profile framework makes it simple to retrieve component and resource information from profiles stored in the various standard formats (XML, Berkeley DB and status files).

Loading a profile from XML, DB or status directory:


my $p = LCFG::Profile->new_from_xml(“example.xml”);


my $p = LCFG::Profile->new_from_bdb(“example.db”);


my $p = LCFG::Profile->new_from_status_dir(“/run/lcfg/status”);

Loading a component from a DB or status file:


my $c = LCFG::Profile::Component->new_from_bdb( “example.bdb”, “client” );


my $c = LCFG::Profile::Component->new_from_statusfile( “/run/lcfg/status/client” );

Retrieving a component (e.g. client) from the profile:


my $c = $p->find_component(“client”);

Retrieving a resource (e.g. client.components) from a component:


my $r = $c->find_resource(“components”);

Getting the resource value:


say $r->value;

For convenience, if the resource is a tag list then you can get the value as a perl list:


@comps = $r->value;
for my $comp (@comps) {

}

LCFG profile handling

Posted May 13, 2016 by squinney
Categories: Uncategorized

Tags: ,

Over the last few months the new libraries for handling LCFG profiles have been shaping up nicely. They are finally reaching a point where they match up with my original aims so I thought I’d give a taste of how it all works. Here’s an example of processing an LCFG XML profile into the Berkeley DB and rpmcfg files required by the client:

use LCFG::Profile;

my $xml    = '/var/lcfg/conf/profile/xml/example.lcfg.org.xml';
my $dbm    = '/tmp/example.lcfg.org.DB2.db';
my $dbm_ns = 'example';
my $rpmcfg = '/tmp/example.lcfg.org.rpmcfg';

my $new_profile = LCFG::Profile->new_from_xml($xml);

my $update_dbm = 0;
if ( -f $dbm ) {
    my $cur_profile = LCFG::Profile->new_from_bdb($dbm);

    my $diff = $cur_profile->diff($new_profile);

    if ( $diff->size > 0 ) {
        $update_dbm = 1;
    }
} else {
    $update_dbm = 1;
}

if ( $update_dbm ) {
    $new_profile->to_bdb( $dbm, $dbm_ns );
    say 'Updated DBM';
}

my $pkgs_changed = $new_profile->to_rpmcfg($rpmcfg);
if ( $pkgs_changed ) {
    say 'Updated packages';
}

This is basically what the LCFG client does whenever it processes a new profile but is a lot nicer than the current rdxprof code!

Platform-specific config

Posted January 13, 2016 by squinney
Categories: Uncategorized

I recently came across this blog article titled Stop writing code that will break on Python 4!. Although the title mentions python 4 it is really discussing “any future major version“.

This is something we have learnt to deal with in LCFG over the years. We often have to tweak configuration slightly when developing support for new platforms and this results in lots of if/elseif/else statements based on the target platform. Once you’ve been through the platform upgrade cycle a few times you learn that the most efficient approach is to special-case the old platform and make the new platform the default. By assuming that the configuration required for the new platform will be the default going forwards (i.e. it sits in the “else” branch) you make the configuration for N+1 and also handle N+M at the same time.

Writing modern C

Posted January 10, 2016 by squinney
Categories: Uncategorized

For the v4 LCFG client project I’ve been writing lots of C. To get my C knowledge up to scratch I’ve been consulting many books of varying vintages which leads to quite a mixture of coding styles. I’m quite keen to create code which is concerned “good” according to modern coding standards but I also want to ensure it will compile on a wide range of Unix-like systems, that seems to mean right now that the standard to aim for is C99. I recently came across an interesting article titled "How to C (as of 2016)" which gives a good summary of many important topics to consider. I’ve already been following many of the suggestions but there are also quite a few which are totally new to me. I’m not sure I agree with all of them (e.g. not using char) but I shall definitely be applying some of them.

Security awareness

Posted November 20, 2015 by squinney
Categories: Uncategorized

I recently came across a series of short blog posts from the SANS Securing the Human site on the topic of Security Awareness. I found them to be quite interesting and thought provoking. If you’re interested in what can be done to improve the security of an organization I’d recommend these as a good starting point:

  1. The 4 Ws to Awareness Success
  2. The Why in Effective Awareness Programs
  3. The Who in Effective Awareness Training
  4. The What in Effective Awareness Training
  5. The How in Effective Awareness Training

LCFG XML Profile changes

Posted August 20, 2015 by squinney
Categories: Uncategorized

Tags:

As part of the LCFG v4 client project I am working on converting the XML profile parsing over to using the libxml2 library. Recent testing has revealed a number of shortcomings in the way the LCFG XML profiles are generated which break parsers which are stricter than the old W3C code upon which the current client is based. In particular the encoding of entities has always been done in a style which is more suitable for HTML than XML. There is really only a small set of characters that must be encoded for XML, those are: single-quote, double-quote, left-angle-bracket, right-angle-bracket and ampersand (in some contexts the set can be even smaller). The new XML parser was barfing on unknown named entities which would be supported by a typical web browser. It is possible to educate an XML parser about these entities but it’s not really necessary. A better solution is to emit XML which is utf-8 compliant which avoids the needs for additional encoding. Alongside this problem of encoding more than was necessary the server was not encoding significant whitespace, e.g. newlines, carriage returns and tabs. By default a standards compliant XML parser will ignore such whitespace. An LCFG resource might well contain such whitespace so it was necessary to add encoding support to the server. In the process of making these changes to the LCFG::Server::Profile::XML module I merged all the calls to the encoder into a call to a single new EncodeData subroutine so that it is now trivial to tweak the encoding as required. These changes will be going out in version 3.3.0 of the LCFG-Compiler package in the next stable release. As always, please let us know if these changes break anything.

MooX::HandlesVia and roles

Posted August 20, 2015 by squinney
Categories: Uncategorized

Tags:

I’ve been using Moo for Perl object-oriented programming for a while now. It’s really quite nice, it certainly does everything I need and it’s much lighter than Moose.

Whilst working on the LCFG v4 client project I recently came across a problem with the MooX::HandlesVia module when used in conjunction with roles. I thought it worth blogging about if only to save some other pour soul from a lot of head scratching (probably me in 6 months time).

If a class is composed of more than one role and each role uses the MooX::HandlesVia module, for example:

{
    package SJQ::Role::Foo;
    use Moo::Role;
    use MooX::HandlesVia;
}

{
    package SJQ::Role::Bar;
    use Moo::Role;
    use MooX::HandlesVia;
}

{
    package SJQ::Baz;
    use Moo;

    with 'SJQ::Role::Foo','SJQ::Role::Bar';

    use namespace::clean;
}

my $test = SJQ::Baz->new();

It fails and the following error message is generated:

Due to a method name conflict between roles 'SJQ::Role::Bar and
SJQ::Role::Foo', the method 'has' must be implemented by 'SJQ::Baz'
at /usr/share/perl5/vendor_perl/Role/Tiny.pm line 215.

It appears that MooX::HandlesVia provides its own replacement has method and this causes a problem when namespace::clean is also used.

The solution is to apply the roles separately, it’s perfectly allowable to call the with method several times. For example:

{
    package SJQ::Baz;
    use Moo;

    with 'SJQ::Role::Foo';
    with 'SJQ::Role::Bar';

    use namespace::clean;
}

PostgreSQL new features

Posted June 10, 2015 by squinney
Categories: Uncategorized

It looks like PostgreSQL 9.4 has some really interesting new features. Today I came across a blog post by 2ndquadrant demonstrating the WITHIN GROUP and FILTER clauses. I don’t think I’ve entirely got my head round the purpose of WITHIN GROUP yet, I suspect I need a couple of good real-world examples. The FILTER clause looks very handy though, I’m sure I’ll be using that when I get the chance.