Febuntoo News Generator

Some of you might already know that I am moving to Manchester/UK this year ( at least ) to attend a MSc course at the University of Manchester. The Computer Science Department offers a variety of courses hance I am not sure which pathway I am gonna choose yet. The Computer System Engineering MSc looks rather good but I will have to wait for the 1st week of lectures before I make my choice.  In any case, moving to UK is quite an experience, and I am really excited about that :)

Even though the motivation about Gentoo development is anything but high, I was quite worried about the fact that I couldn’t bring my two “dev” machines[1][2] with me so all I have is my laptop and my netbook, but lets face it, you can’t do much development on such hardware :-/.  I could easily forward a dozen of ports on my router in order to remotely administer all my services but, when it comes to security, this is a serious drawback. This really left me with one choice: Openvpn :).  Thanks to this guide, the whole setup was easier than I originally thought. Now, I am able to access my local network from anywhere. Furthermore, tmux[3] makes the whole /remote/development process much easier :). What a great app \o/

[1] Linux Phoenix 2.6.34-gentoo-r6-darth-vader

#1 PREEMPT Wed Aug 25 02:09:42 EEST 2010 x86_64

AMD Athlon(tm) 64 Processor 2800+ AuthenticAMD GNU/Linux

Just for stable amd64 keywording and Server services

[2] Linux Mystical 2.6.35-zen2-dark-jedi #1

ZEN SMP PREEMPT Thu Sep 2 21:19:07 EEST 2010 x86_64

AMD Phenom(tm) II X6 1055T Processor AuthenticAMD GNU/Lin

This is actually a *good* dev machine. The 6-core power combined with 4GB@1066 RAM

is an ideal platform for extensive development and compilation 24/7



* dev-db/mysql-5.1.50-r1: Merge time 2 hours, 7 minutes, 45 seconds ( tests enabled )

* sys-devel/gcc-4.4.4-r1: Merge time 15 minutes, 36 seconds

* sys-libs/glibc-2.12.1-r1: Merge time 11 minutes, 5 seconds

This devbox hosts 6 chroors to test all the Qt4 live versions and one  1 gcc-4.5.X chroot

[3]

A foreword: some people might think that I’m writing this just to banter about what I did; my sincere reason to write, though, is to point out an example of why I dislike 5-minutes fixes as I wrote last December. It’s also an almost complete analysis of my process of ebuild maintenance so it might be interesting for others to read.

For a series of reasons that I haven’t really written about at all, I need Quagga in my homemade Celeron router running Gentoo — for those who don’t know, Quagga is a fork of an older project called Zebra, and provides a few daemons for route advertisement protocols (such as RIP and BGP). Before yesterday, the last version of Quagga in Portage was 0.99.15 (and the stable is an old 0.98 still), but there was recently a security bug that required a bump to 0.99.17.

I was already planning on getting Quagga a bump to fix a couple of personal pet peeves with it on the router; since Alin doesn’t have much time, and also doesn’t use Quagga himself, I’ve added myself to the package’s metadata; and started polishing the ebuild and its support files. The alternative would have been for someone to just pick up the 0.99.15 ebuild, update the patch references, and push it out with the 0.99.17 version, which would have categorized for a 5-minutes-fix and wouldn’t have solved a few more problems the ebuild had.

Now, the ebuild (and especially the init scripts) make a point that they were contributed by someone working for a company that used Quagga; this is a good start, from one point: the code is supposed to work since it was used; on the other hand companies don’t usually care for the Gentoo practices and policies, and tend to write ebuilds that could be polished a bit further to actually be compliant to our guidelines. I like them as a starting point, and I got used to do the final touches in those cases. So if you have some ebuilds that you use internally and don’t want to spend time maintaining it forever, you can also hire me to clean them up and merge in tree.

So I started from the patches; the ebuild applied patches from a tarball, three unconditionally and two based on USE flags; both of those had URLs tied to them that pointed out that they were unofficial feature patches (a lot of networking software tend to have similar patches). I set out to check the patches; one was changing the detection of PCRE; one was obviously a fix for --as-needed, one was a fix for an upstream bug. All five of them were on a separate patchset tarball that had to be fetched from the mirrors. I decided to change the situation.

First of all, I checked the PCRE patch; actually the whole PCRE logic, inside configure is long winded and difficult to grok properly; on the other hand, a few comments and the code itself shows that the libpcreposix library is only needed non non-GNU systems, as GLIBC provides the regcomp/regexec functions. So instead of applying the patch and have a pcre USE flag, I changed to link the use or not of PCRE depending on the elibc_glibc implicit USE flag; one less patch to apply.

Second patch I looked at was the --as-needed-related patch that changed the order of libraries link so that the linker wouldn’t drop them out; it wasn’t actually as complete as I would have made. Since libtool handles transitive dependencies fine, if the libcap library is used in the convenience library, it only has to be listed there, not also in the final installed library. Also, I like to take a chance to remove unused definitions in the Makefile while I’m there. So I reworked the patch on top of the current master branch in their GIT, and sent it upstream hoping to get it merged before next release.

The third patch is a fix for an upstream bug that hasn’t been merged in a few releases already, so I kept it basically the same. The two feature patches had new versions released, and the Gentoo version seems to have gone out of sync with the upstream ones a bit; for the sake of reducing Gentoo-specific files and process, I decided to move to use the feature patches that the original authors release; since they are only needed when their USE flags are enabled, they are fetched from the original websites conditionally. The remaining patches are too small to be part of a patchset tarball, so I first simply put them in files/ are they were, with mine a straight export from GIT. Thinking about it a bit more, I decided today to combine them in a single file, and just properly handle them on Gentoo GIT (I started writing a post detailing how I manage GIT-based patches).

Patches done, the next step is clearing out the configuration of the program itself; the ipv6 USE flag handles the build and installation of a few extra specific daemons for for the IPv6 protocol; the rest are more or less direct mappings from the remaining flags. For some reason, the ebuild used --libdir to change the installation directory of the libraries, and then later installed an env.d file to set the linker search path; which is generally a bad idea — I guess the intention was just to follow that advice, and not push non-generic libraries into the base directory, but doing it that way is mostly pointless. Note to self: write about how to properly handle internal libraries. My first choice was to see if libtool set rpath properly, and in that case leave it to the loader to deal with it. Unfortunately it seems like there is something bad in libtool, and while rpath worked on my workstation, it didn’t work on the cross-build root for the router though; I’m afraid it’s related to the lib vs lib64 paths, sigh. So after testing it out on the production router, I ended up revbumping the ebuild already to unhack it — if libtool can handle it properly, I’ll get that fixed upstream so that the library is always installed, by default, as a package-internal library, in the mean time it gets installed vanilla as upstream wrote it. It makes even more sense given that there are headers installed that suggest the library is not an internal library after all.

In general, I found the build system of quagga really messed up and in need of an update; since I know how many projects are sloppy about build systems, I’d probably take a look. But sincerely, before that I have to finish what I started with util-linux!

While I was at it, I fixed the installation to use the more common emake DESTDIR= rather than the older einstall (which means that it now installs in parallel as well); and installed the sample files among the documentation rather than in /etc (reasoning: I don’t want to backup sample files, nor I want to copy them to the router, and it’s easier to move them away directly). I forgot the first time around to remove the .la files, but I did so afterwards.

What remains is the most important stuff actually; the init scripts! Following my own suggestions the scripts had to be mostly rewritten from scratch; this actually was also needed because the previous scripts had a non-Gentoo copyright owner and I wanted to avoid that. Also, there were something like five almost identical init scripts in the package, where almost is due to the name of the service itself; this means also that there had to be more than one file without any real reason. My solution is to have a single file for all of them, and symlink the remaining ones to that one; the SVCNAME variable is going to define the name of the binary to start up. The one script that differs from the other, zebra (it has some extra code to flush the routes) I also rewrote to minimise the differences between the two (this is good for compression, if not for deduplication). The new scripts also take care of creating the /var/run directory if it doesn’t exist already, which solves a lot of trouble.

Now, as I said I committed the first version trying it locally, and then revbumped it last night after trying it on production; I reworked that a bit harder; beside the change in libraries install, I decided to add a readline USE flag rather than force the readline dependency (there really isn’t much readline-capable on my router, since it’s barely supposed to have me connected), this also shown me that the PAM dependency was strictly related to the vtysh optional component; and while I looked at PAM, (Updated) I actually broke it (and fixed it back in r2); the code is calling pam_start() with a capital-case “Quagga” string; but Linux-PAM puts it in all lower case… I didn’t know that, and I was actually quite sure that it was case sensitive. Turns out that OpenPAM is case-sensitive, Linux-PAM is not; that explains why it works with one but not the other. I guess the next step in my list of things to do is check out if it might be broken with Turkish locale. (End of update)

Another thing that I noticed there is that by default Quagga has been building itself as a Position Independent Executable (PIE); as I have written before using PIE on a standard kernel, without strong ASLR, has very few advantages, and enough disadvantages that I don’t really like to have it around; so for now it’s simply disabled; since we do support proper flags passing, if you’re building a PIE-complete system you’re free to; and if you’re building an embedded-enough system, you have nothing else to do.

The result is a pretty slick ebuild, at least in my opinion, less files installed, smaller, Gentoo-copyrighted (I rewrote the scripts practically entirely). It handles the security issue but also another bunch of “minor” issues, it is closer to upstream and it has a maintainer that’s going to make sure that the future releases will have an even slicker build system. It’s nothing exceptional, mind you, but it’s what it is to fix an ebuild properly after a few years spent with bump-renames. See?

Afterword: a few people, seemingly stirred up by a certain other developer, seems to have started complaining that I “write too much”, or pretend that I actually have an uptake about writing here. The main uptake I have is not having to repeat myself over and over to different people. Writing posts cost me time, and keeping the blog running, reachable and so on so forth takes me time and money, and running the tinderbox costs me money. Am I complaining? Not so much; Flattr is helping, but trust me that it doesn’t even cover the costs of the hosting, up to now. I’m just not really keen on the slandering because I write out explanation of what I do and why. So from now on, you bother me? Your comments will be deleted. Full stop.

A Chromium security stabilisation requires Cups 1.4 stable, which gives architecture teams some headaches. It is a major update and several stuff can break, tinderboxing it by building packages that depend on it is one side of the task, but maybe something breaks on runtime. On my own systems CUPS 1.4 runs fine for some months in mostly stable environments, but maybe there are more issues out there that we need to be aware of.  Please point out any issues that might be there on stable systems (maybe upgrade your stable system right now and tell us).

Hi folks,

last week I have been attending to the LinuxCon Brazil 2010, that was the first edition of this event here in Brazil.

All the talks were pretty good and the event was a great oportunity to meet nice people, good projects and learn some cool stuff (and get an autograph of Linus Torvalds as well :)

Unfortunately the Gentoo presence in the event was null. Well, not really null because I was there, but it seems that the Brazilian users are not very interested in Gentoo and I'm not sure if someone will be able to change this some day.

Maybe I'll write more stuff about the LinuxCon soon.

Stay tuned! ;)

I’ll be honest, the Reader felt like another cool gadget at first, but right now it really shows how easy it is to bring with me a number of books, without needing bookmarks, and without the physical constrain of book size — might sound very puny, but think about going around with a couple of volumes of Robert Jordan’s Wheel of Time and thank call me back.

Now this added with the ability to buy just-released books that aren’t available in Europe just yet, and the fact that I don’t pay any shipping on them, also meant I started getting hooked more and more to some book series, more than I would have before. Unfortunately, book series are expensive; not excessively expensive by themselves, but when you read through an instalment in less than a week, at the end of the month you’ll feel again the weight of the books on your credit card statement.

So I’m now thinking simply of dividing books in “fast readers” and “slow readers”, and get myself a rule so that I don’t spend a lot of money in fast readers one after the other. The obvious separation between the two would be “stuff I enjoy enthusiastically” and “stuff I really read just for the kicks”… but given I’m no longer in school, I have near to nothing within that set — there has been a few technical books I endured more than enjoyed, before, but that’s part of my job anyway.

It’s not a matter of taste, it’s even less a matter of page count; some books might be short, like Hawthorne’s The Scarlet Letter, but while they are interesting, they take me the longest time to read through them. Others might look “fatter”, like the Dresden Files series, but they capture me to a point I have difficulty turning away from the book.

But the first rule I have to get myself to respect is “Finish the dead-tree versions first!” — No point in getting new stuff as long as I haven’t finished all the technical books I have here. Well, it’s mostly alright for me given that I try to stay away from technical books during the summer so I can relax; not that it worked out pretty well this summer (no vacation, no break), but what I read was nothing I had before, nor something I had in dead tree form so it didn’t count.

So anyway, I’ll be back for a while to read stuff that makes a difference for what I do daily (packaging and coding), in dead-tree form for the most part. And a note here is worth it. I’m not sure whether the problem is with DocBook itself, but as good as O’Reilly’s content is, the ePub versions of their books tend to be .. difficult to read on the Reader itself; I’ll have to see if converting them with Calibre makes a difference, that would be interesting… but as it is I feel like reading them in PDF form would be probably easier; I just wish there will be some iPad-like tablet running Android sometime soon, that would definitely help — even more because CJKV Information Processing is exclusively available in PDF form, d’oh!