Planet Conary

Erik Troan and I co-authored two editions of Linux Application Development, so I have some idea of what it takes to write a Linux programming book. It's a lot of work to do well, doing proper research, trying to keep up to date, trying to meet the needs of a wide variety of readers.

Over the past several years, I have had a sense that the Linux man pages were becoming more complete, with a growing body of precise and nuanced detail. At some point, I became aware that it was Michael Kerrisk's fine work that I was appreciating, as he built substantially on the fine base provided by the previous maintainers Rik Faith and Andries Brouwer.

Michael Kerrisk then picked up the task of writing a book to cover similar material as Linux Application Development, but in both greater detail and extended scope. His new work, The Linux Programming Interface, has after long labor been published, and it looks like it was worth the wait. After reading a few initial sample chapters (the preface, and chapters 2, 9, and 30), I can say with great relief that it appears that Erik and I do not need to prepare a third edition of Linux Application Development — we can name TLPI as LAD's successor. Erik and I wrote LAD because it was the book we needed as a reference that was not otherwise available. That reason appears to be dissipating.

I'm looking forward to reading the whole book after it arrives, and reporting my impressions in more detail later!

This post has been sitting in my Drafts folder for a while now, as I wasn’t sure when the right time would be to publish it. It is basically my personal reflection on the last 5 years I have worked doing translations for free and open source software (FOSS) and a few lessons I learned along the way. It is also a rant against those who took my labor for granted. Hopefully this quick summary will help you decide whether you want to continue reading the rest of the post or not.

My adventures in the translation (or localization) world started some time in the middle of 2005. I had just started using Ubuntu as my main distribution and being carried away by the buzz and excitement surrounding this new comer, I started looking for ways to “give back”. Not that I hadn’t tried it before, but to tell you the truth, Ubuntu had back then the only friendly and welcoming community out there that wouldn’t treat you with scorn and arrogance if you were a new user.

Eventually I got to learn about the Ubuntu Brazil team and their effort to translate the desktop applications into the Brazilian Portuguese language. Now, I’ve been living in the United Stated for about 2 decades now and technological terms and jargon in Portuguese had never really entered into my vocabulary. In other words, I had absolutely no idea how to say things like “hard drive“, “File“, “Copy, or anything really in Portuguese. But I was determined to lend a helping hand and proceeded to learn on my own and by asking around.

Now, this is not a post about Ubuntu, so I’ll fast forward a bit to the time I became the coordinator for the Ubuntu Brazilian Translation team and was “in charge” of getting this massive collection of applications translated into Brazilian Portuguese before every single release. It was hard work but at the end of the day it felt great to know that more Brazilians would be able to enjoy GNU/Linux in their native tongue. Heck, under my leadership we delivered several releases of Ubuntu with very high levels of translations and absolutely no thank you from those profiting from out work!

After a couple of years I started to feel pretty good about my background in the software translations world. Filled with the best of intentions and the whole “Ubuntu” philosophy, I approached a few upstream translation teams to offer some help and see if our team could lend them a hand. Oddly enough, most refused my offer which only made me more confused. If they were struggling to get to 100% translations, why would they refuse help from “seasoned translators” such ourselves?

After organizing a few IRC meetings where many didn’t show up, some of the GNOME Brazil guys decided to give me a chance and hear my proposal. In the end we settled on the following plan: We, the “Ubuntu guys” would provide the labor (our team was much bigger than all other teams put together), and the GNOMErs would tell us what to do, how to do it, and tell us to “do it over” if our work did not conform with their standards. If this sounds a bit one sided to you, it’s because it was. There was some unexplained hostility towards us that we could not understand. Some of the members of my team eventually excused themselves, telling me in private how frustrating the whole experience was. “We’re breaking our backs here and they treat us like garbage,” said a disgruntled translator. This was in 2006.

This same type of treatment followed us wherever we went. Always the unprovoked hostility and unwillingness to work together. It wasn’t until around the middle of 2007 that someone finally broke the silence and told me: “You guys got the heart in the right place, but did you know that nothing that you have ever translated for Ubuntu comes back our way?” I didn’t know what to say. “What do you mean?” We’re good guys… the whole World benefits from our work… doesn’t it?”

Turns out I had been mistaken about the benefits of doing translations for a distribution. Turns out also that a lot of upstream translators who are not Ubuntu users (and therefore do not have an account in Launchpad) also have strong feelings against the work being done by the “Ubuntu guys”. But instead of channeling their ill feelings toward the entity that designed the machinery, their anger fell on the translators, peons in the whole scheme of things.

I’m glad to say that I woke up from the stupor that had held me for quite some time in a state of delirium and even fanboyism. Yes, I was an Ubuntu fanboy not too long ago! I too drank from the koolaid and was a major source of free PR and goodwill for Canonical. It was early 2007 when I finally moved on and decided to work directly with the upstream projects, embracing a new world full of opportunities and chances for someone as dedicated as myself.

Boy, was I wrong about certain things! For a long time the stigma of having done translations for a distribution stuck to me, causing many unnecessary discussions and personal attacks from people who I had never met or heard before. Worse of all, the internal interests and political agendas within the smaller groups was very harsh and no matter how many packages I translated or how many hours I spent organizing, teaching, reviewing and translating massive documents, I was never given the acknowledgement by my peers or anyone for that matter. Did you know that GNOME 2.26 was literally translated by two people into Brazilian Portuguese? And that one of them was yours truly?

But I kept going for a long time, choosing to ignore all of these things and focus in the main cause: deliver a completely localized operating system for those who speak Brazilian Portuguese. Through the years I saw coordinators and committers being replaced by people with no background what so ever in translations or even from outside the team! There has always been intrigue and malicious interests in pretty much every community out there, but I had always hoped that one day my hard work would be recognized and aspired to one day become the coordinator for the GNOME Brazilian team. Alas, that day never came.

But I kept going and joined several other teams who received me with open arms. For quite some time I was the sole translator, reviewer and committer for Xfce, LXDE, and Openbox. I even joined the effort to localize MeeGo and make it upstream friendly. Whatever free time I managed to get my hands on was spent either translating, reviewing or committing translations to these projects (and GNOME, off course). Even while juggling a brand new career with a steep learning curve, a second child, moving to a different state and seeing my bank account dwindle down, I still made the time to keep going.

Until I ran out of gas! This morning I have chosen to step down as the coordinator for the LXDE and MeeGo teams and have already passed this position to one of my teammates. I am also stepping down as the administrator of these 2 teams in order to focus on my family and some of my pet projects. I will eventually pass the coordination of the Xfce team as well and will only act as a member of said teams, offering a helping hand every now and then (this already applies to GNOME).

This does not mean that I’m no longer involved with the localization of free and open source software. It only means that I’m now sitting on the backseat and am happy to let someone else drive. It also means that I feel unappreciated and even though the thought that every single GNOME user who runs his/her system in Brazilian Portuguese is doing it because of the fruit of my labor and others makes me feel very proud, the truth is that it does not put food on my table.

I have to finish this post by publicly thanking Margie Foster from the MeeGo project for being the only person who has shown appreciation and gratitude for the work I’ve done! Really, thank you!

For people like us
In places like this
We need all the hope we can get

While U2 and REM were selling out arenas during the 80s, there was this little scrappy band called The Call, fronted by one Michael Been, that turned some impressive arena-sized albums about love, hope, anger, and faith.

“I Still Believe” remains one of my favorite songs of theirs:

It’s sad that most people know this song as played by Tim Capello, playing himself as the oily, muscle-bound sax player that covered it during one of the early scenes in The Lost Boys.

Michael Been, RIP (1950-2010)

I had a page open to check the progress of an Amtrak train, so that I could know when to pick up a passenger. Not a lot of info was available. As of 8:30PM, it just said "on time" for 9:00PM (though the timetable said 9:13PM instead of 9:00PM, just to keep things interesting).

I arrived at the station at 9:00PM to find out that the train had arrived at 8:30PM. When I returned home, I refreshed the page from the Amtrak site, curious what it would say. There were two surprises. The first was that it said that the train had actually arrived at 8:40PM. The second was how late the site said this 20-minutes-early train was.

For a while now I have wanted to make it easier for desktop applications to integrate more with social networks, enabling users to easily interact with their friends in many different contexts.  Could be posting a status update based on the context they are in right now, commenting on a friends photo while browsing their Facebook photo album in Shotwell, etc.  Let me introduce libgwibber!

A brief introduction to libgwibber, a library (C, vala, mono, and python) for using the Gwibber Service as well as provide some GTK widgets to easily embed into existing GTK applications.  Bindings for as many languages as possible was very important to me, I really want to make it as easy as possible for any desktop application to use Gwibber as a desktop service.

The API currently provides access to the common things an application developer might care about:

• signals for service availability and account changes
• refresh
• start and stop gwibber-service
• retrieving accounts
• toggling send_enabled status on an account
• looking up an account
• posting a status update
• URL shortening
• retrieving the version of the current running gwibber-service

I just finished porting the MeMenu from the Ayatana project to use libgwibber, which will be in the next release.

So far libgwibber-gtk includes just one widget, a posting entry.  This widget includes:

• A TextView
  • with an overlay character counter
  • built in URL shortening
  • integrated with NetworkManager, disabled when in offline mode
• AccountTargetBar
  • includes toggle buttons for each account, connected to signals to track the current status globally
• Send button

Some examples of the Gwibber entry widget can already be found embedded in the Ubuntu Software Center and the Ubuntu One Music Store plugin in rhythmbox.

Other widgets I hope to add in the future include:

• stream view (message view of any stream)
• stream navigation
• search
• friend browser (browse friends based on type of content, like friends with images that can be displayed in shotwell or f-spot)
• profile (view your own or someone else’s profile)
• comment (comment on various types of shared content, photos, links, etc)
• image uploader

I would love suggestions for these or others, please let me know!

Some GTK examples:

C example

entry-c.c
To build entry-c.c:

gcc `pkg-config --cflags gwibber-gtk` entry-c.c `pkg-config --libs gwibber-gtk` -o entry-c

Vala example

entry-vala.vala
To build entry-vala.vala:

valac --pkg gtk+-2.0 --pkg gwibber --pkg gwibber-gtk entry-vala.vala

Mono example

entry-mono.cs
To build entry-mono.cs

gmcs -target:exe -out:entry.exe -pkg:gwibber-gtk-sharp-0.0 entry-mono.cs

Python example

entry-python.py
Nothing to build, so just run it with:

python entry-python.py

Here are some examples of using libgwibber with python and gobject introspection:

# Import Gwibber using introspection
from gi.repository import Gwibber

# Get a service object
service = Gwibber.Service()

# Get the current Gwibber version
version = service.version()

# Make the Gwibber service refresh
service.refresh()

# Post a status update to all your enabled accounts
service.send_message("Something very interesting here, blah, blah, blah")

Some applications I would love to see use libgwibber:

Shotwell and F-Spot: Browse online photos from your social networks from right inside you existing photo library tool.  Including the ability to comment, tag, like and share.

gnome-utils: Add image upload support to gnome-screenshot

Evolution: Display the last status update from a contact when viewing an email or browsing contacts.

GTG: Status updates when tasks are complete

I am generating gtk-doc docs for libgwibber, but they aren’t very useful yet.  libgwibber is written in vala and I haven’t figured out a way to get docstrings to pass from vala to the generated C.  If anyone knows how to do that, please let me know.

Hopefully people find libgwibber useful, please let me know if you have suggestions, bug reports, or want to contribute!

Received the following email this morning:

Last night our Transifex installation turned 1 year old. I think with
101 users and  4012 submissions in 45 languages it has been quite a
step forward compared to the previous mailing list-based submission
system. For that I'd like to thank all the people involved, most
noticeably Jannis for the initial setup, the Transifex guys for help
and fixing bugs, Og for the appliance we've been running for the last
half year and of course the Translators with all their patience and
effort to make it a success.

Cheers,
The Xfce development team.

This has an extra special meaning for me, not only because I am one of the 101 users contributing translations to the Xfce project, or because I’m a huge supporter of the work the Transifex guys have been doing, but because of their use of the Transifex Appliance!

You see, a while back I needed a “pet” project to use as a learning tool to learn about creating, maintaining and deploying software appliances with the technologies we develop here at work. Transifex was the project that caught my attention, mainly for being a tool for translators and for using Django under the hood.

I’ve been maintaining the Transifex Appliance for quite some time now, all by my lonesome self and putting a lot of my free time. So it feels great to hear that the appliance is being used in a production environment and that it is used by several users!

So, congratulations to the Xfce translators and Nick Schermer for sticking with the appliance and providing tons of great feedback on how to improve it!

I got tired of it, that’s all.

rBuild 1.2.6 is a maintainence release

Bug Fixes:

• building packages now defaults to no-recurse. Recursion can still be enabled using the --recurse option. (RBLD-311)
• rbuild checkout --derive no longer throws an unhandled exception when called outside a product directory (RBLD-325)
• The rmake strictMode configuration is now set for builds (RBLD-328)

rMake 2.0.5 is a maintainence release

New Features:

• rMake is now compatible with versions of Conary that implement the proxyMap configuration item. (CNY-3427)

Bug Fixes:

• The copyInConfig configuration item now defaults to False (RMK-1052)
• Since Conary now de-duplicates proxy requests using file locking on individual changesets, rMake now defaults to increasing the open file limit for rMake itself. (RMK-1054)

Conary 2.1.22 is a maintainence release

Bug Fixes:

• A bug introduced in Conary 2.1.21 that prevented `cvc cook` from functioning has been corrected. (CNY-3490) (CNY-3490)
• Fix an inconsistency between Conary and RPM databases when RPM fails to install a requested package. (CNY-3488)
• Rewrite derived package shadow creation code because the previous attempt to support capsules was fragile and frankly a bad idea. (RBLD-326)
• Fix exception when doing a repository cook when the recipe sources a multiurl of mirrors. (CNY-3485)

Manifests are great for imaging systems, but what else can they do for you?

Sometimes you can provision a system, use it for a while, then discard it. That’s a nice gig if you can get it. But many applications need long-running persistent systems (whether physical, virtual, or cloud), and long-running systems need updates at every level, from OS patching to application configuration. And work that goes into updates is frequently repeated when updating the provisioning script.

Traditionally, IT change management focuses on, well, changes. That’s obvious enough! But model-driven automation offers a better way. Just as a building architect delivers a plan change by updating the blueprint—not by delivering a list of error-prone, inefficient step-by-step changes—platform and application engineers can more efficiently and accurately control change by updating the system model.

Here’s how you drive change, such as an OS update or a new application to deploy, in rPath.

For example, let’s consider an application engineer, Alice, who needs to upgrade JBoss in the standard blueprint from 4.1.2 to 4.1.3:

1. Alice selects the 4.1.3 version of JBoss from a drop-down menu in her standard blueprint—At the appropriate level of the blueprint hierarchy, you can focus on the model (not the change) and alter the model to reflect the new requirement. If Charlie the platform engineer needs to add a standard patch to the corporate-standard build of CentOS, he follows the same process as Alice but at a different level of the hierarchy.
2. rPath automatically generates a new manifest version—This ensures that all dependencies and IT policies are still met.
3. Diane, a system administrator, finds affected systems—Since the rPath inventory is versioned, it’s easy to see all systems that are on the previous version of the blueprint.
4. Diane schedules an update—She orders rPath to update one or more systems to match the new manifest in the next maintenance window.
5. rPath automates the update—rPath examines the current state of each system, determines what must change to match the new blueprint, and makes the appropriate incremental changes—creating, modifying, and deleting the files required to turn JBoss 4.1.2 into JBoss 4.1.3.

Why is this a better way?

• Perfect synchronization with provisioning—Traditionally, patching and provisioning are separate systems and are perpetually out of sync. With rPath, you use the same manifest for updating existing systems and deploying new systems.
• “Correct by construction”—rPath drives risk out of the update process by completely validating the “to be” model for a system before making any change to the “as is” reality.
• Engineering leverage—Consider an OS patch across 400 diverse systems. The patch may have different requirements, conflicts, and dependencies on different systems, so applying that patch manually is a massive effort. With rPath, you change one blueprint in the hierarchy, and everything else is automated. That represents massive leverage improvement for skilled IT labor.

The result is faster, less dangerous maintenance windows—and no more repeating yourself.

Next week we’ll look at application #3 for manifests: compliance.

I can’t claim to have looked hard for a Python book on object oriented programming (OOP) but I was immediately attracted to the title of this book. Sure, you can find small tutorials here and there about some specific facet of OOP but I don’t recall ever reading something that covers designing public interfaces using abstraction, encapsulation, etc, etc with good and practical examples! If you have, please drop me a link in the comments.

Python 3 Object Oriented Programming by Dusty Phillips does a very decent job of not only introducing the reader to the terminology and the object oriented paradigm (something that is not too complicated to understand) but also offers a comprehensive step-by-step guide that will take you from theory to a real world project. I’ve always felt that anyone can pick up a book about programming and learn its syntax by heart. But putting it all together and designing something that will actually work is something that you usually learn by reading other people’s code or, if you’re lucky enough, from a mentor.
Overall I felt that the book was well written with a great selection of sample code. Whether you already know how to do object oriented programming for other languages or are new to the whole concept and want to learn everything about it, I definitely recommend this book!

Make sure to check the free chapter (Chapter No.7: Python Object Oriented Shortcuts).

Conary 2.1.21 is a maintainence release

New Features:

• The --no-recurse option is now allowed with the "conary migrate" command. This can be used to rebuild a system from a manifest file.
• A no-scripts option has been added. If set, capsule scripts, trove scripts, and tag scripts are skipped during an update. (CNY-3452)
• Conary now records in /var/log/conary the order in which RPM chooses to install encapsulated RPMs. (CNY-3460)
• When RPM fails to unpack a package when being invoked as non-root, Conary will now warn about the failure but will not fail the entire transaction. (CNY-3462)
• Conary now validates RPM capsule operations against the RPM database as early as possible to flag errors when they occur. (CNY-3455)

Bug Fixes:

• Make negative openpgp key cache entries to avoid repeated server queries for keys which don't exist (CNY-3450) (CNY-3450)
• Process directory creation for other file types (CNY-3451)
• Fixed a repository crash when converting a cached changeset for an older client. (CNY-3453)
• The getCapsulesTroveList function was restored to sqldb for use in recreatedb.
• Fixed signing of packages with pycrypto 2.1 (CNY-3465)
• Fixed a bug which caused an internal server error when the same trove permission was applied to more than two roles. (CNY-3469)
• When multiple installed conary components shared an underlying RPM capsule removing one of those components would result in the conary database becoming out of sync with the rpm database (CNY-3470)
• Changesets can contain shared files where one is a diff and the other is not. The diff will be used (CNY-3483)

Internal Changes:

• Hotspot profiling hooks have been removed. (CNY-3250)
• added cksig script for verifying (and fixing) signatures in a repository
• Add argument to the API for cooking troves to allow someone to specify a changeset file instead of committing the changeset to the repository. (CNY-3466)
• The transport layer is now handling network errors more reliably. (CNY-3429)

In part because my children are perhaps not quite so careful with CDs as I am, I ripped all my CDs (at least, all that will still read!) to flac, ogg, and mp3 files. The second half of the project was to get a media player that could play all those files -- preferably over the network from my main server so that I don't have to copy files onto an SD card every time I get a new CD and rip it.

When the Archos 7 Home Tablet was announced, it seemed like the obvious thing to use. WiFi to access the server. Should play oggs just fine. I don't really care very much about whether it can access the Android market, though the restrictions on the market seem rather silly to me.

After I downloaded Subsonic and installed the server portion on my home server and the client on my tablet, I had a working solution for audio. While the default RPM packaging for Subsonic is sub-optimal (making important directories world readable/writable, for example) and this makes me convinced that I would not want to expose a Subsonic server implementation to the world, on my internal network I'm fine with it, and it gives me all the functionality I need to play my music collection on my home stereo. (The only problem is that for reasons I haven't researched, I have to use mp3s instead of oggs, even though I can play ogg files directly on the tablet.)

However, I'm not all that impressed with the Archos 7 Home Tablet. As reported by others, the native browser crashes almost instantly when you try to actually use it. It rarely survives a google search from the google search widget. I'm not impressed that Archos shipped something this broken, and less impressed that Archos has not yet released an updated firmware to resolve this issue, nor do they have updates available in their proprietary "applib" for the malfunctioning browser. Using Dolphin Browser or Opera seems to be the only way to use this tablet to browse the web.

I'm also not impressed that this late, they are still shipping an Android 1.5 OS.

Kindly, a user has stepped up to the plate and delivered a firmware update (shame on Archos for their lack of attention to this device!) that resolves at least some issues. So far, the browser hasn't crashed for me after applying the update. As a bonus, the user-supplied firmware enables the Android Market on the device, which made it a bit easier to get my apps back after wiping the system as part of applying the update.

It looks like there will soon be a whole crop of better tablet devices coming out with similar hardware but newer versions of Android. In retrospect, if I could make my decision again, I would wait and purchase a different tablet. Maybe at Kmart, of all places...

It’s been a while since I last posted anything new. That’s because I took some time off from work and headed down to my old stomping grounds… Brazil! It had been 5 years since I visited my friends and relatives and since I couldn’t make it to GUADEC and take my family on vacation at the same time, we hoped on a straight flight to Rio de Janeiro from Charlotte‘s international airport.

While waiting for my connection flight, I had the great pleasure of finally meeting in person my good friend KurtKraut, someone who I met online during my ‘buntu days and have kept in touch for the last 5 years. It was great to hang out with him and not have to chat via Skype or IRC! I definitely hope to one day host him here in North Carolina one of these days.

We spent some amazing 8 days (excluding days spent at airports and travelling by car to/from our final destination) enjoying some relatively mild days by the beach, even though it was Winter down there! Every day we’d be invited to eat lunch or dinner at someone’s place and at any given moment my dad’s house where we were staying, would be swarmed by an army of cousins, uncles, aunts and friends who wanted to say hi and meet my kids!

When we weren’t busy chatting, we were busy eating and drinking! I went overboard this time and eat a lot of everything that got in front of me! One funny thing about going back to the place where you grew up after so long is that you miss silly, little stupid things… such as the bubble gum you used to buy when you’re little… the candy, ice cream, bread, even soda that everyone who’s still living there takes for granted. I must have gained quite a few pounds with all the food, beer and caipirinhas consumed in this trip, but what the heck! I don’t get to visit Brazil very often, so it is worth it.

Now, next year will be the 20th anniversary that my parents came to the USA with 3 teenagers who couldn’t speak a single sentence in English, so my family is planning a big family reunion. Hopefully I will be able to sneak out a bit and get to meet some other friends from the South of Brazil… through the years I’ve received many invitations for dinners and BBQ Brazilian style! And who am I to disappoint my friends?

Last week I dug into two key rPath concepts: the blueprint and the manifest. Manifests drive many automated tasks in rPath, starting with image generation.

Traditionally, deployment poses a difficult choice. You can deploy each system by installing its components one by one. That’s completely flexible, but very slow and error prone. Or, you can deploy each system from a physical, virtual, or cloud “golden image.” That’s fast and reliable, but doesn’t solve the fundamental problem—it really just begs the question. Instead of manually maintaining lots of individual systems, you are manually maintaining lots of golden images.

rPath solves the dilemma by automatically generating images from manifests. Given a manifest, rPath can automatically generate:

• Installable ISO and WIM images for physical provisioning
• Virtual machines for any hypervisor, including VMware, Xen, and Hyper-V
• Cloud images for any cloud, including Amazon EC2 and Eucalyptus

Once provisioning is complete, you can simply discard the image. Since rPath generated the image from completely versioned inputs, you can regenerate the same image at any time. We believe large artifacts like images should always be ephemeral and recreatable—object code, not source code.

This technique gives the best of both worlds. The blueprint yields complete flexibility and customization, while the generated image enables rapid provisioning. Provisioning becomes faster and more reliable while policy setters and platform engineers retain flexible control over system contents.

Next week we’ll talk about how the same manifest drives incremental system updates.

On top of raw system artifacts (such as packages and configuration files), rPath layers several unique system modeling concepts. Here’s how two of the most important concepts at rPath—blueprints and manifests—work together to deliver model-driven automation.

Blueprints

An rPath blueprint is a comprehensive model for one or more systems. It describes all components to include on a system: OS platform, OS patches, middleware stacks, applications, compliance policies, and configurations. The description is declarative and human-understandable; you can create and edit blueprints with an easy-to-use GUI or with a powerful command line interface.

Blueprints are hierarchical. For example, instead of specifying corporate-standard Linux security requirements in the web server blueprint, you can create a corporate standard Linux blueprint, and use that as the basis for the web server blueprint. The web server blueprint becomes simple and easy to understand, since it needs to describe only the specialized web server requirements—such as configuring Apache—and inherits the definition of the corporate standard blueprint. That eliminates redundant effort in patching and software deployment, and reduces the risk of accidental change.

Blueprints are version-controlled. Each change to a blueprint is recorded as an immutable version, so you can always find the exact specification that created a particular system. That leads to:

• Faster troubleshooting—As demonstrated in The Visible Ops Handbook, 80% of outages stem from deliberate configuration changes. Version control makes it easy to see all recent changes that affected a system, eliminating hours of detective work from the troubleshooting process.
• Reproducibility—Complete versioning makes it possible to rebuild a system from any of its versions, enabling disaster recovery and capacity expansion.
• Auditability—It’s easy to obtain any previous version of a complete system model, making it possible to answer detailed questions about the prior state of a system.

Manifests

How do blueprints drive automation? The first step is the automated generation of a manifest. A manifest is a granular, flat bill of materials for a system. For example, if the blueprint called for Red Hat Enterprise Linux 5, the manifest lists the hundreds of specific Linux packages that are included in that OS.

rPath automates system construction steps when generating a manifest:

• Dependency resolution—rPath deeply models dependency requirements for every artifact in the hub. When generating a manifest, rPath adds software to the manifest as necessary to satisfy all dependencies.
• IT policy verification—rPath can apply custom IT policy to validate any aspect of a manifest. For example, regulatory requirements and internal security requirements are easy to enforce at manifest construction time.

Manifest build automation makes your systems “correct by construction.” Before a production system is ever provisioned or updated, build automation catches and prevents many common configuration errors.

This two step process—describe a blueprint, then generate a manifest—cleanly separates the human-input side of system creation and the automatable grunt-work side of system creation, making both sides faster and more effective. The result is better productivity and predictability in IT operations.

What’s Next?

The manifest is a great model, but what can you actually do with it? In upcoming posts, we’ll dig into the tasks rPath automates given a manifest, such as image generation and incremental system updates.

The Google Security Team just posted a blog entry attempting to restore meaning to "responsible disclosure." It is absolutely worth reading.

As I see it, the phrase "responsible disclosure" has been stripped of meaning by vendors who use it as an excuse not to prioritize their customers' computer security. Irresponsible vendors have tried to make "responsible disclosure" mean a one-sided arrangement in which vendors get to set all the timelines for disclosure of all vulnerabilities. This irresponsible attempt at redefinition is inevitably alienating "white hat" software vulnerability researchers, as it creates a more vulnerable software ecosystem.

Fundamentally, "responsible disclosure" means that the researcher makes a best effort to put the needs of the end user first. When it is reasonable to assume that the vulnerability is newly discovered, it helps the end user to disclose first to the maintainer of the software (whether that's a vendor or otherwise) first. It also helps the end user to give the maintainer time to fix it right so that the vulnerability is really fixed and new vulnerabilities are not created. But when the maintainer doesn't bother to give the vulnerability priority, that just gives attackers more time to discover and make use of the vulnerability to subvert the end users' systems, which does not serve the end users' interests.

When there is reason to think that a vulnerability is known outside of the responsible security investigation community, it is clearly responsible to immediately disclose at least enough information to allow security-conscious end users secure their systems against attack. It should also be noted that that is almost always enough information to allow the unscrupulous to separately discover and make use of the vulnerability, so in this case responsible handling of the security flaw involves the maintainer making it a top priority to find an immediate mitigation, even if further work is required (also at high priority) to fully resolve the issue.

Some have suggested that the "responsible disclosure" emperor is wearing no clothes. I disagree. I think the problem is that irresponsible vendors have substituted an unclothed mannequin for the emperor, and are trying to pass off irresponsible handling of vulnerabilities as "responsible disclosure".

Responsible disclosure must be, first of all, responsible, and the responsibility is primarily to the end user. Security disclosure that primarily addresses vendor convenience is irresponsible.

In the early days of team software development, everyone worked on a common directory of source code—and struggled to avoid conflicting changes and premature feature release. This became completely unworkable as software became more complex and teams became bigger.

The development community discovered an excellent solution in version control. With version control, you can consume large numbers of changes and easily organize them into stages of development and releases. Here are some universal features and benefits of version control:

1. Supply chain—Version control tells you who added an object and when. That makes everyone accountable for their changes and speeds troubleshooting.
2. Changesets—Given where you are (a version) and where you want to be (another version), version control systems can compute the change required to get there. That eliminates the menial, error-prone labor required by manual change.
3. Immutable versions—Versions are permanent—and lightweight, so there’s no need to purge them. The data you need is always on hand for auditing and reproduction.
4. Branches—A version control system partitions content by lifecycle stage and purpose. That prevents rough drafts from leaking into production, eliminating a common source of error and risk. Branches also enable strong access control, eliminating the risk of inappropriate access to content.

Version control is universal in software development. Why not in IT operations? The complexity, and consequences of error, are just as great—perhaps greater.

So, we have:

rPath’s Second Law of Automation: Version control every artifact and every relationship that feeds into deployed software systems.

With version control, change at any level is no longer a problem—it’s simply data to organize, absorb, and promote through the lifecycle. The benefits of supply chain, changesets, immutable versions, and branches are just as useful in operations as they are in software development.

But how do you realize those benefits? In my next few posts, we’ll explore how rPath applies the first and second laws to real-life automation problems.

I not so recently mentioned that I was pleased with work that Jet City Devices did to my wife's smartphone. What I didn't know at the time was that while the screen was no longer cracked, the touch sensor was not working correctly; it was only occasionally registering touch on some parts of the screen, at best.

When my wife finally mentioned this to me, I mailed Matt at Jet City Devices about the problem. He gave me things to try to make sure it wasn't software, and said if they didn't work, to send the phone back, that it was a rare defect.

It wasn't a software problem. But instead of sending the phone back, my wife just learned to use the d-pad to use most features of the phone, until we were past Jet City Devices's 90-day warranty on the fix. Oops.

I finally bought a cheap dumbphone (Motofone F3) for my wife so that she would not have to be without a phone for a few days, and I emailed Matt. He didn't even bring up being past the warranty, and told us to send in the phone and he'd fix it. He didn't realize how long it had been, I think, so he didn't realize to give us his new mailing address, as he had moved office. My wife shipped off the phone. (Incidentally, she liked the Motofone F3; small, light, relatively indestructible; I think if she didn't want google maps and spreadsheets on her phone she would have stuck with the F3!)

After a week and a half of not hearing anything, I mailed Matt again. He worked out that the phone had been delivered to the new tenants of his old space, drove over to his old space and picked up the phone from the new tenants, fixed it, and had it back in the mail that day.

As he promised, the touch sensor is now working.

After today’s World Cup Final, I am even more committed to boycott Soccer (or Football for the other 99.9% of the nations out there) by not watching it.

Too much acting. Too much cheating.

Imagine this discussion with your child (and no, it didn’t happen to mine, but I am sure it could):

“Daddy, who is Cristiano Ronaldo?”

“He’s this very famous player that earns millions per year.”

“Wow, he is really good. Look at his skills. The other team can only stop him with fouls, see? Look at the replay, you see how they… Oh, wait… They didn’t even touch him. He just fell off his feet and the other guy got red carded.”

“Well, you see, sometimes they play a little bit of acting,  you know, to improve their odds…”

“But isn’t that cheating?”

“It is, but…”

“So why do I get punished if I cheat at my test, or I am called out for plagiarizing, but they get away with it? And they have no shame that everybody will see the replay, and realize how crooked they are? Including the referees who got fooled for a second, and will get (rightfully) stonewalled after the game? And why is FIFA’s slogan ‘Fair Play’? Is it really fair to cheat?”

I am not original, I read some of these opinions on other sites. I am sure a lot of people feel the same. I am just wasting zeros and ones here.

Spain deserved to win, they were the better team (although I must admit I did not watch the whole game). Netherlands did not deserve the silver medal, after all the theatricals they played. No matter how talented Robben is, I lost all respect for him the moment he fell off his feet and claimed a penalty kick or whatever he was claiming when he was booked.

I am sure that, 4 years from now, I will forget all this, and I will waste other 90-minute chunks of my life. And I will again feel sorry for that. If FIFA does nothing to make the game what it used to be, I am afraid the game is doomed.

What does a building architect hand off to a builder? Not a list of step-by-step instructions—that would be impossibly detailed and (impossible to revise).

Instead, the architect creates a blueprint: a detailed, comprehensive picture of how the finished building should look, inside and out. Then engineers work backwards from the blueprint to plan and execute detailed construction steps.

Blueprints enable:

• Reproducibility—Given an up-to-date blueprint for any component, it’s possible to make a copy of that component. To build a copy of a house, you really need a blueprint for the house, not the house itself.
• Revision—In the context of a blueprint, it’s easy to describe and validate changes. “Add a door here. Whoa, scratch that—it will block the other doorway.”
• Review—Unlike the opaque components they describe, blueprints are auditable and reportable. “Glad we reviewed the blueprint before construction—it was too close to the street, so the city would have made us take it down.”

Models, or blueprints, occur almost everywhere in design and engineering. But strangely, they are rare in IT.

So, we have:

rPath’s First Law of Automation: Create comprehensive blueprints for every aspect of software systems, and use the blueprints to drive all change.

That’s the difference between a simple version-control repository and rPath system version control. rPath provides deep modeling—blueprints—for services, systems, applications, and OS components. Those blueprints enable reproducibility, revision, and review for IT systems—speeding change and massively reducing risk.

Who knows, they might even keep the city from bulldozing your house.

In successful, long-lived software projects, developers continually refactor their code. That means that in addition to adding features and fixing bugs, they clean up the hidden structure of the software. That cleaning up leads to easier extensibility and maintainability—which ultimately lead to higher higher quality and productivity.

The alternative? Software becomes steadily more complicated, becomes harder and slower to change, and eventually must be replaced wholesale or scuttled.

That situation sounds familiar to many sysadmins in charge of production servers. You start with a clean deployment of 20 identical servers with a simple software load on each. Over time you apply patches to the systems. Then app updates. Then a partial web-app customization on half the systems—now you have two classes. Then a new storage driver on a different subset of systems—now you have four.

After a few more rounds of updates, patches, configurations, emergency fixes, accidental changes, cosmic-ray hits, and mistaken configurations by “the new guy,” you find yourself with a bunch of snowflakes. Each system is now unique, brittle, impossible to understand—and risky to change in any way.

Usually, the only answer is to burn it down and start over.

But what if Ops could refactor servers just as programmers refactor code?

Some common refactoring operations are:

1. Finding commonalities in two different components, and pulling out the common part into a single reusable component.
2. Breaking down a monolithic complex component into smaller, reusable pieces.
3. Finding and eliminating components that are no longer used.
4. Pushing functionality up or down the inheritance hierarchy, making it more or less broadly available.

rPath drives change from a flexible model for each system. And those models make it possible to refactor by:

1. Finding commonalities—With a system component library (containing everything from whole operating systems down to individual packages), it’s easy to see that two system models are both using a similar config of the Squid proxy, and to change both to use a single reusable Squid component.
2. Breaking down monoliths—It’s hard to see the pattern in a complex software assemblage because it’s not written down anywhere. Complex applications in rPath are modelled as Conary recipes, which use a powerful declarative language for system automation. When you see your database installer listed as a clean set of steps, it’s easy to break it down into (say) separate pieces to install MySQL, to configure it, and to preload it with your application data.
3. Eliminating unused parts—Conary dependency handling automates this. You don’t have to do detective work to tell whether anyone is still using that old xinetd.conf—just let Conary drop it when it’s no longer needed.
4. Pushing functionality up or down the hierarchy—Step 1 is to have a hierarchy in the first place. With rPath, you can model base systems that describe your common builds, then derive variants for specific purposes. Each variant, such as a JBoss system model, can focus on just the relevant parts (such as JBoss’s required dependencies and configurations), not on the common stuff (such as using the right version of OpenSSH).

The benefit is really the same in both domains. When you continually improve the underlying framework, you don’t have to raze everything and start over from scratch. That’s a bad way to maintain your house and a bad way to maintain software systems. Refactoring means better productivity and better quality.

A while back I wrote about how I first got hooked on coffee and received some really good comments and suggestions for new coffees to try. I was immediately attracted to one coffee in particular: Zoka! Don’t ask me what it was but there was something about the name that tickled my fancy. So I sent their customer service department an email and asked if they would send me a few samples. The very next day I received a reply and the promisse that they would send me a couple of bags. Since I had also received 2 other bags of coffee from friends to try I figured I’d report here how the “coffee experiment” went.

Now, I don’t pretend to be an expert or a connoisseur and my oppinion here is nothing but, well… my own oppinion and is not meant to influence what brand of coffee you should buy. I just enjoy drinking coffee and hope you’ll find this useful.

Zoka‘s coffee arrived at my doorstep a few days later on a Friday afternoon. With names such as “Costa Rica Calle de Copey” and “Tangletown Blend“, the two neatly packed bags of whole beans looked very organic-y. The neat label in the front telling you when that specific bag was put together was a nice touch, almost like someone took their time to hand pick the coffee beans and gently places them into the bag (and it smelled great too!).

I first tried the “Costa Rica Calle de Copey” with its promisse to deliver a rich, fruity flavor. After grinding (espresso) enough coffee for my wife and I and impatiently waiting for the brew, we were both impressed with the strong flavor of melon and hints of lemon! I mean, I usually drink my coffee black with sugar (though I’ve drastically cut down on how much sugar I’m using now) and am not too fond of fruity coffee. Maybe because most of the time fruity coffee is a bit too acidic for me. I must say however that I was not put off by this blend and I can see myself drinking it out on the back porch after dinner.

Today I attended my very first GNOME Board of Directors meeting. Though I won’t be able to vote on issues until July 1st, it was great to witness how the process works and be able to ask questions and receive answers directly from current and former board members.

I must say that the entire meeting felt very streamlined and ran smoothly, partially because of people like Vincent, Brian, Germán and Paul who seem to be on top of everything and juggling several balls at the same time! Stormy must sleep very well at night knowing she can count on people of this caliber to steer the project.

Emily and Bastien, new members like myself, also made their presence felt by asking questions and getting involved in some topics that were discussed. I asked  a few questions myself and tried to absorb as much as possible, juggling work and the action taking place on Gobbi and IRC.

I feel bad for not being able to attend GUADEC this year as I’ll probably be the only Board member not to be present… but I am looking forward to working with these guys and gals and fill the shoes of those who are departing this year!

As my kids are starting to want to "look for things on the internet", I started caring about what they might accidentally stumble onto, even in an appropriately supervised context.

I had vaguely heard about OpenDNS for some time, but had not really paid much attention to it. A few relatively recent articles on using it to make an internet connection somewhat more "family-friendly" caught my attention, and I finally signed up for a free account to try it out.

I have a local caching bind which forwarded to the nameservers that TWC provides to me (and to which I redirect all outgoing nameserver traffic via firewall rules), and I really haven't noticed nameservice being slow, so the "speed up your internet" advertising from OpenDNS wasn't ringing true. But the ability to filter out the worst of the sites dedicated to things that I think don't have a place in my home was interesting. So I signed up for a free account, changed a few lines in my bind configuration, and packaged and installed ddclient according to OpenDNS's instructions so that OpenDNS will continue to associate my home network with my home network settings on those rare occasions when my IP changes.

We weren't seeing lots of questionable content before the switch, so the fact that we've seen a total of two sites blocked since we signed up for the service is fine. It says that I can establish what I think are reasonable controls and it won't get in the way of normal activities.

Purely because I appreciate the service (I don't really care very much about saving statistics for longer), I signed up for a paid account. This service seems to me to be worth the $9.95/year.

A few days ago, OpenDNS rolled out a new free service called FamilyShield -- you can use a pre-configured set of filters without setting up any account at all merely by using 208.67.222.123 and 208.67.220.123 as your DNS servers (they include detailed instructions for how to do this on many different OS variants). This is exactly the same thing you'd get by signing up for their service and enabling the same set of filters for your account, so it's easy enough to upgrade to their free service if you want to customize the filters -- you just sign up for a free account, change the IP address you use for the resolvers, choose the filters you want, associate your IP address with your account, and (if you, like most people, have a dynamic IP) set up one of the many dynamic DNS clients available (they list several) to keep that association up to date.

I'm just a satisfied customer.

As my kids are starting to want to "look for things on the internet", I started caring about what they might accidentally stumble onto, even in an appropriately supervised context.

I had vaguely heard about OpenDNS for some time, but had not really paid much attention to it. A few relatively recent articles on using it to make an internet connection somewhat more "family-friendly" caught my attention, and I finally signed up for a free account to try it out.

I have a local caching bind which forwarded to the nameservers that TWC provides to me (and to which I redirect all outgoing nameserver traffic via firewall rules), and I really haven't noticed nameservice being slow, so the "speed up your internet" advertising from OpenDNS wasn't ringing true. But the ability to filter out the worst of the sites dedicated to things that I think don't have a place in my home was interesting. So I signed up for a free account, changed a few lines in my bind configuration, and packaged and installed ddclient according to OpenDNS's instructions so that OpenDNS will continue to associate my home network with my home network settings on those rare occasions when my IP changes.

We weren't seeing lots of questionable content before the switch, so the fact that we've seen a total of two sites blocked since we signed up for the service is fine. It says that I can establish what I think are reasonable controls and it won't get in the way of normal activities.

Purely because I appreciate the service (I don't really care very much about saving statistics for longer), I signed up for a paid account. This service seems to me to be worth the $9.95/year.

A few days ago, OpenDNS rolled out a new free service called FamilyShield -- you can use a pre-configured set of filters without setting up any account at all merely by using 208.67.222.123 and 208.67.220.123 as your DNS servers (they include detailed instructions for how to do this on many different OS variants). This is exactly the same thing you'd get by signing up for their service and enabling the same set of filters for your account, so it's easy enough to upgrade to their free service if you want to customize the filters -- you just sign up for a free account, change the IP address you use for the resolvers, choose the filters you want, associate your IP address with your account, and (if you, like most people, have a dynamic IP) set up one of the many dynamic DNS clients available (they list several) to keep that association up to date.

I'm just a satisfied customer.

“Django 1.2 E-Commerce“ starts with a very ambitious goal: design, develop and deploy a functional ecommerce web site for the fictional CranStore.com company. Sounds great, doesn’t it?

I started flipping through the usual introductory pages explaining what Django is and why use it for a project like this. It was all fairly brief which already led me to believe that knowledge of Django’s inner works and basic setup and configuration was required to follow along.

Well, the instructions are not as streamlined as other programming books I’ve read and it could be fairly tricky to follow the examples and logic if you’re not already familiar with how Django works. There are several typos in the example code as well, mostly due to missing spaces between the commands and arguments. Adding to the confusion is the style the author chose to deliver his explanation of specific code changes, displaying snippets of code that will leave the reader wondering what specific file is being discussed.

Now, if none of these things sound scary to you, you’ll be happy to know that the project itself is fairly well designed and worth your time. It was also the first time I saw an example of integrating Django with Google Checkout to set up a “shopping cart” mechanism, and by the time you’re done with the second chapter, you will have a very basic but functional ecommerce web site.

The subsequent chapters were a blur, talking about adding external modules and services to enhance your site’s searching capabilities as well as exposing the data from your “store” via APIs and generating reports with ReportLab. The author also talks about making use of javascript to add that AJAX-y feeling that we’ve come to expect of most modern sites and how to take advantage of S3 storage to sell your product.

The last chapter finally walks you through a few different ways you can deploy your final project to the world out there. I thought it was interesting to see Fabric being mentioned as a driver for deployment, as I have been playing with it at work to help me perform a series of tests on several different hosts for QAing purposes. Come to think of it, this may have been the first time I’ve seen it mentioned in a book, so I’m glad that this project seems to be picking up steam.

Overall, even with the issues of poor proof checking of the source code and the “everything and the kitchen sink” approach to the first chapters, if you’re not new to Django and need to get some ideas on how to design and develop an ecommerce website, you may want to check out this book. I give it 3 out of 5 stars.

Another busy week at work means a quiet week here…

Heck of a week and GNOME Foundation Board of Directors

137 issues resolved and counting!

One important piece of news is that I have been elected to the GNOME Foundation Board of Directors together with other 6 members! Thank you all of you who voted for me! I intent to devote a bigger chunk of my time to be a proactive[...]

It’s been another crazy/hectic week for me at work, so blogging and pretty much all other activities had to take the back seat.

One important piece of news is that I have been elected to the GNOME Foundation Board of Directors together with other 6 members! Thank you all of you who voted for me! I intent to devote a bigger chunk of my time to be a proactive, hands on and very vocal member of this group and will strive to blog about my experience as much as I can!

Next week should be just as crazy as the last 2 but I will try to blog at least once to let people know what’s going on in my neck of the woods!

Conary 2.1.20 is a maintenance release.

Bug Fixes:

• Injector code for relative changesets creates the new trove once instead of once for every file being merged. (CNY-3444)
• symbol hashes found in RPMs are used by ksym deps as well as kernel deps (CNY-3442)
• UtilizeUser and UtilizeGroup policies now support macros in the user and group argument (CNY-3445)
• Make negative openpgp key cache entries to avoid repeated server queries for keys which don't exist (CNY-3450)

Conary-Policy 1.0.28 is a maintenance release.

• Changed to express dependencies on initscripts functions via a file dependency instead of a trove dependency; the ResolveFileDependencies policy changes to the proper trove dependency. (CNP-191)
• A bug that caused some file dependencies to not be resolved to trove dependencies correctly has been fixed. (CNP-190)

Three more days until my birthday! Want to take a look at my wish list?

It was a very busy week, so I didn’t spend the same amount of time writing as the previous weeks. Thank you for the kind wishes on my birthday!

Also, thank you Conzar, Aigars Mahinovs, Paulo Dias, Tassos Bassoukos and Rodney Dyer for the awesome tips related to playing World of Warcraft on Linux!

Playing World of Warcraft on Linux with Wine

Yes, I’m still alive and kicking. Been really busy at work and haven’t had the time or energy to write anything here. I have however kept my New Year’s resolution of reading a book during lunch break every day! I’m currently reading 2 books at the same time (check what other books I have read and those I have still waiting on my queue in the front page of my blog):

• For The Win by Cory Doctorow
• The Fourth Part of the World by Toby Lester

I feel that it is impossible to read Doctorow’s book and not feel the urge to play World of Warcraft (WoW). Impossible! I think the last time I played this game[...]

Yes, I’m still alive and kicking. Been really busy at work and haven’t had the time or energy to write anything here. I have however kept my New Year’s resolution of reading a book during lunch break every day! I’m currently reading 2 books at the same time (check what other books I have read and those I have still waiting on my queue in the front page of my blog):

• For The Win by Cory Doctorow
• The Fourth Part of the World by Toby Lester

I feel that it is impossible to read Doctorow’s book and not feel the urge to play World of Warcraft (WoW). Impossible! I think the last time I played this game I still lived in Northern New Jersey and was an Ubuntu user (yeah, I’ve matured since then).

So I wanted to play WoW really bad after reading a chapter but wasn’t sure how to do it since I only run GNU/Linux and didn’t feel like setting up a Windows VM just to play a game! Talking to my long time friend Vinny I suddenly realized that I never tried to run the game using Wine, so I decided that once I got home I’d take it for a spin.

After downloading a 3.3MB installer from Blizzard, and installing Wine (sudo conary update wine), I went ahead and ran wine TryWoW.exe. What followed was a series of screens where I simply accepted the defaults and clicked my way through. There were times when a dialog with no messages popped up where all you could do was click on the OK button… so I did.

The installation did require access to the internet and a bundle of approximately 35MB was downloaded in the process. Once the wizard finished I had the option to play the game right there and then! How exciting!

The game takes a few extra seconds to start but eventually I was face to face with the login window. However, I noticed that I had no audio and the video quality lacked a bit. Googling around brought me to these instructions. So I modified my configuration file to include:

SET gxApi “opengl”

SET Sound_SoundOutputSystem “1″

SET Sound_SoundBufferSize “150″

Conary 2.1.19 is a maintenance release.

New Features:

• added --disconnected option as an advanced option for updates (CNY-3105)
• conary q --path /dir/ (with the ending /) now displays the owners of all files in /dir. The list of files is sorted before the owning troves are looked up. (CNY-610)
• The Requires policy can now take an removeFlagsByDependencyClass keyword argument to customize the set of flags to match platform behavior. (CNY-3443)

Bug Fixes:

• Walking troves on the local system no longer walks implied linkages (CNY-3440)
• sizeOverride metadata has no any effect if it is set to zero. (CNY-3441)
• cvc now adds recipes as text files even if --binary is specified. (CNY-3200)
• RPM kernel deps now include the symbol hash if it is defined within the RPM. (CNY-3442)

Conary 2.1.18 is a maintenance release.

Bug Fixes:

• An optimization introduced in Conary 2.1.0 (CNY-3282) caused filter expressions using tuples to express required or forbidden file mode bits at build time. This regression has been fixed. (CNY-3437)
• sizeOverride metadata is now never copied when a new version of a trove is created. (CNY-3438)
• addCapsule now fails with a clear error when the same file is added multiple times.

Internal Changes:

• When looking for an LGPL libelf, the Conary build process wil now look for either "Library" or "Lesser" in the name of the license, and will consider /usr/include/libelf.h as a candidate LGPL header.
• transport.URLOpener now shares no_proxy support with the lookaside cache.

Last night I helped a friend recover his data he had stored on an Iomega NAS.

The disks were fine, the rest of the hardware had failed.

Prior to me being involved in this, my friend had installed Ubuntu on an older machine and had installed both drives.

Not having played with RAID for quite some time, I had to acquire some knowledge first – google to the rescue!

In the process I used the wrong option to mdadm (–create instead of –assemble), so I messed up the RAID descriptor on one of the disks. Fortunately, the second disk was fine.

Here is what I ended up doing:

• install mdadm, a utility to configure RAID devices.

• install lvm2, a utility to configure LVM (Logical Volume Manager).

• Run:

mdadm –assemble /dev/md9 /dev/sdc1 –run

(this adds one of the partitions on the existing drives, /dev/sdc1, into a RAID device /dev/md9 running in degraded mode, i.e. with not enough disks – that’s what –run does)

vgchange -a y

(this scans all drives, including the newly created /dev/md9, for logical volumes)

It should print something about a new device with a rather cryptic name, I think something like /dev/vg1_md9/lv1. lvdisplay will show the available volumes.

This new device has a filesystem that can be mounted:

mkdir /tmp/olddrive
mount /dev/vg1_md9/lv1 /tmp/olddrive -o ro

After this, the directory /tmp/olddrive is associated with the contents of the filesystem.

There may be better ways to achieve the same thing, but this is what worked.

Three more days until my birthday! Want to take a look at my wish list?

Here’s what happened this week:

Not Going to GUADEC 2010

After several weeks agonizing over how to afford a trip to attend GUADEC 2010, I have finally arrived at the conclusion that I won’t be able to make it. Due to my current financial situation[...]

Emma’s Very Busy Week

I’m the proud father of two voracious young readers! From very early on I was able to instill into my daughters the pleasure of reading books. They also happen to be girly girls, so while most kids ask for toys and video games for their birthdays[...]

I’m the proud father of two voracious young readers! From very early on I was able to instill into my daughters the pleasure of reading books. They also happen to be girly girls, so while most kids ask for toys and video games for their birthdays, mine ask for books and clothes/shoes!

During third grade my oldest daughter happened to pick up “Emma’s very busy week” by Heather Dakota from her school’s library and just couldn’t put it down! I believe that throughout the entire third grade she must have borrowed this book at least 6 times! As a young kid, I remember I could read some books that I really enjoyed over and over, enjoying it every single time as if I were reading it for the first time. She must have gotten this from me.

So now with the Summer break fast approaching she has asked me to buy
her own copy of the book. Again, I usually get all of my books from the public library but those I fall in love with, I just got to have them, so she must have gotten it from me too! I sat down in front of the computer and searched on Amazon, Barnes and Noble, Borders, Books on Press, etc, but just could not find a single reference or a way to buy “Emma’s very busy week”.

So I did what everyone does when in doubt and spent some time ‘Googling’ around until I came across Heather Dakota‘s web site. Feeling that I had already exhausted my options, I figured I’d drop her an email and see if she could help us find a place where we could buy the book.

That same day I was pleasantly surprised to receive an email from Heather with a generous offer: she happened to have an extra copy of the book at her house and asked me if I’d mind if she sent it autographed to my daughter! Better yet, she was meeting the illustrator the very next day and said she would also get it autographed!

I profusely thanked her for the offer and decided I would not break the news to my daughter and see how  she would react when she received the package in the mail.

Needless to say, she was in heaven when she received the package yesterday!!! The book was promptly devoured while I took my dog to training class and she’s planning to read it again today!

Thank you Heather for making my daughter’s day! I thank you from the bottom of my heart and we are all looking forward to your next book!

After several weeks agonizing over how to afford a trip to attend GUADEC 2010, I have finally arrived at the conclusion that I won’t be able to make it. Due to my current financial situation and obligations as a father of two, it would be unrealistic to hop on a plane and go to this event right now for me. Unfortunately none of the organizations I approached were able to offer me any type of financial support, understandable due to today’s economy issues.

I want to thank the amazing crew behind the GNOME Travel Committee for a very generous offer and for being so patient and allowing me to take a bit of time to make my decision.

I also want to thank the organizers for choosing my presentation and keeping me updated with the tentative schedule for the event. I’m really sorry I won’t be able to present it and I can only hope that someone else can fill in the vacancy.

Finally, I wish you all a wonderful event and I will be watching from the sidelines for the many amazing things that are sure to come out of GUADEC.

Conary 2.1.17 is a maintenance release.

New Features:

• Added a new option to the mirror configuration, matchTroveSpecs which is a list of regular expressions that are matched against full troveSpecs in order to select which troves will be mirrored. (CNY-3334)
• The addCapsule source action now takes a ignoreAllConflictingTimes boolean argument which causes it to not raise errors when overlapping paths otherwise allowed have conflicting mtimes. (CNY-3415)
• Made conary.lib.magic more resiliant to errors that show up only with certain implementations of libelf. (CNY-3430)

Bug Fixes:

• Output text for the conary update command has been changed to more clearly describe dependency resolution problems. (CNY-3154)
• Shared library RPATH entries with a double leading slash (//) no longer cause paths to be recorded in soname dependencies in certain circumstances. (CNY-3425)
• Allow RPM packages to create /etc/passwd and /etc/group properly, even if the files already exist (due to a info- package, for example) (CNY-3428)
• Running Conary updates in single-threaded mode correctly execute group trove scripts. (CNY-3431)
• Size stored in troveInfo now excludes any capsules (CNY-3432)
• The sizeOverride metadata field can be used to override the reported total file size of a trove. (CNY-3432)

Internal Changes:

• Combined TroveCache and TroveCacheWrapper objects in build code.

rBuild 1.2.4 is a maintenance release.

New Features:

• The "rbuild checkout" command now takes a "--factory=" option. Also, when checking out a package with a name that starts with "factory-" it is automatically set to be a factory. (RBLD-125)
• rBuild now logs all its actions and outputs to a .rbuild/log file either in the checkout (where possible) or under $HOME (otherwise). (RBLD-225)

API Changes:

• The "rbuild checkout --derive" command has been changed to match Conary changes, and will now work only with Conary 2.1.16 or later. (CNY-3348)

Really? I mean, really?

I was just on the phone with Starwood hotels, and to verify my identity they wanted my web password. He clearly typed it in on the other end to make sure it was right. I've never been asked for a password over the phone. Giving it out just felt plain wrong.

Of course, I use the same junk password on piles of web sites I don't care about. It was my password at Red Hat when we had open telnet access. Hard to believe.

Guess I'll go change a few dozen web passwords today though. Thanks Starwood.

I’m really tired of people and pundits going on and on about how we have so much oil that just can’t be reached because the mean old government won’t let the oil companies drill for it. Remember “Drill, Baby, Drill”? ANWR? Yeah, that.

FOX Business News correspondent John Stossel is the latest to blame the government for keeping all those precious lands oil-rig free, all the while pushing the oil companies into dangerous deepwater exploration.

[Former Shell President John Hofmeister is] right. More than 50% of Western land is owned by the federal government. But 75% of that is off limits or restricted for private drilling. Land that the government estimates contains 20 billion barrels. If government would just step out of the way and let people drill, oil companies wouldn’t have to go so far offshore!

Well, the truth is that the oil companies were going to go deep-sea drilling at some point anyway, even if they could have gotten permission to drill on government land. They’re just there sooner, but the main reason they are there has less to do with the government than our insatiable demand for “Texas tea”.

20 billion barrels sure sounds like a lot until you put it in perspective up against what the United States consumes. According to the US Energy Information Adminstration’s summary page, we as a country consume 19.5 million barrels a day. At that rate, 20 billion barrels will give us an extra 2.8 years of oil.

So, even if the government “stepped out of the way”, it will cost a lot of money and time to get those oil wells online and producing, and there will be an environmental cost. And after all that, you’ll have a little less than three years of oil to show for it. Those consumption figures were current as of 2009; by the time we have those hypothetical rigs online, who knows where are consumption will be? Even if consumption were to stay flat or decrease, will the rest of the developing world curb their nascent appetites? (Mumble mumble China mumble.)

Is having an extra three years of oil worth spoiling precious natural resources? According to this chart, the entire Gulf of Mexico is estimated as having only 3.5 billion barrels total. That’s enough to last a paltry 177 days at the US’s current rate of consumption.

While we’re doing some number crunching, let’s think about those thousands of barrels of oil that are currently spewing into the Gulf of Mexico from the foundered Deepwater Horizion. The government estimates—conservatively—that the flow rate is 19,000 barrels of oil a day. At that rate, as of this writing, over 800,000 barrels have spilled in the Gulf of Mexico. It’s a spill so large it’s easily visible from space. It’s going to ruin coastal fishermen’s livelihoods, kill wildlife, and spoil the coastlines of several states. But had we been able to use the oil that’s been spilled, it would only equal the amount of oil that the entire US consumes in a single hour. ONE. LOUSY. HOUR.

The idea that somehow we could ever become energy independent on producing our own oil alone is a complete crock unless we learn to do with a lot less of the stuff and look for more sustainable resources. Oil is not the final answer, and if we act like it is, we’re going to be in a heap of trouble when the wells run dry and we have nothing to show for it.

I’m really tired of people and pundits going on and on about how we have so much oil that just can’t be reached because the mean old government won’t let the oil companies drill for it. Remember “Drill, Baby, Drill”? ANWR? Yeah, that.

FOX Business News correspondent John Stossel is the latest to blame the government for keeping all those precious lands oil-rig free, all the while pushing the oil companies into dangerous deepwater exploration.

[Former Shell President John Hofmeister is] right. More than 50% of Western land is owned by the federal government. But 75% of that is off limits or restricted for private drilling. Land that the government estimates contains 20 billion barrels. If government would just step out of the way and let people drill, oil companies wouldn’t have to go so far offshore!

Well, the truth is that the oil companies were going to go deep-sea drilling at some point anyway, even if they could have gotten permission to drill on government land. They’re just there sooner, but the main reason they are there has less to do with the government than our insatiable demand for “Texas tea”.

20 billion barrels sure sounds like a lot until you put it in perspective up against what the United States consumes. According to the US Energy Information Adminstration’s summary page, we as a country consume 19.5 million barrels a day. At that rate, 20 billion barrels will give us an extra 2.8 years of oil.

So, even if the government “stepped out of the way”, it will cost a lot of money and time to get those oil wells online and producing, and there will be an environmental cost. And after all that, you’ll have a little less than three years of oil to show for it. Those consumption figures were current as of 2009; by the time we have those hypothetical rigs online, who knows where are consumption will be? Even if consumption were to stay flat or decrease, will the rest of the developing world curb their nascent appetites? (Mumble mumble China mumble.)

Is having an extra three years of oil worth spoiling precious natural resources? According to this chart, the entire Gulf of Mexico is estimated as having only 3.5 billion barrels total. That’s enough to last a paltry 177 days at the US’s current rate of consumption.

While we’re doing some number crunching, let’s think about those thousands of barrels of oil that are currently spewing into the Gulf of Mexico from the foundered Deepwater Horizion. The government estimates—conservatively—that the flow rate is 19,000 barrels of oil a day. At that rate, as of this writing, over 800,000 barrels have spilled in the Gulf of Mexico. It’s a spill so large it’s easily visible from space. It’s going to ruin coastal fishermen’s livelihoods, kill wildlife, and spoil the coastlines of several states. But had we been able to use the oil that’s been spilled, it would only equal the amount of oil that the entire US consumes in a single hour. ONE. LOUSY. HOUR.

The idea that somehow we could ever become energy independent on producing our own oil alone is a complete crock unless we learn to do with a lot less of the stuff and look for more sustainable resources. Oil is not the final answer, and if we act like it is, we’re going to be in a heap of trouble when the wells run dry and we have nothing to show for it.