Weekend Project: Get Started with Tahoe-LAFS Storage Grids

Here's an axiom for every major organization to memorize: if you have data, then you have a data storage problem. But while the cloud service players are happy to compete for your business, you may not need to purchase a solution. A number of open source projects offer flexible ways to build your own distributed, fault-tolerant storage network. This weekend, let's take a look at one of the most intriguing offerings: Tahoe LAFS.

Here's an axiom for every major organization to memorize: if you have data, then you have a data storage problem. But while the cloud service players are happy to compete for your business, you may not need to purchase a solution. A number of open source projects offer flexible ways to build your own distributed, fault-tolerant storage network. This weekend, let's take a look at one of the most intriguing offerings: Tahoe LAFS.

Tahoe is a "Least Authority File System" — the LAFS you often see in concert with its name. The LAFS design is an homage to the security world's "principle of least privilege": simply put, Tahoe uses cryptography and access control to protect access to your data. Specifically, the host OS on a Tahoe node never has read or write access to any of the data it stores: only authenticated clients can collect and assemble the correct chunks from across the distributed nodes, and decrypt the files.

Beyond that, though, Tahoe offers peer-to-peer distributed data storage with adjustable levels of redundancy. You can tune your "grid" for performance, fault-tolerance, or strike a balance in between, and you can use heterogeneous hardware and service providers to make up your nodes, providing you with a second layer of protection. Furthermore, although you can use Tahoe-LAFS as a simple distributed filesystem, you can also run web and (S)FTP services directly from your Tahoe grid.

Installation and Testing

The most recent Tahoe release is version 1.9.1, from January 2012. The project provides tarballs for download only, but many Linux distributions now offer the package as well, so check with your package management system first. Tahoe is written in Python, and uses the Twisted framework, as well as an assortment of auxiliary Python libraries (for cryptography and other functions). None are particularly unusual, but if you are installing from source, be sure to double check your dependencies.

Once you have unpacked the source package, execute python ./setup.py build to generate the Tahoe command line tools, then run python ./setup.py test to run the installer's sanity check suite. I found that the Ubuntu package failed to install python-mock, but Tahoe's error messages caught that mistake and allowed me to install the correct library without any additional trouble.

Now that you have the Tahoe tools built, you can connect to the public test grid to get a feel for how the storage system works. This grid is maintained by the project, and is variously referred to as pubgrid or Test Grid. You can experiment with the Tahoe client apps on pubgrid — however, because it is a testbed only, its uptime is not guaranteed, and the maintainers may periodically wipe and rebuild it.

First, run tahoe create-client. This creates a local client-only node on your machine (meaning that it does not offer storage space to the grid), which you will connect to pubgrid by editing the configuration file in ~/.tahoe/tahoe.cfg. Open the tahoe.cfg file and edit the nickname = and introducer.furl = lines.

The nickname is any moniker you choose for your node. During this testing phase, the name makes no difference, but when deploying a grid, useful names can help you keep better tabs on your nodes' performance and uptime. The "introducer" is Tahoe lingo for the manager node that oversees a grid — keeping track of the participating nodes in a publish/subscribe "hub" fashion. The pubgrid's current "FURL" address is pb://tin57bdenwkigkujmh6rwgztcoh7ya7t@pubgrid.tahoe-lafs.org:50528/introducer — but check the Tahoe wiki before entering it in the configuration file, in case it has changed.

Save your configuration file, then run ./tahoe start at the command line. You're now connected! By default, Tahoe offers a web-based interface running at http://127.0.0.1:3456 ... open that address in your web browser, and you will see both a status page for pubgrid (including the grid IDs of nearby peers), and the controls you need to create your own directories and upload test files.

File Storage and Other Front-Ends

Part of Tahoe's LAFS security model is that the directories owned by other nodes are not searchable or discoverable. When you create a directory (on pubgrid or on any other grid), a unique pseudorandom identifier is generated that you must bookmark or scrawl down someplace where you won't forget it. The project has created a shared public directory on pubgrid at this long, unwieldy URI, which gives you an idea of the hash function used.

You can add directories or files in the shared public directory, or create new directories and upload files of your own. But whenever you do, it is up to you to keep track of the URIs Tahoe generates. You can share Tahoe files with other users by sending them the URIs directly. Note also that whenever you upload a file, you have the option to check a box labeled "mutable." This is another security feature: files created as immutable are write-protected, and can never be altered.

In the default setup, your client-only node is not contributing any local disk space to the grid's shared pool. That setting is controlled in the [storage] stanza of the tahoe.cfg file. Bear in mind that the objects stored on your contribution to the shared pool are encrypted chunks of files from around the grid; you will not be able to inspect their contents. For a public grid, that is something worth thinking about, although when running a grid for your own business or project, it is less of a concern.

If all you need to do store static content and be assured that it is securely replicated off-site, then the default configuration as used by pubgrid may be all that you need. Obviously, you will want to run your own storage nodes, connected through your own introducer — but the vanilla file and directory structure as exposed through the web GUI will suffice. There are other options, however.

Tahoe includes REST API in tandem to the human-accessible web front-end. This allows you to use a Tahoe LAFS grid as the storage engine to a web server. The API exposes standard GET, PUT, POST, and DELETE methods, and supports JSON and HTML output. The API is customized to make Tahoe's long, human-unreadable URIs easier to work with, and provides utilities to more easily work with operations (such as search) than can take longer on a distributed grid than they would on a static HTTP server.

There is also an FTP-like front-end, which support SSL-encrypted SFTP operations, and a command-line client useful for server environments or remote grid administration. Finally, a "drop upload" option is available in the latest builds, which allows Tahoe to monitor an upload directory, and automatically copy any new files into the grid.

Running Your Own Grid

The pubgrid is certainly a useful resource for exploring how Tahoe and its various front-ends function, but for any real benefit you need to deploy your own grid. Designing a grid is a matter of planning for the number of storage nodes you will need, determining how to tune the encoding parameters for speed and redundancy, and configuring the special nodes that manage logging, metadata, and other helper utilities.

The storage encoding parameters include shares.needed, shares.total, and shares.happy (all of which are configurable in the tahoe.cfg file). A file uploaded to the grid is divided into shares.needed chunks, to be distributed across the nodes. Tahoe will replicate a total of shares.total chunks, so total must be greater-than-or-equal-to needed. If they are equal, there is no redundancy.

The third parameter, shares.happy, defines the minimum number of nodes the chunks of any individual file must be spread across. Setting this value too low sacrifices the benefits of redundancy. By default, Tahoe is designed to be tolerant of nodes whose availability comes and goes, not only to cope with failure, but to allow for a truly distributed design where some nodes can be disconnected and not damage the grid as a whole. There is a lot to consider when designing your grid parameters; a good introduction to the trade-offs is hosted on the project's wiki.

You can run services — such as the (S)FTP and HTTP front-ends discussed above — on any storage node. But you will also need at least one special node, the introducer node required for clients to make an initial connection to the grid. Introducers maintain a state table keeping track of the nodes in a grid; they look for basic configuration in the [node] section of tahoe.cfg, but ignore all other directives.

To start yours, create a separate directory for it (say, .tahoe-my-introducer/, change into the directory, and run tahoe create-introducer . followed by tahoe start .. When launched, the new introducer creates a file named introducer.furl; this holds the "FURL" address that you must paste into the configuration file on all of your other nodes.

You can also (optionally) create helper, key-generator, and stats-gatherer nodes for your grid, to offload some common tasks onto separate machines. A helper node simply assists the chunk replication process, which can be slow if many duplicate chunks are required. You can designate a node as a helper by setting enabled = true under the [helper] stanza in tahoe.cfg.

The setup process key-generators and stats-gatherers is akin to that for introducers: run tahoe create-key-generator . or tahoe create-stats-gatherer . in a separate directory, followed by tahoe start .. Stats gatherers are responsible for logging and maintaining data on the grid, while key generators simply speed up the computationally-expensive process of creating cryptographic tokens. Neither is required, but using them can improve performance.

Setting up a Tahoe LAFS grid to serve your company or project is not a step to be taken lightly — you need to consider the maintenance requirements as well as how the competing speed and security features add up for your use case. But the process is simple enough that you can undertake it in just a few days, and even run multi-node experiments, all with a minimum of fuss.

]]>

Weekend Project: Get Started with Tahoe-LAFS Storage Grids

Here's an axiom for every major organization to memorize: if you have data, then you have a data storage problem. But while the cloud service players are happy to compete for your business, you may not need to purchase a solution. A number of open source projects offer flexible ways to build your own distributed, fault-tolerant storage network. This weekend, let's take a look at one of the most intriguing offerings: Tahoe LAFS.

Tahoe is a "Least Authority File System" — the LAFS you often see in concert with its name. The LAFS design is an homage to the security world's "principle of least privilege": simply put, Tahoe uses cryptography and access control to protect access to your data. Specifically, the host OS on a Tahoe node never has read or write access to any of the data it stores: only authenticated clients can collect and assemble the correct chunks from across the distributed nodes, and decrypt the files.

Beyond that, though, Tahoe offers peer-to-peer distributed data storage with adjustable levels of redundancy. You can tune your "grid" for performance, fault-tolerance, or strike a balance in between, and you can use heterogeneous hardware and service providers to make up your nodes, providing you with a second layer of protection. Furthermore, although you can use Tahoe-LAFS as a simple distributed filesystem, you can also run web and (S)FTP services directly from your Tahoe grid.

Installation and Testing

The most recent Tahoe release is version 1.9.1, from January 2012. The project provides tarballs for download only, but many Linux distributions now offer the package as well, so check with your package management system first. Tahoe is written in Python, and uses the Twisted framework, as well as an assortment of auxiliary Python libraries (for cryptography and other functions). None are particularly unusual, but if you are installing from source, be sure to double check your dependencies.

Once you have unpacked the source package, execute python ./setup.py build to generate the Tahoe command line tools, then run python ./setup.py test to run the installer's sanity check suite. I found that the Ubuntu package failed to install python-mock, but Tahoe's error messages caught that mistake and allowed me to install the correct library without any additional trouble.

Now that you have the Tahoe tools built, you can connect to the public test grid to get a feel for how the storage system works. This grid is maintained by the project, and is variously referred to as pubgrid or Test Grid. You can experiment with the Tahoe client apps on pubgrid — however, because it is a testbed only, its uptime is not guaranteed, and the maintainers may periodically wipe and rebuild it.

First, run tahoe create-client. This creates a local client-only node on your machine (meaning that it does not offer storage space to the grid), which you will connect to pubgrid by editing the configuration file in ~/.tahoe/tahoe.cfg. Open the tahoe.cfg file and edit the nickname = and introducer.furl = lines.

The nickname is any moniker you choose for your node. During this testing phase, the name makes no difference, but when deploying a grid, useful names can help you keep better tabs on your nodes' performance and uptime. The "introducer" is Tahoe lingo for the manager node that oversees a grid — keeping track of the participating nodes in a publish/subscribe "hub" fashion. The pubgrid's current "FURL" address is pb:// This e-mail address is being protected from spambots. You need JavaScript enabled to view it :50528/introducer — but check the Tahoe wiki before entering it in the configuration file, in case it has changed.

Save your configuration file, then run ./tahoe start at the command line. You're now connected! By default, Tahoe offers a web-based interface running at http://127.0.0.1:3456 ... open that address in your web browser, and you will see both a status page for pubgrid (including the grid IDs of nearby peers), and the controls you need to create your own directories and upload test files.

File Storage and Other Front-Ends

Part of Tahoe's LAFS security model is that the directories owned by other nodes are not searchable or discoverable. When you create a directory (on pubgrid or on any other grid), a unique pseudorandom identifier is generated that you must bookmark or scrawl down someplace where you won't forget it. The project has created a shared public directory on pubgrid at this long, unwieldy URI, which gives you an idea of the hash function used.

You can add directories or files in the shared public directory, or create new directories and upload files of your own. But whenever you do, it is up to you to keep track of the URIs Tahoe generates. You can share Tahoe files with other users by sending them the URIs directly. Note also that whenever you upload a file, you have the option to check a box labeled "mutable." This is another security feature: files created as immutable are write-protected, and can never be altered.

In the default setup, your client-only node is not contributing any local disk space to the grid's shared pool. That setting is controlled in the [storage] stanza of the tahoe.cfg file. Bear in mind that the objects stored on your contribution to the shared pool are encrypted chunks of files from around the grid; you will not be able to inspect their contents. For a public grid, that is something worth thinking about, although when running a grid for your own business or project, it is less of a concern.

If all you need to do store static content and be assured that it is securely replicated off-site, then the default configuration as used by pubgrid may be all that you need. Obviously, you will want to run your own storage nodes, connected through your own introducer — but the vanilla file and directory structure as exposed through the web GUI will suffice. There are other options, however.

Tahoe includes REST API in tandem to the human-accessible web front-end. This allows you to use a Tahoe LAFS grid as the storage engine to a web server. The API exposes standard GET, PUT, POST, and DELETE methods, and supports JSON and HTML output. The API is customized to make Tahoe's long, human-unreadable URIs easier to work with, and provides utilities to more easily work with operations (such as search) than can take longer on a distributed grid than they would on a static HTTP server.

There is also an FTP-like front-end, which support SSL-encrypted SFTP operations, and a command-line client useful for server environments or remote grid administration. Finally, a "drop upload" option is available in the latest builds, which allows Tahoe to monitor an upload directory, and automatically copy any new files into the grid.

Running Your Own Grid

The pubgrid is certainly a useful resource for exploring how Tahoe and its various front-ends function, but for any real benefit you need to deploy your own grid. Designing a grid is a matter of planning for the number of storage nodes you will need, determining how to tune the encoding parameters for speed and redundancy, and configuring the special nodes that manage logging, metadata, and other helper utilities.

The storage encoding parameters include shares.needed, shares.total, and shares.happy (all of which are configurable in the tahoe.cfg file). A file uploaded to the grid is divided into shares.needed chunks, to be distributed across the nodes. Tahoe will replicate a total of shares.total chunks, so total must be greater-than-or-equal-to needed. If they are equal, there is no redundancy.

The third parameter, shares.happy, defines the minimum number of nodes the chunks of any individual file must be spread across. Setting this value too low sacrifices the benefits of redundancy. By default, Tahoe is designed to be tolerant of nodes whose availability comes and goes, not only to cope with failure, but to allow for a truly distributed design where some nodes can be disconnected and not damage the grid as a whole. There is a lot to consider when designing your grid parameters; a good introduction to the trade-offs is hosted on the project's wiki.

You can run services — such as the (S)FTP and HTTP front-ends discussed above — on any storage node. But you will also need at least one special node, the introducer node required for clients to make an initial connection to the grid. Introducers maintain a state table keeping track of the nodes in a grid; they look for basic configuration in the [node] section of tahoe.cfg, but ignore all other directives.

To start yours, create a separate directory for it (say, .tahoe-my-introducer/, change into the directory, and run tahoe create-introducer . followed by tahoe start .. When launched, the new introducer creates a file named introducer.furl; this holds the "FURL" address that you must paste into the configuration file on all of your other nodes.

You can also (optionally) create helper, key-generator, and stats-gatherer nodes for your grid, to offload some common tasks onto separate machines. A helper node simply assists the chunk replication process, which can be slow if many duplicate chunks are required. You can designate a node as a helper by setting enabled = true under the [helper] stanza in tahoe.cfg.

The setup process key-generators and stats-gatherers is akin to that for introducers: run tahoe create-key-generator . or tahoe create-stats-gatherer . in a separate directory, followed by tahoe start .. Stats gatherers are responsible for logging and maintaining data on the grid, while key generators simply speed up the computationally-expensive process of creating cryptographic tokens. Neither is required, but using them can improve performance.

Setting up a Tahoe LAFS grid to serve your company or project is not a step to be taken lightly — you need to consider the maintenance requirements as well as how the competing speed and security features add up for your use case. But the process is simple enough that you can undertake it in just a few days, and even run multi-node experiments, all with a minimum of fuss.

Subscribe to this comment's feed

Show/hide comments

Show/hide comment form

Accessibility Leaders in Linux

Accessibility to computers for people with vision, hearing, or physical impairments needs to be a part of fundamental design, and not an afterthought. Progress in the proprietary world is slow, and even slower in the Linux/FOSS world. But thanks to some dedicated people some significant work has been accomplished, and the groundwork laid for a common platform for all Linux distributions to build on.

Accessibility to computers for people with vision, hearing, or physical impairments needs to be a part of fundamental design, and not an afterthought. Progress in the proprietary world is slow, and even slower in the Linux/FOSS world. But thanks to some dedicated people some significant work has been accomplished, and the groundwork laid for a common platform for all Linux distributions to build on.

What is Accessibility?

A lot of people have rather negative attitudes towards accessible design, and dismiss it as a lot of bother for a tiny percentage of users. Some are even hostile to the whole concept, like it's conferring special privileges on the undeserving. I had a rather spirited discussion with a building contractor once upon a time who went out of his way to park his equipment on the handicap parking spaces when there was abundant regular parking closer to the work site. (The older I get the less I understand humanity.)

It's expensive to retrofit old buildings, sidewalks, public parks, and other structures to be friendlier to people who can't navigate heavy doors, stairs, narrow crowded store aisles, or walk any distance. My number one peeve is tiny restroom stalls with doors that open inwards. Not everyone is agile enough to climb on the darned seat in order to open the door. Car design is pretty much unchanged from the Model T. Public transit is often a contest of the fittest.

I can't think of a single good reason to write off our fellow humans and say "Well they'll just have to accept limitations" when so many limitations can be conquered because they're just engineering problems. Something I run into all the time is "Oh but there aren't many disabled people, why should I go out of my way for one or two percent of the population." Not only is this heartless, it's blind to reality because something like 50 million Americans have some kind of physical impairment, and the number is growing.

A bonus of accessible design is it's good design for everyone. It's absurd that houses, to give one prominent example, are still built with the same inadequate floor plans as they always have. Narrow doors and hallways are unfriendly to everyone, not just wheelchair users, but to anyone who has to move furniture, or carry laundry baskets or bags of groceries. Bathrooms with grab-bars and sit-down showers are safer for everybody, and kitchens always suck because they are designed by people who never use them. Electric outlets are placed as though we will plug things in only once and will never change our minds, so it's all right to make them inconvenient. What does it take to open a window? Can you arrange furniture sensibly and with good flow? And so on...take a look around your own house; chances are it suffers from a legacy of decades of bad designs mindlessly repeated.

There may not be a lot that FOSS can do about such problems, but one arena where we rule is computing, and that is the essential starting point. Being computer-literate and connected means tapping into a universe of information and essential services, and being able to program and design devices. We have a golden opportunity in FOSS to change people's live simply by writing code.

Who Will Write It?

Let's think about this a for a moment. The most-misused saying in FOSS is "Scratch your own itch," and the context is usually some variation on "Users are annoying and I don't want to think about them." Here is the quotation in context from Eric Raymond's The Cathedral and the Bazaar:

"1. Every good work of software starts by scratching a developer's personal itch.

Perhaps this should have been obvious (it's long been proverbial that "Necessity is the mother of invention") but too often software developers spend their days grinding away for pay at programs they neither need nor love."

Raymond also said, "I sent chatty announcements to the beta list whenever I released, encouraging people to participate. And I listened to my beta-testers, polling them about design decisions and stroking them whenever they sent in patches and feedback.

So a personal itch isn't just something that matters only to the programmer, but rather is something he or she has a strong interest in, and is interested in having actual real people use it. My interest in accessible technologies is fueled by two things: I've always been for the underdog, and some people close to me have had unnecessarily difficult lives because of a lack of helpful technology and rational design, and a lack of interest in developing any.

It is a large and complicated problem: impaired hearing, impaired vision, can't type, reading disabilities, can't use a mouse, low stamina for sitting at a computer, and any combination of these. How to translate images and audio, how to compose music or edit photos or draw pictures? Maybe someday we'll have something like Braille displays that display images for people who can't see, and a way to translate music into a form for people who can't hear, and tools for people with various impairments to have both input and output in whatever forms they want, and to invent new ones.

The proprietary world is ahead of FOSS in both software and hardware, and it's not all that advanced, but is stuck at turning plain text to speech, and speech to text. Even that is more challenging than it seems, as this interview with some of the GNOME Accessibility developers illustrates.

Accessible FOSS Leaders

The GNOME 2 project has long provided the best accessibility support: complete keyboard control of the desktop, screen magnification, onscreen keyboard, and the GNOME team even invented an open accessibility architecture, the AT-SPI (Assistive Technology Service Provider Interface), because they recognized that it needs to be fundamental part of GNOME's architecture. GNOME 3 accessibility support is incomplete, so stick with GNOME 2. KDE4 aims to support AT-SPI someday, but it's not there yet. KDE4 has its own sets of assistive technologies, but they only work in KDE4.

The Fedora Accessibility Guide makes the case for Linux and FOSS as the logical accessibility leaders:

"While the Graphical User Interface (GUI) is convenient for sighted users, it is often inhibiting to those with visual impairments because of the difficulty speech synthesizers have interpreting graphics. Linux is a great operating system for users with visual limitations because the GUI is an option, not a requirement. Most modern tools including email, news, web browsers, calendars, calculators, and much more can run on Linux without the GUI. The working environment can also be customized to meet the hardware or software needs of the user."

The Fedora Accessibility Guide has a lot of good information and is updated with nearly every Fedora release. Fedora always bundles a lot of accessibility applications.

Vinux, based on Ubuntu 10.10 Maverick Meerkat, is a complete live Linux distribution optimized for blind and visually impaired users. It bundles screen readers, full-screen magnifiers, built-in support for USB Braille displays, and optimized fonts and colors.

The Braille display support is a big deal because it's ready at bootup. It also comes with Compiz, which has some nice magnifying and navigating features using the mouse. Vinux can be installed to a hard drive, but you'll need someone who can see to do it. Vinux is the best and easiest way to get the best Linux accessibility applications working out of the box. Whatever you do, do not ever upgrade Vinux unless the maintainers say it is OK, because an upgrade will probably break the applications you depend on.

The Orca screen reader is the most fully-featured Linux screen reader. It supports multiple speech synthesizers and Braille displays. You need GNOME 2 for Orca to work because there are many glitches in GNOME 3. As GNOME and KDE continue to present moving targets stick with Vinux for best performance and least hassles.

Linux has several text-to-speech readers such as eSpeak and Festival that work fairly well, and Emacspeak, like Emacs, is a self-contained universe that is amazingly functional — inside of Emacs. The only good speech-to-text application for Linux (that I know of) is Julius. Julius is maintained by the Nagoya Institute of Technology and Kyoto University, and is quite sophisticated, though a fair bit of work to set up.

There is much untapped potential in Linux and FOSS for leading the way in accessible and assistive technologies. Perhaps someday a desktop roadmap will look like a journey, a succession of improvements that build on what came before, instead of a series of cliffs and do-overs.

]]>

What's New in LibreOffice 3.5

The LibreOffice 3.5 release is due out shortly, and this release comes with a number of improvements that free office suite users will find useful. From grammar checking to better importing for Microsoft Office documents, LibreOffice 3.5 contains a number of useful improvements. This release also contains preliminary work for porting LibreOffice to the Web and mobile devices.

The LibreOffice 3.5 release is due out shortly, and this release comes with a number of improvements that free office suite users will find useful. From grammar checking to better importing for Microsoft Office documents, LibreOffice 3.5 contains a number of useful improvements. This release also contains preliminary work for porting LibreOffice to the Web and mobile devices.

A quick reminder about LibreOffice releases. The LibreOffice folks maintain several branches that are meant for different use cases. The .0 releases (like 3.5.0 here) are meant for users that are happy to get new features fast, with the understanding that there may still be bugs. The bugs should not be show-stoppers, but may not be suitable for deploying for production use in a business. The .1 release is considered "stable," and the .2 release is "very stable" and .3 or later are dubbed "rock solid." See the post about the release cycles on the Document Foundation blog for more.

The upshot here is that you should only jump on the 3.5.0 release if you're willing to accept that it could have bugs that might impact your daily work. So far, I haven't run into any major issues with .0 releases, but I still would recommend that users think twice before doing irreplaceable work in a .0 release. Save early, and save often in any case.

New in Writer

Naturally, the first LibreOffice component I check out is Writer. Looking over the release notes you'll find a bunch of new goodies in Writer. Let's take a look at some of the most interesting.

The LightProof grammar checker extension is now bundled with LibreOffice 3.5. It has checks for improper punctuation, mixing up "a" and "an," word duplication, and more. It's not as extensive as the grammar checker in Microsoft Word (for example), but it's also less prone to false positives.

Want to add to the existing checks? You can install the LightProof editor and compile or modify rules. This is a bit rough at the moment, but should be improved in future releases. (Also, how many folks actively edit grammar rules? Not too many, I suspect.) In the meantime, if we could just ask that folks learn the difference between "its" and "it's" and "then" and "than," that'd be lovely.

Aside from trying to check users' grammar, the 3.5 release includes a much improved header/footer editing system. Instead of having to dig through several menus to find the header editor, just click on the header (or footer, if that's what you want to edit). Simple, right?

The 3.5 release also contains a number of improvements for font hinting, has a better page break indicator, updates the word count dialog as you type, and much more. Sadly, 3.5 does not bring a "write this TPS report for me" feature, but I hear the AI division of the Document Foundation is making great strides on that one.

New in Calc

If spreadsheets are your thing, there's a few goodies in Calc that may pique your interest.

LibreOffice now gives users the ability to specify the number of worksheets in a new document, so if the standard three is insufficient or over-generous, you can tweak that. I don't think I've ever needed more than 10 worksheets in Calc, but if you're really into huge spreadsheets, Calc now supports up to 10,000 sheets.

Calc now has a multi-line input bar, so if you're working on putting a longer piece of text into a spreadsheet or working on a really long equation, this will be much easier. If you don't know it's there, it's not terribly obvious. At the end of the formula bar, there's a little arrow icon. Just click that to expand the input bar.

If you use the Conditional Formatting feature, you'll be pleased to know that you can now set an unlimited number of rules. (I believe the previous limit was three.)

The 3.5 release of Calc also includes a number of new Calc functions, and a number of performance improvements.

New in Impress

One of the first things you notice in Impress is that you go straight to a new presentation rather than the wizard to select a template. I'm a fan of this change. If you want the wizard, though, you can still go to File -> Wizards -> Presentation. Win all the way around.

If you're importing a newer PowerPoint document (.pptx), you'll see that the custom shapes import is much improved. See the before and after on the wiki, or try out the example presentation. Likewise, the import for Smart Art from PowerPoint has been improved significantly.

One of the things I've noticed in prior releases of Impress is that if you try to export a presentation to PDF with some of the slides hidden, it doesn't work. This can be a bit of a PITA if you're hoping to, say, export a presentation or part of one for a third party that doesn't need to see the entire presentation. Now Impress allows hidden slides to stay hidden when you export them.

The Future!

What's really interesting in the 3.5 release isn't really ready for prime time. At FOSDEM, Micheal Meeks talked about LibreOffice and (among other things) the work that's going on for LibreOffice mobile and Web. Meeks demonstrated the Web front-end which can render into an HTML5 canvas in the browser.

To be sure, this is not ready for prime time. Meeks' demo at FOSDEM crashed a bit and was not as snappy as running LibreOffice on the desktop. But this may be a good way forward for folks that want to run LibreOffice in the browser. Why not a complete re-write? Meeks talked about this at FOSDEM, and also in the interview that he did just before. Basically, re-writes don't go so well. Says Meeks:

"My personal feeling is that doing the last 30% of a WYSIWYG web office suite takes 95% of the time – you see, the first bits (that the browser supports natively) are really rather easy. After that, you get to write a huge mound of JavaScript, which you distribute and run under a JIT in several not-very-compatible browsers. Of course, we can always abandon WYSIWYG, perhaps printing onto pages is really truly dead, but I'm not so sure that crisp, repeatable paginated layout is really truly gone for good yet. That's particularly true for drawings, posters, marketing etc."

He went on to talk about some of the spectacular failures of re-writes, like Corel's attempt to re-write its office suite in Java and Micropro's re-write of WordStar. Both failed completely. Instead, Meeks says that he'd like to "focus on a single product, with substantial source-code re-use for different segments."

The mobile app is also in development, and Meeks says that he's "optimistic that we can get something useful on-line for the end of 2012 or early 2013, but that of course depends on community involvement and help." The Android port should start with a document viewer, "evolving over time to allow editing."

If you're the type of person who loves to ride the cutting edge, I'd recommend grabbing 3.5 as soon as its out. If not, just hang tight for the 3.5.1 release or 3.5.2 if you're feeling really cautious. The next (3.5.1) release is tentatively scheduled for around March 5th, and 3.5.2 is tentatively scheduled for April 2nd through April 8th. The feature freeze for 3.6.0 is scheduled for the week of June 4th with the final release the week of July 30th.

]]>

Accessibility Leaders in Linux

Accessibility to computers for people with vision, hearing, or physical impairments needs to be a part of fundamental design, and not an afterthought. Progress in the proprietary world is slow, and even slower in the Linux/FOSS world. But thanks to some dedicated people some significant work has been accomplished, and the groundwork laid for a common platform for all Linux distributions to build on.

What is Accessibility?

A lot of people have rather negative attitudes towards accessible design, and dismiss it as a lot of bother for a tiny percentage of users. Some are even hostile to the whole concept, like it's conferring special privileges on the undeserving. I had a rather spirited discussion with a building contractor once upon a time who went out of his way to park his equipment on the handicap parking spaces when there was abundant regular parking closer to the work site. (The older I get the less I understand humanity.)

It's expensive to retrofit old buildings, sidewalks, public parks, and other structures to be friendlier to people who can't navigate heavy doors, stairs, narrow crowded store aisles, or walk any distance. My number one peeve is tiny restroom stalls with doors that open inwards. Not everyone is agile enough to climb on the darned seat in order to open the door. Car design is pretty much unchanged from the Model T. Public transit is often a contest of the fittest.

I can't think of a single good reason to write off our fellow humans and say "Well they'll just have to accept limitations" when so many limitations can be conquered because they're just engineering problems. Something I run into all the time is "Oh but there aren't many disabled people, why should I go out of my way for one or two percent of the population." Not only is this heartless, it's blind to reality because something like 50 million Americans have some kind of physical impairment, and the number is growing.

A bonus of accessible design is it's good design for everyone. It's absurd that houses, to give one prominent example, are still built with the same inadequate floor plans as they always have. Narrow doors and hallways are unfriendly to everyone, not just wheelchair users, but to anyone who has to move furniture, or carry laundry baskets or bags of groceries. Bathrooms with grab-bars and sit-down showers are safer for everybody, and kitchens always suck because they are designed by people who never use them. Electric outlets are placed as though we will plug things in only once and will never change our minds, so it's all right to make them inconvenient. What does it take to open a window? Can you arrange furniture sensibly and with good flow? And so on...take a look around your own house; chances are it suffers from a legacy of decades of bad designs mindlessly repeated.

There may not be a lot that FOSS can do about such problems, but one arena where we rule is computing, and that is the essential starting point. Being computer-literate and connected means tapping into a universe of information and essential services, and being able to program and design devices. We have a golden opportunity in FOSS to change people's live simply by writing code.

Who Will Write It?

Let's think about this a for a moment. The most-misused saying in FOSS is "Scratch your own itch," and the context is usually some variation on "Users are annoying and I don't want to think about them." Here is the quotation in context from Eric Raymond's The Cathedral and the Bazaar:

"1. Every good work of software starts by scratching a developer's personal itch.

Perhaps this should have been obvious (it's long been proverbial that "Necessity is the mother of invention") but too often software developers spend their days grinding away for pay at programs they neither need nor love."

Raymond also said, "I sent chatty announcements to the beta list whenever I released, encouraging people to participate. And I listened to my beta-testers, polling them about design decisions and stroking them whenever they sent in patches and feedback.

So a personal itch isn't just something that matters only to the programmer, but rather is something he or she has a strong interest in, and is interested in having actual real people use it. My interest in accessible technologies is fueled by two things: I've always been for the underdog, and some people close to me have had unnecessarily difficult lives because of a lack of helpful technology and rational design, and a lack of interest in developing any.

It is a large and complicated problem: impaired hearing, impaired vision, can't type, reading disabilities, can't use a mouse, low stamina for sitting at a computer, and any combination of these. How to translate images and audio, how to compose music or edit photos or draw pictures? Maybe someday we'll have something like Braille displays that display images for people who can't see, and a way to translate music into a form for people who can't hear, and tools for people with various impairments to have both input and output in whatever forms they want, and to invent new ones.

The proprietary world is ahead of FOSS in both software and hardware, and it's not all that advanced, but is stuck at turning plain text to speech, and speech to text. Even that is more challenging than it seems, as this interview with some of the GNOME Accessibility developers illustrates.

Accessible FOSS Leaders

The GNOME 2 project has long provided the best accessibility support: complete keyboard control of the desktop, screen magnification, onscreen keyboard, and the GNOME team even invented an open accessibility architecture, the AT-SPI (Assistive Technology Service Provider Interface), because they recognized that it needs to be fundamental part of GNOME's architecture. GNOME 3 accessibility support is incomplete, so stick with GNOME 2. KDE4 aims to support AT-SPI someday, but it's not there yet. KDE4 has its own sets of assistive technologies, but they only work in KDE4.

The Fedora Accessibility Guide makes the case for Linux and FOSS as the logical accessibility leaders:

"While the Graphical User Interface (GUI) is convenient for sighted users, it is often inhibiting to those with visual impairments because of the difficulty speech synthesizers have interpreting graphics. Linux is a great operating system for users with visual limitations because the GUI is an option, not a requirement. Most modern tools including email, news, web browsers, calendars, calculators, and much more can run on Linux without the GUI. The working environment can also be customized to meet the hardware or software needs of the user."

The Fedora Accessibility Guide has a lot of good information and is updated with nearly every Fedora release. Fedora always bundles a lot of accessibility applications.

Vinux, based on Ubuntu 10.10 Maverick Meerkat, is a complete live Linux distribution optimized for blind and visually impaired users. It bundles screen readers, full-screen magnifiers, built-in support for USB Braille displays, and optimized fonts and colors.

The Braille display support is a big deal because it's ready at bootup. It also comes with Compiz, which has some nice magnifying and navigating features using the mouse. Vinux can be installed to a hard drive, but you'll need someone who can see to do it. Vinux is the best and easiest way to get the best Linux accessibility applications working out of the box. Whatever you do, do not ever upgrade Vinux unless the maintainers say it is OK, because an upgrade will probably break the applications you depend on.

The Orca screen reader is the most fully-featured Linux screen reader. It supports multiple speech synthesizers and Braille displays. You need GNOME 2 for Orca to work because there are many glitches in GNOME 3. As GNOME and KDE continue to present moving targets stick with Vinux for best performance and least hassles.

Linux has several text-to-speech readers such as eSpeak and Festival that work fairly well, and Emacspeak, like Emacs, is a self-contained universe that is amazingly functional — inside of Emacs. The only good speech-to-text application for Linux (that I know of) is Julius. Julius is maintained by the Nagoya Institute of Technology and Kyoto University, and is quite sophisticated, though a fair bit of work to set up.

There is much untapped potential in Linux and FOSS for leading the way in accessible and assistive technologies. Perhaps someday a desktop roadmap will look like a journey, a succession of improvements that build on what came before, instead of a series of cliffs and do-overs.

Subscribe to this comment's feed

Show/hide comments

Show/hide comment form

A Look at the LinuxQuestions.org Members Choice Awards Winners

Every year LinuxQuestions.org hosts a members choice awards, which lets members of the site vote for their favorite Linux distributions and open source applications. There's not a lot of change in the results from last year, but the results do show a few interesting changes. GNOME has been unseated as favorite desktop, the GIMP has gone up in the polls even further, and LinuxQuestions.org has its first-ever tie in the NoSQL category.

Of course the first result most folks will want to know about is the favorite desktop distribution. Once again, Ubuntu has taken the crown, though not quite as handily as last year. Ubuntu came in first with 21.83%, compared to 28.56% last year. Interestingly, fewer people voted on the desktop distribution this year (875 compared to 970 last year). Fedora and openSUSE also lost ground between 2010 and 2011, but Slackware, Linux Mint, Arch, and CrunchBang all made decent gains.

On the server side, Debian came in first once again with 31.15% of the vote. (Compared to 29.35% last year.) CentOS dropped by about 1% to 14%, and Ubuntu LTS dropped about 3% from last year to 9.03%.

Desktop winner this year? KDE, with about 33% of the vote. This may be because the GNOME users are split between Cinnamon, GNOME Shell, Mate, and Unity. Xfce jumped from third place last year to second, with 27.59% this year. Unity had only 4.63% of the vote, and GNOME Shell had 19.14%. Again, with the desktop vote fewer people voted in 2011 than in 2010 (627 this year versus 708 last year). Openbox is now the champion of the window manager category – last year it was Compiz.

Cassandra had NoSQL database of the year last year, but this year it's tied with MongoDB. (Both had 26.23% each.) MySQL remains the most popular database, but it slipped below 50% this year.

LibreOffice now dominates with 81% of the votes – in 2010 OpenOffice.org was the office suite of the year, with more than 55%. LibreOffice had 36.5% last year. Note that some choices were no longer offered for the 2011 members choice, including Lotus Symphony and StarOffice.

Favorite text editor? Vim once again, with more than 31% of the vote. (But down, slightly from 35.88% last year.)

On the mobile side, Android is the clear and dominant winner with nearly 70% of the vote. CyanogenMod comes in second with a respectable 14.34% of the vote.

The full results are on the LinuxQuestions.org site.

What Does it Mean?

It should go without saying that the LinuxQuestions.org awards are in good fun and give a bit of insight into community choices, but probably shouldn't be taken as gospel for the larger community. It is interesting to see how tastes change year over year, and it's also a reminder just how much choice Linux users have.

If you didn't vote in the LinuxQuestions.org awards, give us a shout in the comments about your favorites.

Comming Up:


Asbestos death rates are on the rise, learn what you need to lookout for when it comes to asbestos and mesothelioma.

Need a Job? New Study Says Learn Linux.

No one disputes that that tech jobs are fueling the economy in the U.S. and around the world. The U.S. President said in his recent State of the Union address that there are twice as many openings in the science and technology sector as there are people to fill them. But where exactly are these jobs? And, who exactly is landing them?

Today, we have new data that helps us understand where are the tech jobs and that tells us we need more trained people in the most profitable and rewarding areas of tech.

The first-ever Linux Jobs Report released today surfaces two of the most lucrative areas in the tech jobs market - Linux development and Linux systems administration. Eighty-one percent of recruiters surveyed for the report say hiring Linux talent is a priority in the year ahead. And, 63% percent will hire Linux talent over candidates with other skill areas.

A NYU Professor recently said "code is the literacy of the future" (CNN: Computer Geeks King in Job Hunt). We agree. And, we believe that Linux is an important currency in that future. It powers the Internet. It runs Google, Facebook, Amazon, Twitter and millions of other Internet technology companies. It runs the world's supercomputers, data centers, smartphones, financial institutions and stock exchanges, and the list goes on. It's no surprise that with its widespread ubiquity that today there is also growing demand for talent to support it. In fact, when the Linux Jobs Report survey respondents were asked why hiring Linux talent was a priority in the year ahead, most reported their companies are growing, increasing their use of Linux and requiring in-house expertise to support the OS.

But the Linux Jobs Report also finds a wrinkle in an otherwise positive story: Linux and open source developers can be hard to find. Eight-five percent of those surveyed say that finding Linux talent is really difficult.

Google, Facebook, Amazon, Qualcomm, IBM, Intel and hundreds of other companies who rely on Linux to support their businesses, especially their highly-valued data centers and embedded systems, are paying big bucks to find and retain Linux talent. The Linux Jobs Report shows that nearly 1/3 of the companies surveyed are giving pay increases to these professionals that are above the industry norm. Dice's 2012 Salary Report backs this up, showing that Linux professionals have seen salaries go up by 5% over the last few years, while tech professionals overall have seen just a 2% increase. The 15% bump in bonus payouts to Linux professionals just solidifies the point.

It's become glaringly obvious that students and mid-level career professionals who can confidently write Linux code can also write their own ticket to long-term job security. It’s a really good time to know Linux.

Getting involved in open source projects and understanding the open development model are more important than ever, and the good news is that the “University of Open Source” is open to everyone. There are no entrance exams, no admissions counselors, and no student loans; all you need is a connection to the Internet. And, it doesn't matter where you live or what your local economy is dictating. In a world that is flat, Linux and open source software development is a global opportunity for job seekers working anywhere, any time.

Looking for a place to start? Check out Jon Corbet’s guide on participating in the Linux kernel community. We also invite you to check out our Linux training courses, which are taught by leaders from the Linux and Linux kernel communities.

Get all the results from the Linux Jobs Survey and Report by downloading it here: http://www.linuxfoundation.org/publications/linux-foundation/2012-linux-jobs-report

Subscribe to this comment's feed

Show/hide comments

Show/hide comment form

What's New in LibreOffice 3.5

The LibreOffice 3.5 release is due out shortly, and this release comes with a number of improvements that free office suite users will find useful. From grammar checking to better importing for Microsoft Office documents, LibreOffice 3.5 contains a number of useful improvements. This release also contains preliminary work for porting LibreOffice to the Web and mobile devices.

A quick reminder about LibreOffice releases. The LibreOffice folks maintain several branches that are meant for different use cases. The .0 releases (like 3.5.0 here) are meant for users that are happy to get new features fast, with the understanding that there may still be bugs. The bugs should not be show-stoppers, but may not be suitable for deploying for production use in a business. The .1 release is considered "stable," and the .2 release is "very stable" and .3 or later are dubbed "rock solid." See the post about the release cycles on the Document Foundation blog for more.

The upshot here is that you should only jump on the 3.5.0 release if you're willing to accept that it could have bugs that might impact your daily work. So far, I haven't run into any major issues with .0 releases, but I still would recommend that users think twice before doing irreplaceable work in a .0 release. Save early, and save often in any case.

New in Writer

Naturally, the first LibreOffice component I check out is Writer. Looking over the release notes you'll find a bunch of new goodies in Writer. Let's take a look at some of the most interesting.

The LightProof grammar checker extension is now bundled with LibreOffice 3.5. It has checks for improper punctuation, mixing up "a" and "an," word duplication, and more. It's not as extensive as the grammar checker in Microsoft Word (for example), but it's also less prone to false positives.

Want to add to the existing checks? You can install the LightProof editor and compile or modify rules. This is a bit rough at the moment, but should be improved in future releases. (Also, how many folks actively edit grammar rules? Not too many, I suspect.) In the meantime, if we could just ask that folks learn the difference between "its" and "it's" and "then" and "than," that'd be lovely.

Aside from trying to check users' grammar, the 3.5 release includes a much improved header/footer editing system. Instead of having to dig through several menus to find the header editor, just click on the header (or footer, if that's what you want to edit). Simple, right?

The 3.5 release also contains a number of improvements for font hinting, has a better page break indicator, updates the word count dialog as you type, and much more. Sadly, 3.5 does not bring a "write this TPS report for me" feature, but I hear the AI division of the Document Foundation is making great strides on that one.

New in Calc

If spreadsheets are your thing, there's a few goodies in Calc that may pique your interest.

LibreOffice now gives users the ability to specify the number of worksheets in a new document, so if the standard three is insufficient or over-generous, you can tweak that. I don't think I've ever needed more than 10 worksheets in Calc, but if you're really into huge spreadsheets, Calc now supports up to 10,000 sheets.

Calc now has a multi-line input bar, so if you're working on putting a longer piece of text into a spreadsheet or working on a really long equation, this will be much easier. If you don't know it's there, it's not terribly obvious. At the end of the formula bar, there's a little arrow icon. Just click that to expand the input bar.

If you use the Conditional Formatting feature, you'll be pleased to know that you can now set an unlimited number of rules. (I believe the previous limit was three.)

The 3.5 release of Calc also includes a number of new Calc functions, and a number of performance improvements.

New in Impress

One of the first things you notice in Impress is that you go straight to a new presentation rather than the wizard to select a template. I'm a fan of this change. If you want the wizard, though, you can still go to File -> Wizards -> Presentation. Win all the way around.

If you're importing a newer PowerPoint document (.pptx), you'll see that the custom shapes import is much improved. See the before and after on the wiki, or try out the example presentation. Likewise, the import for Smart Art from PowerPoint has been improved significantly.

One of the things I've noticed in prior releases of Impress is that if you try to export a presentation to PDF with some of the slides hidden, it doesn't work. This can be a bit of a PITA if you're hoping to, say, export a presentation or part of one for a third party that doesn't need to see the entire presentation. Now Impress allows hidden slides to stay hidden when you export them.

The Future!

What's really interesting in the 3.5 release isn't really ready for prime time. At FOSDEM, Micheal Meeks talked about LibreOffice and (among other things) the work that's going on for LibreOffice mobile and Web. Meeks demonstrated the Web front-end which can render into an HTML5 canvas in the browser.

To be sure, this is not ready for prime time. Meeks' demo at FOSDEM crashed a bit and was not as snappy as running LibreOffice on the desktop. But this may be a good way forward for folks that want to run LibreOffice in the browser. Why not a complete re-write? Meeks talked about this at FOSDEM, and also in the interview that he did just before. Basically, re-writes don't go so well. Says Meeks:

"My personal feeling is that doing the last 30% of a WYSIWYG web office suite takes 95% of the time – you see, the first bits (that the browser supports natively) are really rather easy. After that, you get to write a huge mound of JavaScript, which you distribute and run under a JIT in several not-very-compatible browsers. Of course, we can always abandon WYSIWYG, perhaps printing onto pages is really truly dead, but I'm not so sure that crisp, repeatable paginated layout is really truly gone for good yet. That's particularly true for drawings, posters, marketing etc."

He went on to talk about some of the spectacular failures of re-writes, like Corel's attempt to re-write its office suite in Java and Micropro's re-write of WordStar. Both failed completely. Instead, Meeks says that he'd like to "focus on a single product, with substantial source-code re-use for different segments."

The mobile app is also in development, and Meeks says that he's "optimistic that we can get something useful on-line for the end of 2012 or early 2013, but that of course depends on community involvement and help." The Android port should start with a document viewer, "evolving over time to allow editing."

If you're the type of person who loves to ride the cutting edge, I'd recommend grabbing 3.5 as soon as its out. If not, just hang tight for the 3.5.1 release or 3.5.2 if you're feeling really cautious. The next (3.5.1) release is tentatively scheduled for around March 5th, and 3.5.2 is tentatively scheduled for April 2nd through April 8th. The feature freeze for 3.6.0 is scheduled for the week of June 4th with the final release the week of July 30th.

Make Differences


Charities


Articles


Asbestos Deaths


Mesothelioma


Mesothelium


Deaths from Asbestos


Mesothelioma Advice


Mesothelioma Resources


Exposure to Asbestos


Auto Donation California


Car Donation Centers


Car Donation San Francisco


Vehicle Donation California