Multi-Language Planet Sysadmin

DIY photography speed strap

2008-09-08T03:34:33+00:00

To obtain certain lighting effects, you sometimes need to attach things to your flash head, such as gel filters or bounce cards. Normally, this involves fussing around trying to strap things down with a rubber band or opting to gum up your equipment with sticky-backed velcro or tape. Instead you can create a cheap speed strap that's both easily removable and simple to attach things to.

There are commercial solutions for this that can be a bit pricey, but Jake O'Connell found a way to make one on the cheap for about $3. Velcro came out with something called a "Design Strap", which is basically a zip-tie with Velcro on one side. Put a rubber band around your flash and then wrap a couple Design Straps over that and you have an attachment system that stays put, comes off easy, and is simple to attach things to. Just add velcro to your gels, bounce cards or whatever and you can quickly affix them to the flash however you like.

DIY 3 Dollar 10 Second Speed Strap

More Examples of Windows Multimedia Redirection

2008-09-08T03:28:23+00:00

In my last entry, "CL" posted a comment wanting more to see more examples of Windows media redirection. Here's another video showing a variety of clips. Talking heads, animation, a couple of VC-1 live streams. Enjoy. Still have the dog barking at the tripod problem, so I apologize for some of the shaky parts. Take a Dramamine if you get queasy. If you want to watch this in a bit better quality, follow this link and click on the "watch in high quality" link once you get into YouTube (right under the "Views" number).

Fanning the Winds of Change in Storage

2008-09-08T03:20:32+00:00

It's been over a month (and three hurricanes in America) since I've posted a blog. More than a few of you've noticed - thanks for the prodding...

It's been a busy summer, on nearly every front. Customer activity hasn't slowed down, and the good news surrounding the (otherwise unfortunate) economic crisis embroiling many customers (especially those in the financial services industry, a heavy concentration for Sun) is that it's whipping up the winds of change. Customers facing spending pressure, or tiring of vendor price increases have new options, and there's a new appetite to explore those options (nothing like mandates from the CEO to reduce spending by 50%).

One of my more interesting recent meetings wasn't with a customer, though, it was with an equity analyst from a global financial institution. Equity analysts publish research that feeds the investment community - their (free) research and financial analysis accompanies buy/hold/sell recommendations to investors (who hopefully generate trading fees for the analyst's employers).

This one analyst hadn't historically followed Sun, and was in the process of developing his first rating. He wanted to focus on our storage plans - more and more of the customers whom he interviewed were focused on storage, and many were talking up a specific open source software technology: ZFS. (Before meeting with me, he'd talked to colleagues in his own IT shop, and was impressed to find some who admitted to running ZFS at home - nothing like touching your customers where they live... if you'd like to have ZFS sent to you, click here or on the LiveCD shown at right.)

Granted, you can see an increasing focus on storage at Sun - the acquisition of MySQL is as much a storage acquisition, as an enhancement to Sun's developer offerings. Discussions of flash memory, the economics of archiving, the Lustre parallel file system, all point to an increasing focus on what Sun sees as an exceptional opportunity for customers (and thus, investors). Storage and computing are converging - and we're about to bring the trends that transformed the server industry a few years ago (mass engagement in open development communities, and scale achieved via clusters of commodity parts vs. proprietary technologies) to the historically closed and proprietary storage industry.

Now, the notion of "engaging customers in open development communities" doesn't sit well among some traditional storage analysts (or our competition) who believe "Storage is too mission critical to tolerate open source software." Although I appreciate that wisdom and experience, I think the market's more nuanced than that - mission critical environments don't tolerate unsupported software, true, which is why we offer 24x7 commercial support for ZFS (on Sun hardware, and Dell, even). But broad global adoption of key open source projects will continue to drive change deep into the world's datacenters. Gartner's prediction that 90% of world's companies will run open source software didn't specify where they'd be running it - "everywhere" is the safest bet.

But back to the equity analyst - he patiently asked, "Great theory, but when will you see revenue results?"

"Last year," I responded. "You're seeing it accelerate."

As many folks know, we shipped our first ZFS based storage systems in 2007 - known as Thumpers. Thumpers finished up this last year generating around $100m in billings, up 80% year over year. From a capacity perspective, we delivered roughly 90 petabytes of Thumper storage in FY2008, to some of the most demanding storage installations on earth (up ~200% y/y). What's fueling the growth? Adoption of ZFS is a clear driver (this chart gives you a sense of where we're seeing adoption - thus revenue opportunity). But ultimately, customers are recognizing they can save money, space and power. Thumpers are roughly twice the capacity in half the space at half the cost of the competition - $1.20/Gigabyte. (They also run Windows and Linux with the same hardware economics).

Now, our view is "OpenStorage" (systems built from commodity parts and open source software) will grow far faster than the proprietary storage market. We plan on driving that growth, and over the next few months, you'll see a tremendous amount of storage innovation targeting the growing breadth of customers wanting better/faster/cheaper/smaller options. Expect to see flash, zfs, dtrace, and good old fashioned systems engineering play a very prominent role in an aggressive push into the storage market.

And in case you missed our announcement last week, our progress was validated by industry analysis - IDC said customers are growing their disk storage business with Sun far faster than with any of our proprietary competition. And at three times the rate of the overall market's growth. A great place to start.

If you'd like to know more, and might be interested in taking a Thumper system for a free trial run, just click here and pick the country in which you're located. We supply most systems at Sun for free trials across the globe (yes, we even cover shipping to you). If you like the system, please buy it. If not, we'll take care of getting it returned to Sun, you owe us nothing. (That's the closest we can get to free hardware downloads...)

As I said to the analyst, you need only look to the results we're already delivering to see the linkage between open innovation and revenue growth. ZFS won't transform demand for our legacy products, but it'll certainly transform the opportunity and industry unfolding before us. But don't just get our opinion, the best folks to validate our approach aren't at Sun, they're among the storage buyers finally feeling the winds of change - at their backs.

LVM on AIX: Extending a Filesystem

2008-09-07T23:16:05+00:00

[Prologue] SysAdmin received this email, looks like the request has been escalated to my boss. L1/L2, Could you please assist Matt ASAP! This is pertaining to UNIX acct so we need to iron out. The SC is 14270227. Make this high priority please. Thanks Issue root filesystem (/) is currently 100% full. Quick Response Team require free disk space to perform some activities [...]

Install google chrome with wine in Ubuntu

2008-09-07T23:03:46+00:00

Google Chrome is an open source web browser developed by Google. The name is derived from the graphical user interface frame, or “chrome”, of web browsers.Chromium is the name of the open source project behind Google Chrome,released under the BSD license.

(...)
Read the rest of Install google chrome with wine in Ubuntu (329 words)

© admin for Ubuntu Geek, 2008. | Permalink | No comment | Add to del.icio.us digg
Who's linking ? Technorati BlogPulse Google
Want more on these topics ? Browse the archive of posts filed under General.

Making terms of service clearer

2008-09-07T22:56:51+00:00

Last week's launch of Google Chrome generated some discussion over the legal language in our new browser's terms of service (TOS). As we noted in a subsequent post on Google Chrome's terms of service:

"... Under copyright law, Google needs what's called a "license" to display or transmit content. So to show a blog, we ask the user to give us a license to the blog's content. (The same goes for any other service where users can create content.) But in all these cases, the license is limited to providing the service."

We've also some seen discussion on a few blogs about how our universal terms of service apply to other products, with some users worried that Google is trying to claim ownership of the content they generate. To be clear: our terms do not claim ownership of your content -- what you create is yours and remains yours. But in lawyer-speak, we need to ask for a 'license' (which basically means your permission) to display this content to the wider world when that's what you intend. This issue is not unique to Google; it applies to lots of other Internet companies that display and transmit user content. You can see some other terms of service here from Amazon, eBay, and Facebook .

In some of our products, such as Gmail and Google Docs, we have included additional terms to make it clear that we do not claim ownership of the content. But even without those additional clarifications, we still wouldn't be claiming ownership of your content -- just a license that gives us your permission to use the content to provide the service. The additional terms are there to reassure our users that they still own their own content, even after giving us the permission we need to help them share and collaborate with others, whether via Gmail, Blogger, YouTube, Google Docs, or other services.

Because, in the end, that's what's most important: making sure you're comfortable using our services to share, publish, and store your stuff. We'll continue to look at our terms of service to make them as clear and user-friendly as possible, because at the end of the day if you're not comfortable, our products won't succeed -- and we know it.

Posted by Mike Yang, Senior Product Counsel

Bob Payne’s Podcast Posted

2008-09-07T19:29:54+00:00

Bob Payne interviewed me at Agile 2008. We spoke about my initial plans for Agile 2009, and my (in-writing) project portfolio book. The link is here: Agile 2009 - Johanna Rothman - Agile Portfolio Management and Agile 2009. I had a blast with Bob.

If you’re wondering why it sounds like I’m chewing my cud (!), it’s because I was shivering. I was wearing a nice shirt and a nice jacket, but it was so cold in the area Bob had set up for the interviews, my teeth were chattering and my mouth was dry because I was so cold. Next year, I will bring my fleece, and possibly a scarf and gloves.

eee pc

2008-09-07T20:03:50+00:00

I got an Asus eee pc (model 900) the other day (I ordered it through buy.com). I'm going to a conference next week (zendcon), and I didn't want to lug around my laptop (it gets pretty heavy after a few hours). I didn't need much--something with a Web browser, an ssh client, something to pull images off of my camera (a Kodak Easyshare v1003), and a text editor for taking notes. The eee pc fits that nicely, and it only weighs about 2.5 pounds.

It's got an 8" screen which I'm finding to be perfectly readable. The keyboard is pretty small, but I'm getting used to it (I'm posting this from the eee pc).

It has an SD/MMC card reader, 3 USB 2.0 ports, and built-in wireless (802.11 g/b). It has 1GB of RAM and a 20GB solid-state hard drive (the thing boots to the login prompt in about 30 seconds). It even has an integrated 1.3Mpixel Web cam.

I'm finding the eeeuser wiki to be very helpful (especially a post about getting gwenview to import photos from my camera).

One little wrinkle I experienced was that the wireless adapter was inadvertently disabled, and it took me a little while to figure out what had happened. The eee pc typically recovers well from going to standby mode, but once when I tried to resume, the screen looked like static and I couldn't get it to do anything. I had to hold down the power button and reboot. When it came back, it refused to join my wireless network. I finally figured out (with the help of the diagnostic tools) that the wireless adapter had been disabled (dunno if I accidentally did that, or if it was a fluke). Anyway, holding down the special function key ("Fn" in the lower left) and hitting F2 toggles the adapter, and I was back in business.

How to setup Boot Password (Grub)

2008-09-07T18:43:33+00:00

Even before the operating system is booted, GRUB enables access to file systems. Users without root permissions can access files in your Linux system to which they have no access once the system is booted. To block this kind of access or prevent users from booting certain operating systems, set a boot password. As the user root, proceed as follows to set a boot password:

feedback to php|arch article

2008-09-07T19:22:16+00:00

I noticed a couple of blog posts written in response to my (somewhat) recent php|arch article:

Reading these was initially discouraging, as both authors are critical of the concept of my article. Although I don't necessarily agree with them, they both make good points in their posts, and I encourage you to read them both if you are interested in the topic. I particularly appreciate the point about table-level locking in MyISAM tables, and how that might affect performance in the frequently-updated value tables.

Although this is not a very useful rebuttal, about all I can say is that the EAV method has worked well for me. As with anything, your mileage may vary. I'm not running reddit or facebook, and the systems I've built on EAV don't have thousands of simultaneous users. So I really can't say how well it would perform in a large-scale deployment.

After I sort of got over being defensive about the whole thing, I'm just glad that people found the article interesting enough to talk about it.

Bad Baby Names: Special Sabbath Edition

2008-09-07T18:12:50+00:00

For reasons unknown to me, the Trib published the vital statistics column today, a Sunday, instead of on its regular Monday schedule. That means you don’t get to start the work week with Bad Baby Names. Instead you get to leisure over them during a lazy Sunday!

Marysa and Shaylyn are the traditional Y-attack names. While bad, these are getting to be a little old and tired.

Luckily Xayton was born to come wake us up. Xayton sounds like a name from the future, possibly the name of an evil robot overlord. In that sense it’s a cool name, but it’s no name I’d want to saddle a little boy with.

The emotional favourites for today don’t really have a bad name, they’re just appropriate for the day. Two sets of parents named their kids Angel. One boy, one girl. Isn’t that nauseating?

But the winner is Ja’miesha. Complete with apostrophe. Jamiesha isn’t even a good name on its own, but throwing in that apostrophe brings things to a whole new level. In Hawaii the
http://en.wikipedia.org/wiki/Okina">
http://en.wikipedia.org/wiki/Okina">
http://en.wikipedia.org/wiki/Okina">ʻokina is used to represent a glottal stop. Does this mean that we’re supposed to pronounce Ja’miesha with a glottal stop in there? Or were the parents just idiots and thought that their little girl’s name would look pretty with an apostrophe (it’s not even an ʻokina because Jamiesha isn’t anywhere near Hawaiian)? I vote the latter.

Tags: bad baby names

SVG via CSS

2008-09-07T15:12:29+00:00

Now that I have my weblog looking reasonably consistent between Gecko and WebKit based browsers, I’ve taken another look at Opera. Opera doesn’t have support for border-radius, but does have support for background images in SVG, which can be used to provide the same effect. My Nav Bar on my test site now employs this technique, and it requires two separate images: 039 on CCD and CCD on FFF.

More complex effects are also possible, with only slightly more work. I’ve applied some of those techniques here.

Frankly, my first reaction to this was mixed. No two ways about it: it requires more work and more page fetches to produce the same result as could be done in CSS. This might be slightly better if one could somehow embed the SVG in the CSS file itself, as many of the SVG files are engineered for a single purpose. This lead me to initially think that having CSS continue to capture the common cases, leaving comparatively advanced techniques like SVG for special purpose or complex effects was the right way to go.

But then I saw this (screenshot) and even (&deity; forbid) this (screenshot), and it occurs to me that SVG opens the door to unubtrusive special purpose and (when judiciously employed) not-so-complex effects. It would be fairly easy to give my Nav Bar a more clearly defined frame, a more tapered shape, and use filter effects to vary the color.

The pluses for SVG in CSS is that it doesn’t require either adjusting your markup or JavaScript to achieve these effects, a desirable characteristic that generally the other techniques don’t share. Nor does it require that your page be XHTML. And there are lots of good techniques for effectively caching static files. The only remaining issue is that this technique works best for backgrounds instead of borders, as backgrounds images (if present) are displayed on top of the background color, making the background color an effective fallback for browsers that don’t support SVG in CSS. Of course, the background can draw a border within the padding, but any CSS provided borders would still show up outside of the background.

Meanwhile, Robert O’Callahan has been exploring other ways to integrate these technologies.

Write a Hadoop MapReduce job in any programming language

2008-09-07T05:58:52+00:00

Hadoop is a Java-based distributed application and storage framework that's designed to run on thousands of commodity machines. You can think of it as an open source approximation of Google's search infrastructure. Yahoo!, in fact, runs many components of its search and ad products on Hadoop, and it's not too surprising that they are a major contributor to the project.

MapReduce is a method for writing software that can be parallelized across thousands of machines to process enormous amounts of data. For instance, let's say you want to count the number of referrals, by domain, in all the world's Apache server logs. Here's the gist of how you'd do it:

Get all the world to upload their server logs to your gigantor distributed file system. You might automate and approximate this by having every web administrator add some javascript code to their site that causes their visitor's browsers to ping your own server, resulting in one giant log file of all the world's server logs. Your filesystem of choice is HDFS, the Hadoop Distributed Filesystem, which handles partitioning and replicating this enormous file between all of your cluster nodes.
Split the world's largest log file into tiny pieces, and have your thousands of cluster machines parse the pieces, looking for referrers. This is the "Map" phase. Each chunk is processed and the referrers found in that chunk are output back to the system, which stores the output keyed by the referrer hostname. The chunk assignments are optimized so that the cluster nodes will process chunks of data that happen to be stored on their local fragment of the distributed file system.
Finally, all the outputs from the Map phase are collated. This is called the "Reduce" phase. The cluster nodes are assigned a hostname key that was created during the Map phase. All of the outputs for that key are read in by the node and counted. The node then outputs a single result which is the domain name of the referrer, and the total number of referrals that were produced from that referrer. This is done hundreds of thousands of times, once for each referrer domain, and distributed across the thousands of cluster nodes.

At the end of this hypothetical MapReduce job, you're left with a concise list of each domain that's referred traffic, and a count of how many referrals it's given. What's cool about Hadoop and MapReduce is that it makes writing distributed applications like this surprisingly simple. The two functions to perform the example referrer parsing might only be about 20 lines of code. Hadoop takes care of the immense challenges of distributed storage and processing, letting you focus on your specific task.

Since Hadoop is written in Java, the natural way for you to create distributed jobs is to encapsulate your Map and Reduce functions into a java class. If you're not a Java junkie, though, don't worry, there's a job wrapper called HadoopStreaming which can communicate with any program you write with the usual STDIN and STDOUT. This lets you write your distributed job in Perl, Python or even a shell script! You create two programs, one for the mapper and one for the reducer, and HadoopStreaming handles uploading them to all of the cluster nodes and passing data to and from your programs.

If you want to play around with this, I really recommend a couple of howtos written by German hacker Michael G. Noll. He put together a walkthrough for getting Hadoop up and running on Ubuntu, and also a nice introduction to writing a MapReduce program using HadoopStreaming (with Python as an example).

Are any Hackszine readers using Hadoop? Let us know what you're doing and point us to more information in the comments.

Hadoop
Running Hadoop On Ubuntu Linux
Writing An Hadoop MapReduce Program In Python

Scobleizer — Tech geek blogger » Blog Archive The Superbowl of Startups «

2008-09-07T04:54:05+00:00

Scobleizer — Tech geek blogger » Blog Archive The Superbowl of Startups «:

"Blogging is NOT reporting. It’s the single voice of a person. When you read me here you are reading me the way I’d talk to you at a cocktail party. You’re hearing my opinions. If I’m doing ‘reporting’ then you’ll know, because of how I source it."

I really like this explanation for blogging. Maybe we need "Blogging is NOT reporting" T-shirts.

I Am Trying To Believe (that Rock Stars aren't Dead)

2008-09-07T02:50:59+00:00

Last Friday night I attended a Nine Inch Nails concert in Philadelphia with Chris Cera of Vuzit (thanks Chris for your help with this post). At 43, Trent Reznor can certainly still grab an audience by the throat and shake it. It was a fantastic show; the kind of show that has you checking to see if there are other tour dates within driving distance.

During a short break in the sonic and visual mayhem, Reznor spoke for a moment and told us emphatically to steal his music. Later, on my way to the car after the show, a member of the band Cube Head was handing out sharpie-labled home-burned demo CD's in the parking lot complete with a hand drawn "copywrong" marking. It was an interesting contrast between established artist and emerging talent and how they are both figuring out how to make their way in the post-vinyl post-jewel-case economy.

I'll come back to that theme in a second, but first a brief aside. Chris (who has some background in real time video processing) and I were blown away by the amazing stage show; it was geek manifest and a video processing tour de force. During about 1/3 of the show the band played sandwiched between at least two giant video monitors, the one in the foreground transparent when its pixels were dormant and opaque when lit up.

The source video for the display was sometimes heavily processed local camera inputs, sometimes it was prerecorded, and sometimes it was electronically generated. Whatever the source, it was frequently and heavily modified by the audio inputs or by the movements of the artists on the stage. With a sweep of his hand Trent would wave away the static hiding him from the audience and then moments later it would fill back in. It's hard to explain but the effect was very cool. Cool enough that trying to figure it out started to distract both of us from the music. There are some videos out there of it in action but none that I found really capture the full effect. Let me know in the comments if you find one.

-- "Steal my music" --

The next day, still curious about how the stage show was done, and with Reznor's call to "steal my music" still in my head, I poked around on the web looking for more info. One of the most interesting things I found was this story about Nine Inch Nail's Year Zero Alternate Reality Game. The way Reznor used this new gaming medium as an extension of his canvas rather than as a promotional stunt (and the nascent geekness it suggests) makes me think he has a much better than average chance to figure out the post RIAA world. Or, it may just be that with the state of distribution being what it is, he realized that while promotion might move more units, it would do it in a way so loosely coupled to monetization as to be pointless.

His comments in the story's sidebar make me think it is probably the latter. In particular: "So a couple years ago I realized that music essentially is free now. I'd prefer, it wasn't, but it is. And hey, I've had a pretty good run. I can still make a living touring." .... "I feel that the right model hasn't revealed itself yet."

Here's the thing, I'm not convinced it's going to reveal itself. Or, more likely, it has revealed itself and he already knows what it is: "I can still make a living touring."

When it comes to income, I don't think there will be another Rolling Stones any more than I think there will be another Microsoft. Reznor's creative partner Rob Sheridan hints at the same thing in his amazing piece on the state of the music business written last year. He does a masterful job of parsing all the deck chair rearranging going on in the industry today but what he is unable to do is offer a meaningful business model to replace it.

Everyone is fishing for the answer (see item #1) but more and more I think it's just not there. After all "sell records" was not some complex business model come down from on high. I can't help but think that if there was an equally effective replacement someone would have thought of it by now.

That's not to say I don't think new and better music distribution and monetization models won't be invented, I just don't think they will capture and concentrate as much value as the one that is dying before our eyes did. I suspect the balance between linear (touring) and leverage (selling stuff while you sit at home) has simply and irrevocably shifted toward the linear.

-- Leverage is dead. Honey I shrunk the head of the long tail. --

When the Victor Talking Machine Company (their original factory seen here from my office window in Camden, NJ) made it possible for an artist to be in more than one place at the same time, they gave the performing artist the gift of leverage (e.g. Enrico Caruso booming out over the Ucayali River in Werner Herzog's great film Fitzcarraldo. If you haven't seen it, trust me, Netflix it).

The key to the whole system was the low marginal cost of the recorded disk and the important fact that it still served to support a constrained distribution channel. The key constraint being that it enabled point of sale monetization of the entire value chain of production, marketing, and distribution. In fact, it worked so well, that it set up a system where recording companies were able to harvest higher than economic rents for their services and leave all but the most popular artists either on the outside trying to break in or in near-perpetual indentured servitude once they did. The long tail was neither; it was truncated and compressed by the recording company's stranglehold on distribution. This established the long running artist's resentment of their record company's. Despite the compression though, the recording was such a leveragable medium that touring, the artists original source of income, was often reduced to promotion for the disc.

As everyone knows, the digital audio file is even more frictionless than the disk. It is even better at creating duplicate Enrico's and freely distributing them around the world. It provides the performing artist near perfect leverage in their quest for attention; they can be anywhere in the world finding any audience. The musical attention economy has become a near-perfect meritocracy (of course merit is a slippery thing in this world). Unfortunately, and paradoxically, the digital audio file is in a sense too frictionless and it severs the connection between distribution and monetization. It's not contained like the vinyl record or the disc that followed it and, therefore, it effectively snatches the gift of leverage back from the artist.

The value that people might pay for in distribution now (higher bit rates, convenient discovery, etc.) will attract some dollars, but the music loving consumer in aggregate is never going to pay as much for music in the future as they did in the past. Back then they were paying for the binary decision of having music or not. Incremental qualities simply won't monetize at the same rate even if a replacement mechanism to capture it is created.

Value isn't being added in traditional distribution and consumers know it, they aren't going to pay for what they aren't getting. Also, since the long tail of attention is no longer truncated by the music industry's artificial barriers, some of the revenue that does remain in the system is going to shift away from the artists at the head toward those down at the tail. There is going to be less money spent on music and what's there is going to be distributed more broadly.

As an aside, the electric production companies should thank their lucky stars that electricity won't fly through the ether as easily as music moves across a network. Here we are after deregulation and the electric generation companies are still getting paid. Their product is unbundled from transmission, but transmission still makes a nice point for collection and monetization.

For the artist seeking attention this seems like a bittersweet time. Make great music and be discovered. Instead of assailing the gate keepers with demo tapes, find an audience through this new frictionless medium and new alternatives to the traditional marketing machine like Pandora, Last.fm, and various p2p networks. But, when you get the attention, then what?

-- The Rock Star's life style is going to go the way of Sun's multiples --

For the artist trying to make a living welcome back to 1890. Cube Head and Reznor are both wage slaves now and the tour isn't about promotion, it is the product again. The linear / leverage balance is leaning back to linear and if you want a model for how you will make a living, look no further than Adelina Patti. She was a typical opera singer of the 1890's who made her living touring in the last years before the Victor company changed the rules of leverage. She toured all over the world and as she got more popular the venues presumably got bigger. It was artistic piece work but with good pay and decent perks.

By the way, If you're a future Polyphonic Spree with all those mouths to feed in the split, I think the post-leverage music business is going to be especially difficult for you. You might just be screwed.

Leverage is dead and frictionless distribution killed it ("video killed the radio star"... and audio files killed the rock star). Well, maybe leverage isn't dead, but if it's still alive, it's alive the way that guy in the wheel barrow is still alive in The Holy Grail. Picture iTunes in the wheelbarrow and you get the idea. Even if it's not dead, what Reznor is acknowledging is that it has shrunk to the point where it is no longer going to be the driving force in an artist's income.

Reznor has an open mind and is clearly preparing himself for a transition into a post RIAA world, but I can't help but think he's going to find a world that is changing as much for him as it is for the record companies he assails. The music industry is clearly a dead man walking, but what I think I'm realizing is that rock stars as we know them are relics too.

It's the same mechanism that is happening to proprietary software companies. The huge valuations of proprietary software companies are based on the leverage inherent in the proprietary software business model. But when a Red Hat comes along selling services and giving away the IP (to over simplify a bit) they are implementing a "cut the legs out from under the old guard" strategy. They will win, or at least force the old guard to shift models. When it's over though, they'll have a business model that will never achieve the same enterprise value per unit of distribution that the old model did. It's easier now for a software company to find and reach an audience, but the subsequent monetization won't achieve the same multiples on units shipped.

The sad truth is that even with the shift to a services model, the Red Hat's of the world probably preserve more leverage than a recording artist does in this new open IP paradigm. They can always hire more people to provide the services that are the key to monetization in their new model. They don't have to try to clone the software's original developer.

However, unless all of us music fans start liking NiN cover bands as much as the real thing, Reznor's leverage in an era where touring is his primary source of income is near zero. He's not the infinitely duplicatable Blue Man Group, he's a modern opera star whose income will be capped by the number of seats in the auditorium and the number of dates he's willing to do on the road (On the other hand, it would be hilarious if he franchised the NiN concept and sent cover bands to small clubs all over the place).

So, what does all of this mean for Trent and Cube Head? I don't really know as this whole post is mostly conjecture. What I do know is that Trent has already made his money so he'll be just fine. Touring is where his art is expressed and he can still get paid doing it so life should be good. He'll probably be paying his taxes with tour income while he's lifestyling off the golden egg the old system laid right before it went to goose heaven.

If Cube Head is good enough, they might just take advantage of frictionless distribution to ride up a more meritocratic power curve in the attention economy. If they are great, they might even find themselves at the head of the curve. When they get there though, they'll likely find themselves on a hill instead of the mountain they remember from all those episodes of MTV Cribs. Oh, and they'll probably be touring their asses off. Pretty much exactly like that 19th century opera star. I really don't think they'll be sitting at home between tours watching the point of sale scan numbers and cashing their royalty checks.

With the disaggregated production -> marketing -> distribution channel value chain I have absolutely no idea how Timbaland is going to get his pay on. That remains a mystery.

By the way, if I'm right and Trent Reznor is a bit of a closet geek, I bet he also has Google alerts set up for himself. So, just in case,... Hey Trent, awesome show the other night. Absolutely kick ass. And by the way, I bought tickets to your show right after I downloaded The Slip.

Slightly Advanced Python: Some Python Internals

2008-09-07T00:09:33+00:00

via Youtube

The Python Object Model

2008-09-07T00:09:19+00:00

via Youtube

Puppet - Centralised configuration management for networks

2008-09-06T23:04:02+00:00

Puppet lets you centrally manage every important aspect of your system using a cross-platform specification language that manages all the separate elements normally aggregated in different files, like users, cron jobs, and hosts, along with obviously discrete elements like packages, services, and files.

(...)
Read the rest of Puppet - Centralised configuration management for networks (106 words)

Add to del.icio.us

Search blogs linking this post with Technorati

Want more on these topics ? Browse the archive of posts filed under Free Tools.

---
Related Articles at Debian Admin:

Aria2 - high speed command line download utility in Debian

Opsview - Enterprise network and Application monitoring based on Nagios

rinetd - Internet TCP redirection server

sshpass - Non-interactive ssh password authentication

autossh - Automatically restart SSH sessions and tunnels

Howto Determine the throughput of a pipe command

My First Day with Python - Initial Thoughts

2008-09-06T22:44:25+00:00

While I've always been a bit of a perl guy I don't want this post to be "perl has x and python doesn't" in tone. Which is lucky really as Python has exceptions and threading as first class features where as perl has... ahem.

So after spending a chunk of today reading a python book and spending some time writing code here's my initial short list of gripes -

except IOError
print adding newlines
Significance of whitespace in blocks. But not like that.
The lack of ++

Considering how picky I can be that's a very short list so Python must sit well with me so far. Now, in order, I can't help but read except IOError as 'catch everything apart from IOError'. This one bugs me more than it should but considering how happy native exceptions in the language made me this just felt mean.

Secondly, print adding newlines. While this might seem trivial every other language I use on a daily basis has a print function that doesn't print a newline so this feels weird. At least it's not called say ;)

Now to the one that I'll get no sympathy on - whitespace in blocks. First up let me say I don't mind about the enforced indentation. I indent anyway so it's not a big deal. I guess I'll hit the odd case when it annoys me (probably involving heredocs) but I've got nothing against it. What does irk me is the lack of block delimiters - whitespace just doesn't cut it for me.

I like my { and } delimited blocks, a nasty voice in my head is telling me to add them but just comment them out ( if x == y: # { ) but that seems very wrong. I've always looked at those examples in C programming books that say...


# incorrect
if ( something )
  print("All's well");
  wellness++;

# this is wrong because wellness is a separate statement
# and not part of the if

... and thought - "just add the damn braces, you'll be back to add more code later anyway." Now I'm learning a language that seems to want me to slip up like this. I'll either get used to this or move to ruby.

Lastly we have the lack of ++ and --. I know the arguments, I've read them before. I disagree. I've never done anything insane with ++ and where I have used it it's saved me typing. Can we have ++ and remove nested ternary ( ? : ) instead please?

I like Python and I think I'll be investing more time in to learning it.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

JavaScript Benchmark Quality

2008-09-06T19:55:15+00:00

Summary: JavaScript Benchmarks aren't adapting well to the rapid increase in JavaScript engine performance. I provide some simple tests for verifying this and propose a modified setup which could be used by all JavaScript Benchmarks to achieve high-quality results.

There now exists three, what I would consider to be, major JavaScript performance benchmarks. Each are released by a major browser vendor. WebKit released SunSpider, Mozilla released Dromaeo, Google released the V8 Benchmark.

Each suite has a variety of tests and a test runner. I'm currently interested in one thing: The quality of the test runner that each of these suites provides.

There are three points that any test runner tries to achieve:

Retrieving accurate numbers. Another way to phrase it: "Retrieving stable numbers." If you run the test suite multiple times will you get identical, or near-identical, numbers?
Reducing the possible error. Does the suite attempt to quantify how much possible error there could be in their results? How large is the error?
Reducing run time. How long does it take to run each test?

The ideal suite would be one that's capable of running an individual test as quickly and accurately as possible with virtually no error. However, in order to get those numbers you need to carefully chose what style of tests you wish to run.

I quantify the current field of tests into three categories:

Slow-running tests: These tests generally take a couple hundred milliseconds on an average consumer machine. This is the style of tests that you generally see in SunSpider and occasionally in Dromaeo. These tests have a distinct advantage: They are generally quite accurate. You don't need to run them very many times (say about 5) in order to get a consistent picture of how the test will run.

Moderate-running tests: These tests take less than one hundred milliseconds. You'll see these tests in Dromaeo. These tests need to be run quite a few times (20-30) in order to get a better picture of their speed.

Fast-running tests: These tests take less than ten milliseconds. You'll see these tests in the V8 Benchmark and in Dromaeo. These tests must be run many, many, times (usually close to a thousand) in order to weed out any possible error. If you were to run one 10ms test 5 times - and get a single result of 11ms that would introduce a significant level of error into your results. Consider tests that take 0-1ms. A deviation within that range can instantly cause error levels around 50% to occur, if enough test iterations aren't completed.

Looking at the above categories the solution seems obvious: Use slow-running tests! You get to run them fewer times and you get accurate results - everything's peachy! But here's the problem: The speed of JavaScript engines are increasing at a rate faster than test suites can adapt. For example, SunSpider was initially developed with all tests running at equal speeds on a modern computer. Now that speed improvements have come along, though, (WebKit improvements, then Mozilla, then SquirrelFish, then TraceMonkey, then V8) those results don't even remotely resemble the tests of old. Most of the results have moved down into the moderate-running range of tests, some even into the fast-running range - but here's the problem: They're still only running the originally-designed number of loops. An example of the difference:

This means that a browser is running a test for 5-10 loops (in both SunSpider or Dromaeo) but the speed of the test no longer matches that assigned number of iterations. At first glance you could say one of two things: 1) Increase the difficulty of the tests or 2) Have them run for more iterations. There are problems with this, though.

While you certainly could increase the complexity of existing tests the result would be a never-ending battle. Improvements would have to land every couple months in order to keep up with the pace of improvement. This would work ok in Dromaeo (since it has versioned tests) but not all suites can handle this behavior. Additionally this now makes the tests less-useful for measuring slower browsers (the tests now take an absurd amount of time to complete as opposed to the very-plausible numbers from before).

Additionally, you could increase the number of test iterations that would occur but not without assigning specific iteration counts to each individual tests. And this is the full problem: How do you know what numbers to choose? Raising the number to 20 iterations may help one browser - but what about another browser which will need 100 iterations to get a proper count?

This leaves us in a bind. Browsers keep getting faster at tests, test suites do the wrong number of iterations, causing the error level to continually increase:

We should take a step back and look at what the test suites are doing to counter-act the above trend from happening - if anything at all.

SunSpider was originally dominated by long-running tests running 5 times each. The tests use to be long-running but are now only in to the medium to fast-running range (depending on the browser). This has caused the accuracy to decrease and error level to increase. Increasing the number of iterations would help (but hinder older browser performance).

Dromaeo has a range of tests (fast, moderate, and long-running) each running 5-10 times each. Dromaeo attempts to correct the number of iterations run, right now, but kind of fails when doing so. It looks at the results of past iterations (especially the error level generated by the results) and decides to run more tests until a stable error level is achieved. The problem with this is the samples are no longer being independently determined. Whereas test runs 1-5 were independent, test runs 6-10 were not (they're only being run due to the fact that previous test runs provided poor results). So while the results from Dromaeo are hyper-stable (they're the most stable performance test that we run at Mozilla) they're not determined in a proper statistical manner. Thus Dromaeo needs to be changed in order for people to be able to gather accurate results without sacrificing its statistical integrity.

The V8 Benchmark takes a completely different strategy for its fast-running tests: Instead of running for a pre-determined number of iterations each test is run continuously until a second of time has passed. This means that individual tests frequently run anywhere from 50-200 times (depending on the machine and browser they run on). Currently the V8 Benchmark does suffer from one shortcoming: There is no error calculation done. Both SunSpider and Dromaeo fit the results to a t-distribution and compute the possible error of the results whereas the V8 Benchmark just leaves them as is.

However, the V8 Benchmark does bring up a very interesting strategy. By picking tests that are simpler (and, arguably, most current complex tests will become "simple" as engines rapidly improve) and running them more times (relative to the complexity of the test) the results become much more stable.

Consider the result: Fast-running tests end up having a smaller error range because they are able to run more within a given allotment of time. This means that the test runner is now self-correcting (adjusting the number of iterations seamlessly). Since all JavaScript engines are getting faster and faster, and complex tests are running in shorter and shorter amounts of time, the only logical conclusion is to treat all tests as if they were fast tests and to run them a variable number of times.

We can test this hypothesis. I've pulled together a simple demo that tests the computational accuracy of the three styles of test (Simple - or Fast-running, Moderate, and Complex - or Slow-running) against the two different types of suites: "Max" style (the style implemented by the V8 Benchmark) and "Fixed" style (the style implemented by SunSpider and Dromaeo). The results are quite interesting:

Fast and moderate-speed tests are incredibly accurate with the max-style of test running. Often their error is no more than 2% (which is really quite acceptable). The quality degrades for the complex tests, but that's ok. Complex tests could be tweaked to consume less resources (thus allowing them to iterate more times and become more accurate).

Right now the accuracy of slow-to-moderate tests in fixed run test suites are suffering from increased rates of error as the engines get faster (as can be seen in the above chart).

The major change that would need to be made to the flexible style of execution (at least the one implemented in the V8 Benchmark) would be some form of error checking and t-distribution fitting. In V8 a series of tests are run over the course of one second. I propose that the resulting number (the total number of tests executed) be saved and the tests are repeatedly run again (perhaps 5 times, or so). The resulting set of 5 numbers (representing, potentially, thousands of test runs) would then be analyzed for quality and error level. I have this measure completely mocked up in my sample test measurement.

The relevant portion of the code is here:

function runTest(name, test, next){
var runs = [], r = 0;

setTimeout(function(){
var start = (new Date).getTime(), diff = 0;

for ( var n = 0; diff < 1000; n++ ) {
test();
diff = (new Date).getTime() - start;
}

runs.push( n );

if ( r++ < 4 )
setTimeout( arguments.callee, 0 );
else {
done(name, runs);
if ( next )
setTimeout( next, 0 );
}
}, 0);
}

Switching to a flexible style of text execution has many benefits:

It'll help to keep the rate of error small for most tests (especially if the tests are tailored to be smaller in nature).
It'll allow the suite to gracefully adapt over time, as engines speed up, without sacrificing the ability to work on old engines.
The maximum number of runs (and, thus, the maximum amount of accuracy) will occur for the engines that are able to complete the tests faster. Since the greatest amount of competition is occurring in these high numbers (as opposed to in older user agents, where there is no progress being made) granting them the most accurate numbers possible makes this ideal.

I plan on prototyping this setup into Dromaeo and releasing it shorting. It'll take much longer to run the full test suite but the result will be quite worth it.

TXS vs DJB - Round One!

2008-09-06T19:05:00+00:00

That's DJB on the left and TXS on the right. I've never seen either of them in the same room at the same time, so there is still a chance they are the same entity. Maybe TXS is out teachng classes, writing software, or doing other academic things when he should be sleeping or maybe DJB is out having TXS' life while he should be sleeping. This could all be one big real life Fight Club scenario (without the fighting). The world may never solve the mystery that is TXS vs DJB.

If you're in the insanely small population that has not seen Fight Club or read the book, then I'm sorry I ruined it for you. Maybe you should get with the modern era if you're going to continue to read my drivel.

I dug that picture of DJB out from archive.org.

Backup data using flyback

2008-09-06T18:43:56+00:00

Apple's Time Machine is a great feature in their OS, and Linux has almost all of the required technology already built in to recreate it. This is a simple GUI to make it easy to use. Installation (Ubuntu/Fedora/Opensuse) To use, make sure you have the following packages installed: Ubuntu: $ sudo apt-get install python python-glade2 python-gnome2 python-sqlite3 python-gconf rsync Redhat/Fedora

Protection from malware using Squid proxy server

2008-09-06T18:43:56+00:00

Malware (for "malicious software") is any program or file that is harmful to a computer user. Thus, malware includes computer viruses, worms, Trojan horses, and also spyware, programming that gathers information about a computer user without permission. Ignoring the threat of malware is one of the most reckless things you can do in today's increasingly hostile computing environment. Malware is

/etc/grub.conf explained

2008-09-06T17:43:56+00:00

The following example shows the structure of a GRUB menu file. The example installation has a Linux boot partition under /dev/sda5, a root partition under /dev/sda7, and a Windows installation under /dev/sda1. gfxmenu (hd0,4)/boot/message color white/blue black/light-gray default 0 timeout 8 title linux root (hd0,4) kernel /boot/vmlinuz root=/dev/sda7 vga=791 resume=/dev/

Internal Security Staff Matters

2008-09-06T17:17:13+00:00

I read Gunter Ollmann's post in the IBM ISS blog with interest today. Gunter is "Director Security Strategy, IBM Internet Security Systems," so he is undoubtedly pro-outsourcing. Here is his argument:

[S]ecurity doesn’t come cheap. While individual security technologies get cheaper as they commoditize, the constant influx of new threats drives the need for new classes of protection and new locations to deploy them...

If you were to examine a typical organizations IT security budget, you’d probably see that the majority of spend isn’t in new appliances or software license renewals, instead it’ll lie in the departments staffing costs...

This is at odds with the way most organizations normally deal with specialized and professional skill requirements... Just about every organization I deal with (including some of the biggest international companies) relies upon external agencies to provide these specialist services and consultancy – as and when required – it’s more cost effective that way.

With that in mind, why are organizations building up their own highly-trained (and expensive) specialist internal security teams? Granted, some of the security technologies being deployed by organizations are relatively complex, but do they really require a Masters degree and CISSP certified experts to babysit them full-time...

Nowadays you can tap in an incredibly broad range of expertise – ranging from hard-core security researchers capable of helping you evaluate the security of new products you’re thinking of buying and deploying throughout your enterprise, through to 24x7 security sentinels; so knowledgeable about the security product you’ve deployed that they’re capable of guaranteeing protection with money-back SLA’s...

Organizations should take a closer look at their security budgets and evaluate whether they’re getting the right value out of their internal teams and whether their skills investment meets the daily need of the business. (emphasis added)

By highlighting the focus on "security products," you can probably predict my response to Gunter's post. Sure, you can get hire experts that may (or may not) be cheaper than internal staff, and they may be smarter in individual products or even defensive tactics, but they are poor with respect to the most critical aspect of modern security: business knowledge. It does not matter if you are the world's greatest packet monkey if you 1) don't know what matters to a business; 2) don't know business systems; 3) don't know what is normal for a business... do I need to continue?

This is the biggest challenge I see for consultants, having been one and having hired them. It's easier to hire a consultant to help configure a security product than it is to figure out if that product is even needed, which to buy, how to get approval and business buy-in, how to support it operationally, and a dozen other decisions.

I agree that certain specialized tasks merit outside support. That list changes from organization to organization. However, beware arguments like Gunter's.

The Analyzer Charged Again

2008-09-06T15:33:53+00:00

I read a name I hadn't seen in years today when I read Kim Zetter's story Israeli Hacker Known as "The Analyzer" Suspected of Hacking Again:

Canadian authorities have announced the arrest of a 29-year-old Israeli named Ehud Tenenbaum whom they believe is the notorious hacker known as "The Analyzer" who, as a teenager in 1998, hacked into unclassified computer systems belonging to NASA, the Pentagon, the Israeli parliament and others.

Tenenbaum and three Canadians were arrested for allegedly hacking the computer system of a Calgary-based financial services company and inflating the value on several pre-paid debit card accounts before withdrawing about CDN $1.8 million (about U.S. $1.7 million) from ATMs in Canada and other countries. The arrests followed a months-long investigation by Canadian police and the U.S. Secret Service.

The Analyzer was the "mastermind" behind Solar Sunrise, one of the original "so easy a Caveman could do it" intrusions -- back in 1998. Solar Sunrise was huge and it was one of several very rude awakenings I remember while serving in the Air Force that decade.

Seeing The Analyzer back in law enforcement custody reminds me of the post I made about Max Ray Butler and somewhat of my post Intruders Selling Security Software. It's all about trust.

Raleigh Convention Center

2008-09-06T11:51:21+00:00

Grand Opening is occurring this weekend. Took a tour yesterday, looks modern, clean, and should attract more businesses into downtown.

The facilities looks to be about a quarter the size of the combined Moscone Center (i.e., North, South, and West combined). I’d wager that the weak link in attracting major (international or even national) conferences is the airport, as Raleigh is neither a major destination nor a national hub.

One unique feature is the shimmer wall which is fun to look both at night and day.

Bejtlich Keynote at 1st ACM Workshop on Network Data Anonymization

2008-09-06T11:30:09+00:00

Brian Trammell and Bill Yurcik were kind enough to ask me to deliver the keynote at the 1st ACM Workshop on Network Data Anonymization (NDA 2008). The one day event takes place 31 October 2008 at George Mason University in northern VA. My talk will discuss the trials and tribulations of OpenPacket.org, and changes planned for the project.