Planet Sysadmin               

          blogs for sysadmins, chosen by sysadmins...
(Click here for multi-language)

September 23, 2014

Ubuntu Geek

Rands in Repose

What Coke Contains

Kevin Ashton on Medium:

Like every other tool, a can of Coke is a product of our world entire and contains inventions that trace all the way back to the origins of our species.

#

by rands at September 23, 2014 02:44 PM

Chris Siebenmann

Go is mostly easy to cross-compile (with notes)

One of the things I like about Go is that it's generally very easy to cross-compile from one OS to another; for instance, I routinely build (64-bit) Solaris binaries from my 64-bit Linux instead of having to maintain a Solaris or OmniOS Go compilation environment (and with it all of the associated things I'd need to get my source code there, like a version of git and so on). However when I dug into the full story in order to write this entry, I discovered that there are some gaps and important details.

So let's start with basic cross compilation, which is the easy and usual bit. This is well covered by eg Dave Cheny's introduction. The really fast version looks like this (I'm going to assume a 64-bit Linux host):

cd /some/where
hg clone https://code.google.com/p/go
cd go/src
./all.bash
export GO386=sse2
GOOS=linux GOARCH=386 ./make.bash --no-clean
GOOS=solaris GOARCH=amd64 ./make.bash --no-clean
GOOS=freebsd GOARCH=386 ./make.bash --no-clean

(See Installing Go from source for a full discussion of these environment variables.)

With this done, we can build some program for multiple architectures (and deploy the result to them with just eg scp):

cd $HOME/src/call
go build -o call
GOARCH=386 go build -o call32
GOOS=solaris GOARCH=amd64 go build -o call.solaris

(Add additional architectures to taste.)

This generally works. I've done it for quite some time with good success; I don't think I've ever had such a cross-compiled binary not work right, including binaries that do network things. But, as they say, there is a fly in the ointment and these cross-compiled binaries are not quite equivalent to true natively compiled Go binaries.

Go cross-compilation has one potentially important limit: on some platforms, Linux included, true native Go binaries that use some packages are dynamically linked into the C runtime shared library and some associated shared libraries through Cgo (see also). On Linux I believe that this is necessary to use the true native implementation of anything that uses NSS; this includes hostname lookup, username and UID lookup, and group lookups. I further believe that this is because the native versions of this use dynamically loaded C shared libraries that are loaded by the internals of GNU libc.

Unfortunately, Cgo does not cross-compile (even if you happen to have a working C cross compiler environment on your host, as far as I know). So if you cross-compile Go programs to such targets, the binaries run but they have to emulate the native approach and the result is not guaranteed to give you identical results. Sometimes it won't work at all; for example os/user is unimplemented if you cross-compile to Linux (and all username or UID lookups will fail).

(One discussion of this is in Alan Shreve's article, which was a very useful source for writing this entry.)

Initially I thought this was no big deal for me but it turns out that it potentially is, because compiling for 32-bit Linux on 64-bit Linux is still cross-compiling (as is going the other way, from 32-bit host to 64-bit target). If you build your Go environment on, say, a 64-bit Ubuntu machine and cross-compile binaries for your 32-bit Ubuntu machines, you're affected by this. The sign of this happening is that ldd will report that you have a static executable instead of a dynamic one. For example, on 64-bit Linux:

; ldd call32 call64
call32:
        not a dynamic executable
call64:
        linux-vdso.so.1 =>  (0x00007ffff2957000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3be5111000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3be4d53000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3be537a000)

If you have both 64-bit and 32-bit Linux and you want to build true native binaries on both, at least as far as the standard packages go, you have to follow the approach from Alan Shreve's article. For me, this goes like the following (assuming that you want your 64-bit Linux machine to be the native version, which you may not):

  1. erase everything from $GOROOT/bin and $GOROOT/pkg
  2. run 'cd src; ./make.bash' on the 32-bit machine
  3. rename $GOROOT/pkg/linux_386 to some other name to preserve it
  4. build everything on your 64-bit machine, including the 32-bit cross-compile environment.
  5. delete the newly created $GOROOT/pkg/linux_386 directory hierarchy and restore the native-built version you saved in step 3.

If you're building from source using the exact same version from the Mercurial repository it appears that you can extend this to copying the pkg/$GOOS_$GOARCH directory between systems. I've tested copying both 32-bit Linux and 64-bit Solaris and it worked for me (and the resulting binaries ran correctly in quick testing). This means that you need to build Go itself on various systems but you can get away with doing all of your compilation and cross-compilation only on the most convenient system for you.

(I suspect but don't know that if you have any Cgo-using packages you can copy $GOPATH/pkg/$GOOS_$GOARCH around from system to system to get functioning native versions of necessary packages. Try it and see.)

Even with this road bump the pragmatic bottom line is that Go cross-compilation is easy, useful, and is probably going to work for your Go programs. It's certainly easy enough that you should give it a try just to see if it works for you.

by cks at September 23, 2014 03:33 AM

September 22, 2014

Google Blog

For those who dream big: Announcing the winners of the 2014 Google Science Fair

Ciara Judge, Émer Hickey and Sophie Healy-Thow became interested in addressing the global food crisis after learning about the Horn of Africa famine in 2011. When a gardening project went awry, they discovered a naturally occurring bacteria in soil called Diazotroph. The girls determined that the bacteria could be used to speed up the the germination process of certain crops, like barley and oats, by 50 percent, potentially helping fulfill the rising demand for food worldwide. Oh—and they’re 16 years old.

Today, Ciara, Émer and Sophie were named the Grand Prize Winner and the 15-16 age category winners of our fourth annual Google Science Fair. They are some of thousands of students ages 13-18 who dared to ask tough questions like: How can we stop cyberbullying? How can I help my grandfather who has Alzheimer's from wandering out of bed at night? How can we protect the environment? And then they actually went out and answered them.

From thousands of submissions from 90+ countries, our panel of esteemed judges selected 18 finalists representing nine countries—Australia, Canada, France, India, Russia, U.K., Ukraine and the U.S.—who spent today impressing Googlers and local school students at our Mountain View, Calif. headquarters. In addition to our Grand Prize Winners, the winners of the 2014 Google Science Fair are:
  • 13-14 age category: Mihir Garimella (Pennsylvania, USA) for his project FlyBot: Mimicking Fruit Fly Response Patterns for Threat Evasion. Like many boys his age, Mihir is fascinated with robots. But he took it to the next level and actually built a flying robot, much like the ones used in search and rescue missions, that was inspired by the way fruit flies detect and respond to threats. Mihir is also the winner of the very first Computer Science award, sponsored by Google.
  • 17-18 age category: Hayley Todesco (Alberta, Canada) for her project Waste to Water: Biodegrading Naphthenic Acids using Novel Sand Bioreactors. Hayley became deeply interested in the environment after watching Al Gore’s documentary “An Inconvenient Truth.” Her project uses a sustainable and efficient method to break down pollutant substances and toxins found in tailing ponds water in her hometown, a hub of the oil sands industry.
  • The Scientific American Science in Action award: Kenneth Shinozuka (Brooklyn, New York) for his wearable sensors project. Kenneth was inspired by his grandfather and hopes to help others around the world dealing with Alzheimer's. The Scientific American award is given to a project that addresses a health, resource or environmental challenge.
  • Voter’s Choice award: Arsh Dilbagi (India) for his project Talk, which enables people with speech difficulties to communicate by simply exhaling.
Hayley, Mihir, Kenneth, Ciara, Sophie and Émer

As the Grand Prize winners, Ciara, Émer and Sophie receive a 10-day trip to the Galapagos Islands provided by National Geographic, a $50,000 scholarship from Google, a personalized LEGO prize provided by LEGO Education and the chance to participate in astronaut training at the Virgin Galactic Spaceport in the Mojave desert.

Thanks to all of our young finalists and to everyone who participated in this year’s Google Science Fair. We started the Science Fair to inspire scientific exploration among young people and celebrate the next generation of scientist and engineers. And every year we end up amazed by how much you inspire us. So, keep dreaming, creating and asking questions. We look forward to hearing the answers.

by Emily Wood (noreply@blogger.com) at September 22, 2014 09:52 PM

Yellow Bricks

VMware EVO:RAIL demos


I just bumped in to a bunch of new VMware EVO:RAIL demos which I wanted to share. Especially the third demo which shows how EVO:RAIL scales out by a couple of simple clicks.

General overview:

Customer Testimonial:


Clustering appliances:

Management experience:

Configuration experience:

x

"VMware EVO:RAIL demos" originally appeared on Yellow-Bricks.com. Follow me on twitter - @DuncanYB.


Pre-order my upcoming book Essential Virtual SAN via Pearson today!

by Duncan Epping at September 22, 2014 07:56 AM

Aaron Johnson

Links: 9-21-2014

  • Fixed mindset vs Growth mindset | Derek Sivers
    Quote: "People in a fixed mindset believe you either are or aren’t good at something, based on your inherent nature, because it’s just who you are. People in a growth mindset believe anyone can be good at anything, because your abilities are entirely due to your actions." Need to somehow get this through to the little dudes in my life.
    (categories: life motivation thinking mindset change failure )

by ajohnson at September 22, 2014 06:30 AM

Chris Siebenmann

Another side of my view of Python 3

I have been very down on Python 3 in the past. I remain sort of down on it, especially in the face of substantial non-current versions on the platforms I use and want to use, but there's another side of this that I should admit to: I kind of want to be using Python 3.

What this comes down to at its heart is that for all the nasty things I say about it, Python 3 is where the new and cool stuff is happening in Python. Python 3 is where all of the action is and I like that in general. Python 2 is dead, even if it's going to linger on for a good long while, and I can see the writing on the wall here.

(One part of that death is that increasingly, interesting new modules are only going to be Python 3 or are going to be Python 3 first and only Python 2 later and half-heartedly.)

And Python 3 is genuinely interesting. It has a bunch of new idioms to get used to, various challenges to overcome, all sorts of things to learn, and so on. All of these are things that generally excite me as a programmer and make it interesting to code stuff (learning is fun, provided I have a motivation).

Life would be a lot easier if I didn't feel this way. If I felt that Python 3 had totally failed as a language iteration, if I thought it had taken a terrible wrong turn that made it a bad idea, it would be easy to walk away from it entirely and ignore it. But it hasn't. While I dislike some of its choices and some of them are going to cause me pain, I do expect that the Python 3 changes are generally good ones (and so I want to explore them). Instead, I sort of yearn to program in Python 3.

So why haven't I? Certainly one reason is that I just haven't been writing new Python code lately (and beyond that I have real concerns about subjecting my co-workers to Python 3 for production code). But there's a multi-faceted reason beyond that, one that's going to take another entry to own up to.

(One aspect of the no new code issue is that another language has been competing for my affections and doing pretty well so far. That too is a complex issue.)

by cks at September 22, 2014 04:21 AM

September 21, 2014

Ubuntu Geek

How to use CloudFlare as a ddclient provider under Ubuntu

DDclient is a Perl client used to update dynamic DNS entries for accounts on Dynamic DNS Network Service Provider. It was originally written by Paul Burry and is now mostly by wimpunk. It has the capability to update more than just dyndns and it can fetch your WAN-ipaddress in a few different ways.
(...)
Read the rest of How to use CloudFlare as a ddclient provider under Ubuntu (288 words)


© ruchi for Ubuntu Geek, 2014. | Permalink | No comment | Add to del.icio.us
Post tags: , , ,

Related posts

by ruchi at September 21, 2014 11:14 PM

SysAdmin1138

The alerting problem

4100 emails.

That's the approximate number of alert emails that got auto-deleted while I was away on vacation. That number will rise further before I officially come back from vacation, but it's still a big number. The sad part is, 98% of those emails are for:

  • Problems I don't care about.
  • Unsnoozable known issues.
  • Repeated alarms for the first two points (puppet, I'm looking at you)

We've made great efforts in our attempt to cut down our monitoring fatigue problem, but we're not there yet. In part this is because the old, verbose monitoring system is still up and running, in part this is due to limitations in the alerting systems we have access to, and in part due to organizational habits that over-notify for alarms under the theory of, "if we tell everyone, someone will notice."

A couple weeks ago, PagerDuty had a nice blog-post about tackling alert fatigue, and had a lot of good points to consider. I want to spend some time on point 6:

Make sure the right people are getting alerts.

How many of you have a mailing list you dump random auto-generated crap like cron errors and backup failure notices to?

This pattern is very common in sysadmin teams, especially teams that began as one or a very few people. It just doesn't scale. Also, you learn to just ignore a bunch of things like backup "failures" for always-open files. You don't build an effective alerting system with the assumption that alerts can be ignored; if you find yourself telling new hires, "oh ignore those, they don't mean anything," you have a problem.

The failure mode of tell-everyone is that everyone can assume someone else saw it first and is working on it. And no one works on it.

I've seen exactly this failure mode many times. I've even perpetrated it, since I know certain coworkers are always on top of certain kinds of alerts so I can safely ignore actually-critical alerts. It breaks down if those people have a baby and are out of the office for four weeks. Or were on the Interstate for three hours and not checking mail at that moment.

When this happens and big stuff gets dropped, technical management gets kind of cranky. Which leads to hypervigilence and...

The failure mode of tell-everyone is that everyone will pile into the problem at the same time and make things worse.

I've seen this one too. A major-critical alarm is sent to a big distribution list, six admins immediately VPN in and start doing low-impact diagnostics. Diagnostics that aren't low impact if six people are doing them at the same time. Diagnostics that aren't meant to be run in parallel and can return non-deterministic results if run that way, which tells six admins different stories about what's actually wrong sending six admins into six different directions to solve not-actually-a-problem issues.

This is the Thundering Herd problem as it applies to sysadmins.

The usual fix for this is to build in a culture of, "I've got this," emails and to look for those messages before working on a problem.

The usual fix for this fails if admins do a little "verify the problem is actually a problem" work before sending the email and stomp on each other's toes in the process.

The usual fix for that is to build a culture of, "I'm looking into it," emails.

Which breaks down if a sysadmin is reasonably sure they're the only one who saw the alert and works on it anyway. Oops.


Really, these are all examples of telling the right people about the problem, but you really do need to go into more detail than "the right people". You need, "the right person". You need an on-call schedule that will notify one or two of the Right People about problems. Build that with the expectation that if you're in the hot seat you will answer ALL alerts, and build a rotation so no one is in the hotseat long enough to start ignoring alarms, and you have a far more reliable alerting system.

PagerDuty sells such a scheduling system. But what if you can't afford X-dollars a seat for something like that? You have some options. Here is one:

An on-call distribution-list and scheduler tasks
This recipe will provide an on-call rotation using nothing but free tools. It won't work with all environments. Scripting or API access to the email system is required.

Ingredients:

    • 1 on-call distribution list.
    • A list of names of people who can go into the DL.
    • A task scheduler such as cron or Windows Task Scheduler.
    • A database of who is supposed to be on-call when (can substitute a flat file if needed)
    • A scripting language that can talk to both email system management and database.

Instructions:

Build a script that can query the database (or flat-file) to determine who is supposed to be on-call right now, and can update the distribution-list with that name. Powershell can do all of this for full MS-stack environments. For non-MS environments more creativity may be needed.

Populate the database (or flat-file) with the times and names of who is to be on-call.

Schedule execution of the script using a task scheduler.

Configure your alert-emailing system to send mail to the on-call distribution list.

Nice and free! You don't get a GUI to manage the schedule and handling on-call shift swaps will be fully manual, but you at least are now sending alerts to people who know they need to respond to alarms. You can even build the watch-list so that it'll always include certain names that always want to know whenever something happens, such as managers. The thundering herd and circle-of-not-me problems are abated.

This system doesn't handle escalations at all, that's going to cost you either money or internal development time. You kind of do get what you pay for, after all.

How long should on-call shifts be?

That depends on your alert-frequency, how long it takes to remediate an alert, and the response time required.

Alert Frequency and Remediation:

  • Faster than once per 30 minutes:
    • They're a professional fire-fighter now. This is their full-time job, schedule them accordingly.
  • One every 30 minutes to an hour:
    • If remediation takes longer than 1 minute on average, the watch-stander can't do much of anything else but wait for alerts to show up. 8-12 hours is probably the most you can expect reasonable performance.
    • If remediation takes less than a minute, 16 hours is the most you can expect because this frequency ensures no sleep will be had by the watch-stander.
  • One every 1-2 hours:
    • If remediation takes longer than 10 minutes on average, the watch-stander probably can't sleep on their shift. 16 hours is probably the maximum shift length.
    • If remediation takes less than 10 minutes, sleep is more possible. However, if your watch-standers are the kind of people who don't fall asleep fast, you can't rely on that. 1 day for people who sleep at the drop of a hat, 16 hours for the rest of us.
  • One every 2-4 hours:
    • Sleep will be significantly disrupted by the watch. 2-4 days for people who sleep at the drop of a hat. 1 day for the rest of us.
  • One every 4-6 hours:
    • If remediation takes longer than an hour, 1 week for people who sleep at the drop of a hat. 2-4 days for the rest of us.
  • Slower than one every 6 hours:
    • 1 week

Response Time:

This is a fuzzy one, since it's about work/life balance. If all alerts need to be responded to within 5 minutes of their arrival, the watch-stander needs to be able to respond in 5 minutes. This means no driving or doing anything that requires not paying attention to the phone such as kid's performances or after-work meetups. For a watch-stander that drives to work, their on-call shift can't overlap their commute.

For 30 minute response, things are easier. Driving short trips is easier, and longer ones so long as the watch-stander pulls over to check what each alert is when they arrive. Kid performances are still problematic, and longer commutes just as much.

And then there is the curve-ball known as, "define 'response'". If Response is acking the alert, that's one thing and much less disruptive to off-hours lie. If Response is defined as "starts working on the problem," that's much more disruptive since the watch-stander has to have a laptop and bandwidth at all times.

The answers here will determine what a reasonable on-call shift looks like. A week of 5 minute time-to-work is going to cause the watch-stander to be house-bound for that entire week and that sucks a lot; there better be on-call pay associated with a schedule like that or you're going to get turnover as sysadmins go work for someone less annoying.


It's more than just make sure the right people are getting alerts, it's building a system of notifying the Right People in such a way that the alerts will get responded to and handled.

This will build a better alerting system overall.

by SysAdmin1138 at September 21, 2014 11:00 PM

Aaron Johnson

WHAT I DID THIS WEEKEND: 09/21/2014

  • Did a long drive out to Offa’s Dyke (nice little playground and trail for the kids by the visitors center but otherwise nothing to write home about) and Croft Castle (more of a manor but had a beautiful garden, a nice playground and some cool WWI outfits to try on) on Saturday.
  • Took the boys geocaching around our house on Sunday, found our first five caches on a great 3.5 mile hike / walk we did right from the house. Have a profile now on geocaching.com. Looking forward to finding some caches on our trip to Iceland in a couple weeks. Bought a couple tracker bugs from Amazon that we’ll leave in Iceland.
  • Long run for the week (training for half marathon distance) was 8.3 miles this afternoon in beautiful weather. Was really interesting to see my heart rate (which seems to be relatively high on the other runs I’ve done based on the 220 – your age thing I’ve read in various places) for the first 45 minutes staying WELL below what it usually is, first time I’ve felt like the training (which is easy run on Monday, tempo run on Wednesday, interval / hill run on Friday) has paid off. I’m still super slow, but that might have been the longest run I’ve ever done in my life. Doesn’t look like tapiriik is syncing stuff from Garmin back to Runkeeper. :(

by ajohnson at September 21, 2014 09:20 PM

Server Density

Chris Siebenmann

One reason why Go can have methods on nil pointers

I was recently reading an article on why Go can call methods on nil pointers (via) and wound up feeling that it was incomplete. It's hard to talk about 'the' singular reason that Go can do this, because a lot of design decisions went into the mix, but I think that one underappreciated reason this happens is because Go doesn't have inheritance.

In a typical language with inheritance, you can both override methods on child classes and pass a pointer to a child class instance to a function that expects a pointer to the parent class instance (and the function can then call methods on that pointer). This combination implies that the actual machine code in the function cannot simply make a static call to the appropriate parent class method function; instead it must somehow go through some sort of dynamic dispatch process so that it calls the child's method function instead of the parent's when passed what is actually a pointer to a child instance.

In non-nil pointers to objects, you have a natural place to put such a vtable (or rather a pointer to it) because the object has actual storage associated with it. But a nil pointer has no storage associated with it and so you can't naturally do this. That means given a nil pointer, how do you find the correct vtable? After all it might be a nil pointer of the child class that should call child class methods.

Because Go has no inheritance this problem does not come up. If your function takes a pointer to a concrete type and you call t.Method(), the compiler statically knows which exact function you're calling; it doesn't need to do any sort of dynamic lookup. Thus it can easily make this call even when given a nil. In effect the compiler gets to rewrite a call to t.Method() to something like ttype_Method(t).

But wait, you may say. What about interfaces? These have exactly the dynamic dispatch problem I was just talking about. The answer is that Go actually represents interface values that are pointers as two pointers; one which is the actual value and another points to (among other things) a vtable for the interface (which is populated based on the concrete type). Because Go statically knows that it is dealing with an interface instead of a concrete type, the compiler builds code that calls indirectly through this vtable.

(As I found out, this can lead to a situation where what you think is a nil pointer is not actually a nil pointer as Go sees it because it has been wrapped up as an interface value.)

Of course you could do this two-pointer trick with concrete types too if you wanted to, but it would have the unfortunate effect of adding an extra word to the size of all pointers. Most languages are not interested in paying that cost just to enable nil pointers to have methods.

(Go doesn't have inheritance for other reasons; it's probably just a happy coincidence that it enables nil pointers to have methods.)

PS: it follows that if you want to add inheritance to Go for some reason, you need to figure out how to solve this nil pointer with methods problem (likely in a way that doesn't double the size of all pointers). Call this an illustration of how language features can be surprisingly intertwined with each other.

by cks at September 21, 2014 05:38 AM

September 20, 2014

Everything Sysadmin

How the internet has affected what books get published?

Someone recently asked me how the rise of the Internet has affected what books get published, specifically related to books about operating systems and other open source projects.

This is based on what I've been told by various publishers and is "conventional wisdom". Of course, an actual publisher may disagree or explain it differently, or have counterexamples.

This is the email I sent in reply:

One way that the internet has changed the book industry that is not well-known outside of publishing circles is that it has lead to the death of the reference book.

It used to be for every language or system, someone would make easy money printing a book that lists all the system calls, library calls, or configuration fields in a system. Each would be listed along with a definition, and sometimes a longer explanation or example. These reference books (and the easy profits from them) have disappeared because that's all on the web for free. For example, the entire standard Python library is documented on python.org and is updated constantly. Why publish it in a book?

Another tier of books that have disappeared are the "getting started" books for open source projects. In an effort to better market themselves, open source projects have excellent tutorials online for free. Often they are interactive. (Check out the interactive Docker Tutorial and the Go Playground to see what I mean).

Part of what has caused this is the commercialization of open source projects. If you are selling support, your easiest sales are to people that already use the product and now need support. Therefore anything you do to expand the number of people using the product expands the number of potential customers for the support services. Therefore, a greedy company gives away any documentation related to how to get started as it increases their base of potential customers. This is quite radical if you think about it... when you sell commercial software you hide this information so you can sell "professional services" that help people get started. For them, the potential market is all the people that don't use the product already. (Footnote Simon Philips's keynote at LCA 2009 )

As a result, the books that can be printed and sold profitably now need to have have some kind of "culture" component (i.e. what we used to call "soft skills"), or are general education to bring people up to speed so they can understand what's already out there, or the "cookbook" style books that list situation after situation. Actually, I love the cookbook-style books because it is more likely that I'll serendipitously come across something new by reading all the examples in a row. If I don't understand the example at least I know where to turn if I ever find myself in that situation. If I do decypher the example enough to understand why it works, I've learned a lot about the system.

What should an OS vendor do? I think they should be the ones producing cookbooks and also facilitating user-generated cookbooks. The official cookbook is a good way to proscribe marketing-endorsed methods. However there is a much bigger need for user-generated cookbooks full of the niche situations that only users can come across. Vendors should facilitate, wiki-style, this kind of thing. They may fear that it is a tacit endorsement of methods they don't approve of, but they need to recognize that a user getting their needs fulfilled is higher priority than perfection.

-Tom

P.S. I just realized that my employer's website, [stackoverflow.com)[http://stackoverflow.com] is just that kind of fulfillment. One of our sub-sites AskUbuntu.com provides exactly that kind of "user-generated cookbook" with the ability to gently push people in the right direction. (Plug: other vendors could choose to partner with us rather than create a site from scratch.)

September 20, 2014 07:28 PM

Chris Siebenmann

My view on using VLANs for security

I've recently read some criticism of the security value of VLANs. Since we use VLANs heavily I've been thinking a bit about this issue and today I feel like writing up my opinions. The short version is that I don't think using VLANs is anywhere close to being an automatic security failure. It's much more nuanced (and secure) than that.

My overall opinion is that the security of your VLANs rests on the security of the switches (and hosts) that the VLANs are carried on, barring switch bugs that allow you to hop between VLANs in various ways or to force traffic to leak from one to another. The immediate corollary is that the most secure VLANs are the ones that are on as few switches as possible. Unfortunately this cuts against both flexibility and uniformity; it's certainly easier if you have all of your main switches carry all of your VLANs by default, since that makes their configurations more similar and means it's much less work to surface a given VLAN at a given point.

(This also depends on your core network topology. A chain or a ring can force you to reconfigure multiple intermediate switches if VLAN A now needs to be visible at a new point B, whereas a star topology pretty much insures only a few directly involved switches need to be touched.)

Because they're configured (partly) through software instead of purely by physical changes, a VLAN based setup is more vulnerable to surreptitious evil changes. All an attacker has to do is gain administrative switch access and they can often make a VLAN available to something or somewhere it shouldn't be. As a corollary, it's harder to audit a VLAN-based network than one that is purely physical in that you need to check the VLAN port configurations in addition to the physical wiring.

(Since basically all modern switches are VLAN-capable even if you don't use the features, I don't think that avoiding VLANs means that an attacker who wants to get a new network on to a machine needs the machine to have a free network port. They can almost certainly arrange a way to smuggle the network to the machine as a tagged link on an existing port.)

So in summary I think that VLANs are somewhat less secure than separate physical networks but not all that much less secure, since your switches should be fairly secure in general (both physically and for configuration changes). But if you need ultimate security you do want or need to build out physically separate networks. However my suspicions are that most people don't have security needs that are this high and so are fine with using just VLANs for security isolation.

(Of course there are political situations where having many networks on one switch may force you to give all sorts of people access to that switch so that they can reconfigure 'their' network. If you're in this situation I think that you have several problems, but VLANs do seem like a bad idea because they lead to that shared switch awkwardness.)

Locally we don't have really ultra-high security needs and so our VLAN setup is good enough for us. Our per-group VLANs are more for traffic isolation than for extremely high security, although of course they and the firewalls between the VLANs do help increase the level of security.

Sidebar: virtual machines, hosts, VLANs, and security

One relatively common pattern that I've read about for virtual machine hosting is to have all of the VLANs delivered to your host machines and then to have some sort of internal setup that routes appropriate networks to all of the various virtual machines on a particular host. At one level you can say that this is obviously a point of increased vulnerability with VLANs; the host machine is basically operating as a network switch in addition to its other roles so it's an extra point of vulnerability (perhaps an especially accessible one if it can have the networking reconfigured automatically).

My view is that to say this is to misread the actual security vulnerability here. The real vulnerability is not having VLANs; it is hosting virtual machines on multiple different networks (presumably of different security levels) on the same host machine. With or without VLANs, all of those networks have to get to that host machine and thus it has access to all of them and thus can be used to commit evil with or to any of them. To really increase security here you need to deliver fewer networks to each host machine (which of course has the side effect of making them less uniform and constraining which host machines a given virtual machine can run on).

(The ultimate version is that each host machine is only on a single network for virtual machines, which means you need at least as many host machines as you have networks you want to deploy VMs on. This may not be too popular with the people who set your budgets.)

by cks at September 20, 2014 03:21 AM

September 19, 2014

Google Blog

Through the Google lens: search trends Sept 12-18

-Welcome to this week’s search trends. May I take your order?
-Can I have a referendum on independence, a totally inappropriate flight passenger with a Hollywood baby on the side?
-Coming right up!

Flag and country
“They may take away our lives, but they'll never take our freedom!” That was Sir William Wallace battlecry for Scottish independence in the film Braveheart. While this week’s events in Scotland weren’t quite as cinematic, the results could have been revolutionary. On Thursday the world watched and searched as an unprecedented numbers of Scots went to the polls to answer the question, "Should Scotland be independent from the United Kingdom?" Turns out the majority of people don’t think it should, and voted to stay a member of the U.K. Party leaders have now promised significant constitutional changes for the entire kingdom. What would Wallace have made of that?

The comeback kings
Everybody loves a comeback and search had its fair share this week. First up, nostalgia for the 90’s brought Surge soda back from the dead. Thanks to a Facebook campaign called "The SURGE Movement," Coca-Cola will now sell its "fully-loaded citrus” soft drink for a limited time on Amazon. And the Chicago Bears denied the 49ers a win in their brand-spanking-new stadium when they rallied to overturn a 13-point deficit in the last quarter to beat San Francisco 28-20.



Airing dirty laundry
Hard plastic-y seats, broken recliner adjusters, zero leg room—flying economy isn’t always the most pleasant experience. And depending on who you’re sitting next to, your easy two-hour flight could turn into a nightmare before you even take off. But the passengers of the world aren’t having it, not anymore. This week, “passenger shaming” went viral on social media as traumatized travelers shared photos of the most absurdly obnoxious unconscientious things some passenger do on flights—we’re talking bare feet, bare skin... well, you should just see for yourself.

But at least those offending fliers were shielded in anonymity. Singer Robin Thicke wasn’t afforded the same luxury, revealing in a court deposition this week that he had little to do with the creation of last year’s song of the summer “Blurred Lines.” As part of his defense against a copyright infringement lawsuit, Thicke admitted that he was under the influence of drugs and alcohol for most of 2013—bringing a whole new meaning to the song’s title.

And the winner is ...
The hipster revolution has finally taken over the United States! Need proof? Searchers don’t. When New Yorker Kira Kazantsev won the the title of Miss America, the Internet discovered that the U.S.A’s new leading lady is a former food blogger. She’s even reported on her state’s crown foodie jewel, the cronut. Miss America wasn’t the only who got to bask in the limelight; boxing world champion Floyd “Money” Mayweather Jr. won his rematch with contender Marcos Maidana by an unanimous decision. The victory brings his undefeated tally to 47… somehow the title world champion is starting to sound like an understatement.

Love on the set!
For Orange is the New Black screenwriter Lauren Morelli, life imitated art a bit more than she probably expected. While writing the hit program, Morelli decided to divorce her husband and start a relationship with Samira Wiley, an actress from the show. Meanwhile, searchers learned that Mindy Kaling considers former The Office castmate and on-screen boyfriend B.J. Novak “the love that got away.” But while not all on-set relationships last, some couples not only make it work but also take their relationship to the next level. That’s the route taken by Ryan Gosling and Eva Mendes, who met while making the movie The Place Beyond the Pines. The power couple welcomed baby girl Gosling earlier this week.

Tip of the week
The NFL season’s just getting started so it’s time to hunker down and plan your football viewing schedule. Just say, “OK Google, show me the NFL schedule” to coordinate your life for the next four months. We’ll see you back in the spring.

by Emily Wood (noreply@blogger.com) at September 19, 2014 03:36 PM

Chris Siebenmann

What I mean by passive versus active init systems

I have in the past talked about passive versus active init systems without quite defining what I meant by that, except sort of through context. Since this is a significant division between init systems that dictates a lot of other things, I've decided to fix that today.

Put simply, an active init system is one that actively tracks the status of services as part of its intrinsic features; a passive init system is one that does not. The minimum behavior of an active init system is that it knows what services have been activated and not later deactivated. Better active init systems know whether services are theoretically still active or if they've failed on their own.

(Systemd, upstart, and Solaris's SMF are all active init systems. In general any 'event-based' init system that starts services in response to events will need to be active, because it needs to know which services have already been started and which ones haven't and thus are candidates for starting now. System V init's /etc/init.d scripts are a passive init system, although /etc/inittab is an active one. Most modern daemon supervision systems are active systems.)

One direct consequence is that an active init system essentially has to do all service starting and stopping itself, because this is what lets it maintain an accurate record of what services are active. You may run commands to do this, but they have to talk to the init system itself. By contrast, in a passive init system the commands you run to start and stop services can be and often are just shell scripts; this is the archetype of System V init.d scripts. You can even legitimately start and stop services outside of the scripts at all, although things may get a bit confused.

(In the *BSDs things can be even simpler in that you don't have scripts and you may just run the daemons. I know that OpenBSD tends to work this way but I'm not sure if FreeBSD restarts stuff quite that directly.)

An active init system is also usually more communicative with the outside world. Since it knows the state of services it's common for the init system to have a way to report this status to people who ask, and of course it has to have some way of being told either to start and stop services or at least that particular services have started and stopped. Passive init systems are much less talkative; System V init basically has 'change runlevel' and 'reread /etc/inittab' and that's about it as far its communication goes (and it doesn't even directly tell you what the runlevel is; that's written to a file that you read).

Once you start down the road to an active init system, in practice you wind up wanting some way to track daemon processes so you can know if a service has died. Without this an active init system is basically flying blind in that it knows what theoretically started okay but it doesn't necessarily know what's still running. This can be done by requiring cooperative processes that don't do things like detach themselves from their parents or it can be done with various system specific Unix extensions to track groups of processes even if they try to wander off on their own.

As we can see from this, active init systems are more complicated than passive ones. Generally the more useful features they offer and the more general they are the more complicated they will be. A passive init system can be done with shell scripts; an attractive active one requires some reasonably sophisticated C programming.

PS: An active init system that notices when services die can offer a feature where it will restart them for you. In practice most active init systems aren't set up to do this for most services for various reasons (that may or may not be good ones).

(This entry was partly sparked by reading parts of this mail thread that showed up in my Referer logs because it linked to some of my other entries.)

by cks at September 19, 2014 04:58 AM

September 18, 2014

Ubuntu Geek

Vesta – Simple & Clever Hosting Control Panel

We made an extremely simple and clear interface. Instead of adding more elements to work with, we prefer to remove as much as possible. The main goal was to improve the ergonomics of the control panel by reducing unnecessary movements and operations. It is all about using less, because less is more. We hope you will love it as we do.

(...)
Read the rest of Vesta – Simple & Clever Hosting Control Panel (302 words)


© ruchi for Ubuntu Geek, 2014. | Permalink | No comment | Add to del.icio.us
Post tags: , ,

Related posts

by ruchi at September 18, 2014 11:07 PM

systemBash

Centos pip python install error

While attempting to install Thumbor on a CentOS server I recently had the following error message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
------------------------------------------------------------
/usr/bin/pip run on Thu Sep 18 21:07:45 2014
Getting page https://pypi.python.org/simple/pycrypto/
URLs to search for versions for pycrypto in /usr/lib64/python2.6/site-packages:
* https://pypi.python.org/simple/pycrypto/
Analyzing links from page https://pypi.python.org/simple/pycrypto/
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.0.1.tar.gz#md5=4d5674f3898a573691ffb335e8d749cd (from https://pypi.python.org/simple/pycrypto/), version: 2.0.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.1.0.tar.gz#md5=1d3eb04f06e6f09a080bc37fb019f9bf (from https://pypi.python.org/simple/pycrypto/), version: 2.1.0
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.2.tar.gz#md5=4f0ed728b14b98f09120cb2ec461ec98 (from https://pypi.python.org/simple/pycrypto/), version: 2.2
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.3.tar.gz#md5=2b811cfbfc342d83ee614097effb8101 (from https://pypi.python.org/simple/pycrypto/), version: 2.3
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.1.tar.gz#md5=c2a1404a848797fb0806f3e11c29ef15 (from https://pypi.python.org/simple/pycrypto/), version: 2.4.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.tar.gz#md5=274fa44c30a320d56460a93fdd95e702 (from https://pypi.python.org/simple/pycrypto/), version: 2.4
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz#md5=783e45d4a1a309e03ab378b00f97b291 (from https://pypi.python.org/simple/pycrypto/), version: 2.5
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda (from https://pypi.python.org/simple/pycrypto/), version: 2.6.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.tar.gz#md5=88dad0a270d1fe83a39e0467a66a22bb (from https://pypi.python.org/simple/pycrypto/), version: 2.6
Using version 2.6.1 (newest of versions: 2.6.1, 2.6, 2.5, 2.4.1, 2.4, 2.3, 2.2, 2.1.0, 2.0.1, 2.0.1)
Downloading/unpacking pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda

  Running setup.py egg_info for package pycrypto

    running egg_info
    writing pip-egg-info/pycrypto.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pycrypto.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pycrypto.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/pycrypto has version 2.6.1, which satisfies requirement pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda
Installing collected packages: pycrypto

  Found existing installation: pycrypto 2.0.1

    Uninstalling pycrypto:

      Removing file or directory /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
      Successfully uninstalled pycrypto

  Running setup.py install for pycrypto

    Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed
    running install
    running build
    running build_py
    running build_ext
    running build_configure
    checking for gcc... gcc

    checking whether the C compiler works... yes

    checking for C compiler default output file name... a.out

    checking for suffix of executables...

    checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

    configure: error: cannot run C compiled programs.

    If you meant to cross compile, use `--host'.

    See `config.log' for more details

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

        core.setup(**kw)

      File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

        dist.run_commands()

      File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

        self.run_command(cmd)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

        return _install.run(self)

      File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

        self.run_command('build')

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

        raise RuntimeError("autoconf error")

    RuntimeError: autoconf error

    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_py

running build_ext

running build_configure

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `--host'.

See `config.log' for more details

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

    core.setup(**kw)

  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

    self.run_command(cmd)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

    return _install.run(self)

  File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

    self.run_command('build')

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

    raise RuntimeError("autoconf error")

RuntimeError: autoconf error

----------------------------------------

  Rolling back uninstall of pycrypto

  Replacing /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/usr/lib/python2.6/site-packages/pip/commands/install.py", line 271, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 1185, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 592, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

It essentially boils down to:

1
2
3
4
checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

Weird, I have gcc and all compile programs installed.

It took me a fair time of troubleshooting, but I finally figured out it was because it was attempting to build this in /tmp, which I have set to mount at noexec for security purposes. This disallows execution of programs in this directory.

Running

1
mount -oremount,exec /tmp

Allowed it to run without issue.

by Dave at September 18, 2014 07:17 PM

Server Density

Automated Google Cloud and Google Compute Engine monitoring

Today we’re releasing the Server Density integration into Google Compute Engine as an official Google Cloud Platform Technology Partner. Server Density works across all environments and platforms and is now fully integrated into Google’s cloud infrastructure products, including Compute Engine and Persistent Disks, to offer alerting, historical metrics and devops dashboards to Google customers.

Google Cloud graphs

Server Density customers can connect their Google Cloud accounts to automatically monitor and manage instances across Google data centers alongside existing environments and other cloud providers. Many customers will run systems across multiple providers in a hybrid setup, so Server Density is uniquely placed to help with that because even though we have specialist integration into Google, it works well anywhere – cloud, hybrid and on-prem.

$500 credit for Google/Server Density customers

Server Density normally starts at $10/m to monitor Linux, Windows, FreeBSD and Mac servers but Google Cloud customers can monitor up to 5 servers for free for life (worth over $500/year). Google is also offering Server Density customers $500 in credits to trial Google Cloud Platform. To find out more and sign up, head over to our website for details.

The post Automated Google Cloud and Google Compute Engine monitoring appeared first on Server Density Blog.

by David Mytton at September 18, 2014 01:00 PM

Chris Siebenmann

Ubuntu's packaging failure with mcelog in 14.04

For vague historical reasons we've had the mcelog package in our standard package set. When we went to build our new 14.04 install setup, this blew up on us; on installation, some of our machines would report more or less the following:

Setting up mcelog (100-1fakesync1) ...
Starting Machine Check Exceptions decoder: CPU is unsupported
invoke-rc.d: initscript mcelog, action "start" failed.
dpkg: error processing package mcelog (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 mcelog
E: Sub-process /usr/bin/dpkg returned an error code (1)

Here we see a case where a collection of noble intentions have had terrible results.

The first noble intention is a desire to warn people that mcelog doesn't work on all systems. Rather than silently run uselessly or silently exit successfully, mcelog instead reports an error and exits with a failure status.

The second noble intention is the standard Debian noble intention (inherited by Ubuntu) of automatically starting most daemons on installation. You can argue that this is a bad idea for things like database servers, but for basic system monitoring tools like mcelog and SMART monitoring I think most people actually want this; certainly I'd be a bit put out if installing smartd didn't actually enable it for me.

(A small noble intention is that the init script passes mcelog's failure status up, exiting with a failure itself.)

The third noble intention is that it is standard Debian behavior for an init script that fails when it is started in the package's postinstall script to cause the postinstall script itself to exit out with errors (it's in a standard dh_installinit stanza). When the package postinstall script errors out, dpkg itself flags this as a problem (as well it should) and boom, your entire package install step is reporting an error and your auto-install scripts fall down. Or at least ours do.

The really bad thing about this is that server images can change hardware. You can transplant disks from one machine to another for various reasons; you can upgrade the hardware of a machine but preserve the system disks; you can move virtual images around; you can (as we do) have standard machine building procedures that want to install a constant set of packages without having to worry about the exact hardware you're installing on. This mcelog package behavior damages this hardware portability in that you can't safely install mcelog in anything that may change hardware. Even if the initial install succeeds or is forced, any future update to mcelog will likely cause you problems on some of your machines (since a package update will likely fail just like a package install).

(This is a packaging failure, not an mcelog failure; given that mcelog can not work on some machines it's installed on, the init script failure should not cause a fatal postinstall script failure. Of course the people who packaged mcelog may well not have known that it had this failure mode on some machines.)

I'm sort of gratified to report that Debian has a bug for this, although the progress of the bug does not fill me with great optimism and of course it's probably important enough to ever make it into Ubuntu 14.04 (although there's also an Ubuntu bug).

PS: since mcelog has never done anything particularly useful for us, we have not been particularly upset over dropping it from our list of standard packages. Running into the issue was a bit irritating though, but mcelog seems to be historically good at irritation.

PPS: the actual problem mcelog has is even more stupid than 'I don't support this CPU'; in our case it turns out to be 'I need a special kernel module loaded for this machine but I won't do it for you'. It also syslogs (but does not usefully print) a message that says:

mcelog: AMD Processor family 16: Please load edac_mce_amd module.#012: Success

See eg this Fedora bug and this Debian bug. Note that the message really means 'family 16 and above', not 'family 16 only'.

by cks at September 18, 2014 05:58 AM

September 17, 2014

Rands in Repose

The Song of the Introvert

You are a threat.

It’s a strong word. I don’t mean that you intend pain, injury, or damage. But I’m an introvert and you – as a new unknown human – are a threat to me. I don’t know what you want and you most definitely want something and until I figure that out, you’re a threat. See…

I have control issues.

I am mostly calm when I am alone in my Cave. My stuff is where I expect it to be, the furniture is how I like it and the walls are blood red – they surround me completely. There are rarely surprises in my Cave and that is how I like it, thank you very much. My Cave is where I avoid the chaos and…

You are chaos.

You are disorder and confusion. I haven’t figured out an eye contact protocol with you yet, and I don’t know what you want so I don’t understand what motivates you so you are unpredictable. You are an unknown, which means you are full of surprises and surprises aren’t the spice of life, they are new data that don’t yet fit in my system and…

I am addled with systems.

My love of calm predictability has come at a cost. I write everything down in a black notebook – no lines. There are boxes next to items that must be tracked, there are stars for ideas that must be remembered. A yellow highlighter and a .5mm Zebra Sarasa gel pen accompany me everywhere because the presence of this notebook is part of my well-defined system of never missing anything. See, paradoxically, while I would likely prefer to be hiding in my Cave, I also love signal and…

You are high signal.

I am fascinated by how you punctuate your sentences with your hands. You pause for as long as it takes to makes sure you are going to say something of value. Sometimes these pauses are maddeningly long. You are fiercely optimistic and state outlandish impossible things. You are fearless in giving feedback to strangers. You are less fearless, but you can deliver the same feedback with a momentary glance. It’s fascinating how all of you have built all of your systems to get through your day. I am fascinated because…

I am insatiably (quietly) curious.

My curiosity is a defense mechanism. I am desperately trying to get back to my Cave where the surprises are scheduled. I have learned the faster I can learn about you, the faster I will figure out what you want, and that will tell me what motivates you, and when I know what motivates you, I will better understand how to communicate with you. I am not trying to manipulate you, I am not trying to pander to you, I am trying to understand you because…

I am an introvert.

by rands at September 17, 2014 01:36 PM

Tech Teapot

Back to Basics

After a while things stop being new. Things that really used to excite you, stop exciting you. Things that you were passionate about, you stop being passionate about. That’s just how things work.

I wrote my very first computer program 26 years ago this month. It was in college, using a Perkin Elmer mini computer running Berkely BSD 4.2 on a VT220 terminal (with a really good keyboard.) The program was written in Pascal. Pascal was the educational programming language of the time. It was an amazing time. Every time I went near the terminal, I approached with a sense of wonder.

But, over time, the sense of wonder starts to wane. Once somebody starts paying you to do something, the sense of wonder starts to wane real fast. You don’t control it any more. You are likely to be producing something that somebody else wants you to produce. In a way they want you to produce it.

I have been pondering my career recently. Such as it is. You do start pondering your career when you hit the wrong end of your forties. How can I get back that sense of wonder again?

I’ve always had a hankering after learning Lisp. I read about it even before I went to college twenty six years ago, and it has always fascinated me. Pretty well any programming concept you can think of, Lisp usually got there first.

One of my recent discoveries has been a series of books: The Little Schemer, The Seasoned Schemer and The Reasoned Schemer teaching Scheme in quite a unique, accessible and fun style.

Scheme is a modern dialect of Lisp. There are lots of others including Clojure.

I think that learning a language from scratch, just for the fun of it, may just be the tonic for a mild dose of mid-career blues. Hopefully, that sense of wonder may return. I sure hope so.

I’ll let you know :)

The post Back to Basics appeared first on Openxtra Tech Teapot.

by Jack Hughes at September 17, 2014 12:57 PM

Server Density

Photography for tech startups

After having released the new homepage for our server monitoring tool, I thought it’d be a nice idea to share the creative decisions we made, with a focus on photography. It’s not a particularly new idea for a tech company to invest in design, but it is unusual for photography – especially considering we only had a small budget. With that in mind (and a homepage in the balance), we needed to get it right the first time around.

Why photography?

As well as being a great way to digest content (a million words and all that jazz), photography is the number one way to sell a product online. If you take a look at Amazon, Apple or even Google Glass, the way they convince you to hand over your credit card details is with pictures. Text reinforces the boring bits that you need to know, but images sell the products.

Now if you transfer this to the startup scene, although companies are starting sell their software in the setting of how it will be consumed – they tend to be the big ones with endless marketing budgets. Couple the cost with the fact that a software company has nothing tangible to photograph and you start to realise why few startup marketers ‘expose’ themselves to the alien world of photography and art direction.

Despite the obvious difficulties we faced, we wanted to use photos because they’re a great way to link our product to real life, and the many ways in which our customers use our product. Instead of using screenshots to reinforce that message, setting the screenshots within a context that the visitor might be familiar with is a much better way of communicating the benefits of using Server Density. A whole lot more information gets transferred, not to mention they’d look nice… Hopefully!

As it was my first ever time being ‘creative director’ of a photo shoot, it was a nervous time. There was a lot of money riding on the photos, and let’s be honest the success / failure of our redesign. Here’s a few of the major stumbling blocks I came across, alongside how I alleviated them as problems.

The cost

The major downside of focussing designs on beautiful photos is that you first have to take those photos. Unless one of your team is also a still life photographer with 5+ years of experience, you’ll be paying a pretty penny for it. Or you’ll be getting some pretty amateur looking photographs. So first you have to find a great photographer, for a reasonable price.

Enter awesome photographer

This part was rather difficult in my head, but quite simple in practice. As we’re based in London, we needed to find someone local to us. I therefore trawled online for a few hours to find a selection of good photographers. After having a better look through portfolios we ended up with our favourite who I contacted. They key things that came out of our multiple phone conversations were:

  • We needed a relaxed photographer who could roll with the punches.
  • We needed them to bring both indoor and outdoor equipment.

Plan meticulously.

So the idea here is to plan every detail before the photographer comes. All they should then have to do is spend an hour sorting the lighting out and then a few clicks on the camera. In our case and a lot of others in the startup world, our photographs need text to be overlaid on to the images. With that in mind, I gave our photographer a visual reference of how the headings / navigation / call to action would be formatted so he understood the distribution of whitespace that was needed.

Startup photo - whitespace

That was an important step done, because these images need to fit around the content, not the other way round.

Storyboard the photographs

With that complete, it became about storyboarding. This provided the photographer and our team an action plan for the afternoon and a guide for us to use as a reference throughout the day. We stuck to it meticulously:

startup storyboards

Scout the locations

We made sure that we visited each of the locations prior to the day, and that they were suitable for the photograph that we wanted. The two processes weren’t mutually exclusive, and the ideas from the storyboards derived from what we felt was possible. We needed to plan and capture 3 photos that fit our timeframe, which meant they all needed to be located close to our office in Chiswick, London.

Here we are in the park the day before the shoot, picking the best location:

location-scouting-startup

The photographs

We wanted to capture three distinct use cases. Monitoring dashboards displayed on a screen in an office; diagnosing / fixing infrastructure problems at a desk; and getting a monitoring alert on the go.

Looking at these uses of Server Density as: Passive, active and reactive helped with creating a suitable environment for them to be shot in.

Server Monitoring Dashboard

This photo was about showing off how you can use Server Density in your office environment, for this one we wanted to create a hipster feel with an interior brick wall that gave the impression of creative as opposed to clinical. We found a white brick wall that (until a few weeks previous when the roof had been knocked off) had been an interior wall. Here is how it looked to any passers by:

Behind the scenes dashboard

Server Monitoring Graphs

The graphing photograph was about showing Server Density off in a more clinical environment of getting the problem fixed. We did this in our office, where we had all the appropriate equipment. We found a nice wall, moved the office around for a few hours and generally caused chaos in an otherwise rather peaceful working environment.

Oh and I almost forgot about the pièce de résistance – The Deathstar! The end result of this was also a hat-tip to our friends at New Relic, who used a similar homepage image a while ago. Of course, we added our own local touches like coffee from our local shop and our custom dot notebooks.

Server monitoring graphs

Server Monitoring Alerts

Finally it was important to show how you might receive an alert in your day to day life as a devops practitioner or sysadmin. There’s no better change of scenery than a park and we’re lucky enough to have an office a minutes walking distance from greenery.

We also wanted to see the underground in the distance as a reference to London, and to emphasise that you’re away from the hustle and bustle, but still able to stay in control, and on top of your infrastructure.

Monitoring Alerts behind the scenes

A little bit of luck

I consider ourselves pretty fortunate (or maybe it was good judgement) to have ended up with the photographer we did. He was incredibly knowledgeable, shared our vision and was more than happy to problem solve along the way. I think this was the real make or break factor of this project.

We were also lucky in the sense that we didn’t get rain on the day, which was important considering two of our photographs were taken outside. We did however capitalise on this opportunity by timing our shoot correctly. For example, we spent the early afternoon getting the park shot done in the clear daylight; we then completed the office shot which was the easiest; and then had enough time to finish the dashboard shot as the sun was going down. You might be able to see from the previous image, the shadows on the walls would have been a problem any earlier in the day.

The editing

Now I don’t want to destroy the smoke and mirrors that we’ve so eloquently created, but our photos did have to be edited. Particularly the dashboard shot which:

  1. Wasn’t shot inside
  2. Wasn’t mounted on the wall
  3. Didn’t have a lightbulb hanging

Also naturally all of the screenshots were added in at the end of the process so that we can update them going forward as our interface gets tweaked.

The big unveil

In the interest of keeping the page load of this blog post less obscene than it already is, then you can take a look at the final images on our updated homepage.

The legal side

Interestingly we hit a little bit of a stumbling block at the final stage when signing off the pictures. Contrary to what I thought, it isn’t standard for you to own the full copyright of the images once they have been completed. The photographer usually licenses them out to you for an arranged timeframe and predefined use.

Naively this is something that should have been sorted out with our photographer prior to commissioning the photographs, terms of use and length of license should be talked through more formally than we did. However, thankfully our photographer was one of those good old fashioned English gents to which a handshake was as good as a signature, so all was fine in the end and we completed the contract after having completed the photos.

After our 10 year license runs out, we’ll have to talk to him about renewing on separate terms / perhaps redoing the pictures. Goodness knows how big the iPhone will be in 2024!

The post Photography for tech startups appeared first on Server Density Blog.

by Rufus at September 17, 2014 11:27 AM

Daniel E. Markle

2014 Lake Hope Rebel Rally

2014 Lake Hope Rebel Rally
After a hiatus last year due to work, I made it once more to the Lake Hope Rebel Rally this year. Having waited to reserve until after the cabins had filled, I decided to double up the adventure on this trip as my first motorcycle camping trip.


Continue reading "2014 Lake Hope Rebel Rally"

by dmarkle@ashtech.net (Daniel E. Markle) at September 17, 2014 10:51 AM

Raymii.org

Boot to Vim, Vim as Pid 1

This is a response on a great article from Pascal Bourguignon, namely how to run Emacs as PID 1. As we all know, nobody uses emacs. No, all joking aside, I found it to be a good article and wanted to see how I could do that with Vim. Not in User Mode Linux, but by creating an actual ISO. Boot to Vim, as you might want to call it. This is actually fairly simple. Compile vim statically, set it as init= at boot and you're done. We are going to use small 9MB distro named Tiny Core, Core edition and customize that to boot right into our static build of Vim.

September 17, 2014 07:29 AM

Yellow Bricks

Queue Depth info in the VSAN HCL!


I just noticed there has been an update to the VSAN HCL. When I now do a search for a disk controller (vmwa.re/vsanhcl) it immediately shows the queue depth of the controller. This will make life a lot easier, especially for those who prefer to build their own Virtual SAN node instead of using a Ready Node configuration. Although it is just a minor detail it is useful to know, and will definitely make life a lot easier when configuring your component built Virtual SAN nodes.

"Queue Depth info in the VSAN HCL!" originally appeared on Yellow-Bricks.com. Follow me on twitter - @DuncanYB.


Pre-order my upcoming book Essential Virtual SAN via Pearson today!

by Duncan Epping at September 17, 2014 07:10 AM

Chris Siebenmann

In praise of Solaris's pfiles command

I'm sure that at one point I was introduced to pfiles through a description that called it the Solaris version of lsof for a single process. This is true as far as it goes and I'm certain that I used pfiles as nothing more than this for a long time, but it understates what pfiles can do for you. This is because pfiles will give you a fair amount more information than lsof will, and much of that information is useful stuff to know.

Like lsof, pfiles will generally report what a file descriptor maps to (file, device, network connection, and Solaris IPC 'doors', often with information about what process is on the other end of the door). Unlike on some systems, the pfiles information is good enough to let you track down who is on the other end of Unix domain sockets and pipes. Sockets endpoints are usually reported directly; pipe information generally takes cross-correlating with other processes to see who else has an S_IFIFO with the specific ino open.

(You would think that getting information on the destination of Unix domain sockets would be basic information, but on some systems it can take terrible hacks.)

Pfiles will also report some socket state information for sockets, like the socket flags and the send and receive buffers. Personally I don't find this deeply useful and I wish that pfiles also showed things like the TCP window and ACK state. Fortunately you can get this protocol information with 'netstat -f inet -P tcp' or 'netstat -v -f inet -P tcp' (if you want lots of details).

Going beyond this lsof-like information, pfiles will also report various fcntl() and open() flags for the file descriptor. This will give you basic information like the FD's read/write status, but it goes beyond this; for example, you can immediately see whether or not a process has its sockets open in non-blocking mode (which can be important). This is often stuff that is not reported by other tools and having it handy can save you from needing deep dives with DTrace, a debugger, or the program source code.

(I'm sensitive to several of these issues because my recent Amanda troubleshooting left me needing to chart out the flow of pipes and to know whether some sockets were nonblocking or not. I could also have done with information on TCP window sizes at the time, but I didn't find the netstat stuff until just now. That's how it goes sometimes.)

by cks at September 17, 2014 06:04 AM


Administered by Joe. Content copyright by their respective authors.