Joey chatter
who didn't start the fire

Home from 3 days away, the house was 62°F despite outside being 36°. Passive solar heating FTW!

Joey chatter

Black Friday Thursday

Anna (Anna and Mark: Waldeneffect)
Wood-stove humidifier

Wood stove humidifierLack of humidity is not something our farm usually suffers from. But last week's cold spell was notable in the extremely low water content of the air, which led to all kinds of minor discomforts in my nose and lips.

Luckily, Mark can fix just about anything. He was less than impressed with the pots I pulled out of kitchen circulation to fill with water and place atop the wood stove, so he instead went out and tracked down an old tea kettle. The vessel had been used as part of a deer deterrent, but with our deer situation much more under control lately, we felt comfortable repurposing the kettle as a wood-stove humidifier.

I wonder what this repurposed kettle will turn into next. Heck, maybe we'll even use it to make tea!

Anna (Anna and Mark: Waldeneffect)
Pie(s) are squared
Pies in progress

I know, I know, pies are meant to be round. But this Thanksgiving, pies are squared (or at least rectangular). In the past, I've carefully carried pies out across our floodplain...only to find specks of mud atop my perfect crust or meringue once we reached our destination. Not this year! Instead, I've upgraded to lidded casserole dishes...which have the added benefit of making a very deep-dish pie.

I hope your pies are similarly mud-free! Happy Thanksgiving!

Joey chatter

If it helps, I often compile once, after 5 hours of work.

(More haskell nut programmer than senior programmer however.)

Joey chatter
ampersand generates invalid feeds for posts with an ampersand in the title. I need to post some stuff to run that post off my feed.

Not the first, or the last rss/atom thing to be broken this way..

Joey chatter

Worth noting that most battery monitors also lie about "100% charged", which tends to really be 95% or so due to a degraded battery. You can see the real number with acpi -V.

I once wrote a battery meter that didn't lie, and users hated that.

Joey chatter

I have not run attic, but I have read the docs. Its encryption seems much less sophisticated than obnam; it just uses simple AES and not public key.

I have been tuning the performance of my local obnam backup though, and got it down to 15 minutes.

Joey chatter

Just got an email from a propellor user whose propellor code is his first haskell program.

This makes me very happy, because my xmonad code was my first haskell program. Now I've given it forward.

Anna (Anna and Mark: Waldeneffect)
High-density pear trees
Fruiting high density pears

In an earlier post, I teased you by saying that next year's high-density experiment will veer off in an entirely new direction. But, really, it's the same experiment...just with a different species of tree.

High-density apple orchards have become big business in the U.S., but at this time, pears are mostly grown in a more traditional, spaced-out setting. However, one report I read mentioned that high-density pear plantings are already common in Europe, suggesting that close plantings can be appropriate for this other pome as well. Since I have several additional pear varieties that I want to try out but not enough space for several additional full-size trees, I figured --- why not experiment with a high-density planting for pears?

The best option for high-density pear trees appears to be a 4-foot spacing with the limbs tied down to 45 degrees below the horizontal. To make this work, the New York State Horticultural Society experimenters recommend using semidwarf rootstocks like OHF87, which appeared to be quite acceptable in high-density plantings during the eight years of their study (and, the author thought, most likely also for the entire life span of the orchard). I ended up buying OHF513 instead for my own planting since the nursery I wanted to order from uses this similarly-sized rootstock rather than OHF87, so I guess in a few years I'll be able to report on how well OHF513 does for high-density plantings.

High density pears

There are a few downsides to high-density pear plantings that aren't a factor when similar strategies are used on apples. First, the fruits on high-density pear trees tend to be on the small side, and pear rootstocks also aren't as precocious as those used to dwarf apples. As a result, the high-density pear researchers found that, even when planting feathered trees, you really shouldn't expect your first small pear crop until the third year after planting, and major production won't begin until the fourth year. If you're starting with rootstocks that you graft at home, you should add another year onto that figure, meaning that we probably won't see any pears from our planned row until about 2018.

But what could be more fun than grafting five little pear trees and setting aside another garden row for planting out the young trees at this time next year? Nothing! So, of course, I have to give it a shot.

6 am Quintain

Dreams, steps down a dirt road.

All alternate realities.

Last night i flipped channels

left sealed for decades, 

Reminded I am not only me.

Anna (Anna and Mark: Waldeneffect)
Roaring winds
Disgruntled quick hoop

Wind chimesWe live deep down in a valley (known locally as a holler) where we seldom feel breezes and even less seldom are faced with strong winds. So...I get lazy. I lay down cardboard kill mulches with just a rock or two to weigh the sheets down (if that), and this fall I minimized the number of bricks holding down the sides of our quick hoops to a mere six per 15-foot span.

But I've noticed recently that big changes in temperature do bring winds, even down here in our holler. And those roaring winds toss cardboard around the yard and whip right through lazily built quick hoops. The results are shown above.

When I went out to fix my quick hoops Monday afternoon, though, I still didn't increase the brick count. With one wind rushing through our valley already this winter, chances are we won't see another until March.

Joey chatter

Oh for a little technical literacy.

Joey chatter

Small library, it'll be offsite.

Joey chatter

Wonder how to tell the library that their DNS server is broken. "Your DNS server is handing out the same IP address to multiple clients at once and I know because I worked around it like this" seems unlikely to work.

Anna (Anna and Mark: Waldeneffect)
Chicken books on sale

Chicken booksThe chicken-lovers among you will be thrilled to hear that I'm celebrating Thanksgiving early by putting my chicken books on sale! But before you go nodding off, you can get the first book without plunking down a cent --- The Working Chicken is currently free on Smashwords and at Barnes & Noble. Find out why hard-nosed homesteaders don't name their chickens and much more in this photo-rich introduction to backyard chicken care.

If that introduction tempts your appetite, my more in-depth series, Permaculture Chicken, includes three books bound to make your chicken-keeping adventure run more smoothly. And each ebook is marked down to 99 cents this week --- buy them all and save 74%! Here are the links: Permaculture Chicken: Incubation Handbook, Pasture Basics, and Thrifty Chicken Breeds. Maybe next year you can grow your own free-range chicken for Thanksgiving!

Thanks for reading! And if you like what you read, why not make my day by leaving a review?

Bewitched little laughing man

I see him first in the stump of a tree

after we picked dandelion greens,

heaved ice daggers off a bank,

and watched them race in the creek.


Pointing him out I almost touch him but I'm saved.

Mom sees him too and is ready 

to escape this mist before we too

become figures locked in hollow tree trunks.


I have felt very safe here with the eyes 

in the forest watching me through seasons.

As we drive away I see the mythical mountain creek silkie 

splash her lengthy crawdad tail as she submerges.

mark (Anna and Mark: Waldeneffect)
New tricks
making a door to block cats from hallway

This is the first year we've trained Huckleberry and Strider to be good in the morning.

They've had to sleep on the porch at night due to Huckleberry's problem of waking everybody up at the crack of dawn.

With this new hallway door we block off his chance to be a morning cat.

Joey chatter
eDSLs are wonderful

Roughing out how the propellor config file will look when using it to do a clean reinstall of a system.

So, after the basic clean reinstall, some things need to be done to eg, fix up the network interfaces and get sshd going. But you only ever want those actions to run after the clean install, not every time propellor runs thereafter. Turns out I already had this handy onChange combinator, that solved this beautifully.

& os (System (Debian Unstable) "amd64")
& cleanInstallOnce (confirm "")
     `onChange` propertyList "fixing up after clean install"
             [ fixupNetworkInterfaces
             , fixupRootSsh
             -- , installDistroKernel
             -- , installGrub
& Apt.installed ["sshd"]
-- rest of system properties here
Anna (Anna and Mark: Waldeneffect)
Three new high-density apple experiments

Dwarf apple rootballAlthough it's a little premature to count our two-year-old high-density apple experiment as a success (since frost nipped all of the blooms this spring), I'm feeling very positive about the system. Planting the apple trees close together allows me to try out lots of different varieties, which in turn makes it easy to select varieties that resist cedar apple rust and our other local bugaboos. The high-density row doesn't take up much precious garden space, and the summer pruning (although frequent) is simple and fun. No wonder Mark and I chose to plant two more high-density apple rows this fall!

With this second planting, I'm experimenting in three different directions. Two years ago, I mostly chose trees grafted onto Bud 9, M26, and Geneva 11 rootstock, meaning that the trees are true dwarfs, but I also included two trees on a semi-dwarf (MM111) rootstock. The semidwarf trees grew very well...but they've already gotten quite a bit bigger than their neighbors. So, when I grafted Planting an apple treeonto MM111 for some of this year's new trees, I expanded the within-row spacing to 6 feet, hoping that the additional elbow room will help our semidwarf apples achieve their full potential while still toeing the high-density line. I also plan to train the MM111 trees' limbs down considerably below the horizontal this time around, which I was a bit more cautious about in previous years but which I've since decided is definitely a good option for high-density apples in the backyard.

I also opted to branch out and try yet another rootstock this year --- M7, which will produce trees midway in size between the true dwarfs on Bud 9 and the semidwarfs on MM111. My M7 trees went into the ground at 53-inch spacing but will otherwise be treated the same as the MM111 trees. I'll be curious to see, over the next few years, which rootstock turns out to be the best fit for high-density plantings on our farm. It's a bit of a tradeoff --- the more dwarfing the rootstock, the more precocious the tree, meaning that we'll get more fruits faster. But, at the same time, truly dwarf rootstocks have a hard time growing if you don't give them constant TLC, and a few of the trees in my original planting (on Geneva-11 or Bud 9 rootstock) did fail to thrive. Hopefully, either the M7 or MM111 trees (or both) will provide a happy middle ground --- apple trees that do pretty well without watering and other bonus attention, but that also produce within a few years after planting.

Espaliering an apple

I've read lots of good and bad about espaliers (my third high-density apple experiment), so I earmarked only one tree for this final endeavor. I settled on an informal design set against the south side of our front porch and began by bending the young tree so the top was nearly horizontal. As watersprouts inevitably pop up from the flattened trunk, I'll probably bend them at a 45-degree angle to create a type of lattice pattern...or whatever seems to make sense from the growth pattern of the tree. Since I'm far from confident that my espalier will thrive, though, I chose our Chestnut Crab for the experiment ---after all, I'm mostly growing this sweet crabapple variety out of sentimental attachment to a similar tree of my youth, so I won't feel too bad if I don't get high yields.

I'll keep you posted on all three new plantings in the years ahead...and hopefully will be able to report in summer 2015 about our first big crop from our older high-density planting. In the meantime, stay tuned for another post about next year's high-density experiment, which will veer off in yet another direction.

Joey chatter

oh debootstrap, why must you depend on perl?

(I know why, I used to maintain debootstrap. Still very annoying.)

Joey chatter

Taught propellor how to bounce off other servers and install itself onto arbitrary linux boxes. Not a virus, but..

This transcript(lightly edited for clarity) shows it bouncing from my laptop to my server (which has better bandwidth), from which it uploads itself and all its deps to a redhat VM I spun up for testing.

Next and final step: Writing the evil function ... convertOS :: System -> Property

joey@darkstar:~/src/propellor#joeyconfig>./propellor --spin --via clam
Propellor build ... done
Git commit ... done
Push to central git repository ... done
Identity added: /home/joey/.ssh/id_rsa (rsa w/o comment)
Pull from central git repository ... done
git branch origin/joeyconfig gpg signature verified; merging
Updating aa98672..95f78a0
Sending privdata (11 bytes) to clam ... done
Sending git update to clam ... done
Propellor build ... done
sh: git: command not found
sh: apt-get: command not found
propellor.tar32695.                           100% 6479KB   6.3MB/s   00:00
Uploading locally compiled propellor as a last resort ... done
Sending privdata (11 bytes) to ... done has ipv4 ... ok overall ... ok
Shared connection to closed.
Connection to clam closed.
mark (Anna and Mark: Waldeneffect)
Goats and sticks
apple planting day

I was a little worried about having the goats grazing on oats so close to our new apple trees, but it seems like they're not interested in anything with bark yet.

Joey chatter

So that's what a $250 cluster of 5 servers looks like these days. Wow.

I wonder if someone could start a small business putting those together, in 5, 10, etc server sizes and sell them at some reasonable markup. It would probably need a 3d printed case; the power connections look somewhat fragile.

Anna (Anna and Mark: Waldeneffect)
Failed experiment: Goats in the woods
Goats eating honeysuckle

So, my goats-in-the-woods experiment lasted all of about two hours. I let the girls loose, settled down to write...and soon heard Artemesia yelling at the top of her lungs. Abigail had circled around to the part of our boundary that has the lowest fence and had hopped right over, but our doeling's stubby little legs didn't allow her to follow. I guess it's a good thing that Artemesia is part Nubian since there was no missing her anguished yells as she was left Dwarf doelingbehind. Or maybe our doeling was just telling on her big sister? Either way, I pulled Abigail out of the garden before she could do any damage, then I stuffed both goats back into the pasture with the honeysuckle trees shown above.

For experiment number two, I decided to open the door on the far side of the starplate coop, meaning that our goats would have to walk through some rough terrain to circle around the fenced pastures and reach our core homestead. Sure enough, when I came back from walking Lucy, I discovered that our goats had decided to explore in the opposite direction. But Artemesia was yelling again, and I got worried (even though our doeling sometimes just likes to yell) and went to see what was up. No one was in trouble, but both goats followed me right home, negating that experiment.

Doe with horns

Next, I decided to try tethering Abigail on the far side of the starplate coop. I figured that Artemesia would stay close to her companion, and that everyone would be happy. So when I heard non-Nubian yelling I guessed that our doe must have gotten her chain hung up. Nope. Artemesia had decided to wander far afield in search of honeysuckle, and her big sister was having a fit at being left alone. So, once again, I stuffed the girls back into the pasture for safe keeping. I guess they're stuck eating hay now except when I take them out on monitored walks...unless I come up with another supposedly bright technique for letting them run wild in the woods.

propelling containers

Propellor has supported docker containers for a "long" time, and it works great. This week I've worked on adding more container support.

docker containers (revisited)

The syntax for docker containers has changed slightly. Here's how it looks now:

example :: Host
example = host ""
    & Docker.docked webserverContainer

webserverContainer :: Docker.Container
webserverContainer = Docker.container "webserver" "joeyh/debian-stable"
    & os (System (Debian (Stable "wheezy")) "amd64")
    & Docker.publish "80:80"
    & Apt.serviceInstalledRunning "apache2"
    & alias ""

That makes have a web server in a docker container, as you'd expect, and when propellor is used to deploy the DNS server it'll automatically make point to the host (or hosts!) where this container is docked.

I use docker a lot, but I have drank little of the Docker KoolAid. I'm not keen on using random blobs created by random third parties using either unreproducible methods, or the weirdly underpowered dockerfiles. (As for vast complicated collections of containers that each run one program and talk to one another etc ... I'll wait and see.)

That's why propellor runs inside the docker container and deploys whatever configuration I tell it to, in a way that's both replicatable later and lets me use the full power of Haskell.

Which turns out to be useful when moving on from docker containers to something else...

systemd-nspawn containers

Propellor now supports containers using systemd-nspawn. It looks a lot like the docker example.

example :: Host
example = host ""
    & Systemd.persistentJournal
    & Systemd.nspawned webserverContainer

webserverContainer :: Systemd.Container
webserverContainer = Systemd.container "webserver" chroot
    & Apt.serviceInstalledRunning "apache2"
    & alias ""
    chroot = Chroot.debootstrapped (System (Debian Unstable) "amd64") Debootstrap.MinBase

Notice how I specified the Debian Unstable chroot that forms the basis of this container. Propellor sets up the container by running debootstrap, boots it up using systemd-nspawn, and then runs inside the container to provision it.

Unlike docker containers, systemd-nspawn containers use systemd as their init, and it all integrates rather beautifully. You can see the container listed in systemctl status, including the services running inside it, use journalctl to examine its logs, etc.

But no, systemd is the devil, and docker is too trendy...


Propellor now also supports deploying good old chroots. It looks a lot like the other containers. Rather than repeat myself a third time, and because we don't really run webservers inside chroots much, here's a slightly different example.

example :: Host
example = host "mylaptop"
    & Chroot.provisioned (buildDepChroot "git-annex")

buildDepChroot :: Apt.Package -> Chroot.Chroot
buildDepChroot pkg = Chroot.debootstrapped system Debootstrap.BuildD dir
    & Apt.buildDep pkg
    dir = /srv/chroot/builddep/"++pkg
   system = System (Debian Unstable) "amd64"

Again this uses debootstrap to build the chroot, and then it runs propellor inside the chroot to provision it (btw without bothering to install propellor there, thanks to the magic of bind mounts and completely linux distribution-independent packaging).

In fact, the systemd-nspawn container code reuses the chroot code, and so turns out to be really rather simple. 132 lines for the chroot support, and 167 lines for the systemd support (which goes somewhat beyond the nspawn containers shown above).

Which leads to the hardest part of all this...


Making a propellor property for debootstrap should be easy. And it was, for Debian systems. However, I have crazy plans that involve running propellor on non-Debian systems, to debootstrap something, and installing debootstrap on an arbitrary linux system is ... too hard.

In the end, I needed 253 lines of code to do it, which is barely one magnitude less code than the size of debootstrap itself. I won't go into the ugly details, but this could be made a lot easier if debootstrap catered more to being used outside of Debian.


Docker and systemd-nspawn have different strengths and weaknesses, and there are sure to be more container systems to come. I'm pleased that Propellor can add support for a new container system in a few hundred lines of code, and that it abstracts away all the unimportant differences between these systems.


Seems likely that systemd-nspawn containers can be nested to any depth. So, here's a new kind of fork bomb!

infinitelyNestedContainer :: Systemd.Container
infinitelyNestedContainer = Systemd.container "evil-systemd"
    (Chroot.debootstrapped (System (Debian Unstable) "amd64") Debootstrap.MinBase)
    & Systemd.nspawned infinitelyNestedContainer

Strongly typed purely functional container deployment can only protect us against a certian subset of all badly thought out systems. ;)

Anna (Anna and Mark: Waldeneffect)
Cold November
Frozen fava beans

There's something psychologically colder about nights that get down into the single digits. Or maybe it's not completely psychological. Gates freeze shut, my hands ache when I go out to do my morning chores, and the uncovered winter crops begin to die back.

Wood stove

Last year at this time, we enjoyed a similar cold spell, but the lowest low in November 2013 was 15. No wonder I ran through the firewood I had alloted for November 2014 by the middle of this month and have already started into December's wood.

Frosty dog

Everyone else on the farm is glad that we're due to enjoy a bit more fall weather this coming week as the current Arctic burst goes back where it belongs. But Lucy loves the cold, so she might be sad to see it go. Don't worry, Lucy --- there are many more frosty mornings ahead!

mark (Anna and Mark: Waldeneffect)
Manger danger
goat manger in action

Of course the goats wanted to be on top of the new manger.

The thin plywood lid was collapsing when they stood on it, which could be a safety issue if they fall the wrong way.

Adding some 2x4's for support makes it more standable.

Anna (Anna and Mark: Waldeneffect)
Goat books for beginners

Storey's Guide to Raising Dairy GoatsI know that some weeks it seems like all I do is talk about goats and books. So why not shake it up...and talk about goat books?!

When I first started researching goats, my first stop was Storey's Guide to Raising Dairy Goats. The Storey series is usually a safe bet for encyclopedia-style information on livestock combined with beautiful pictures, and this book was no different (although a little less in-depth than some). If you've never met a goat before and are only going to get one book, this is probably the one to buy.

But once I finished that beginner guide...I still felt like a beginner. So I moved on to Raising Goats Naturally. Deborah Niemann's book is also an introduction to goat care, but it's written in a more chatty, first-person fashion (a lot like my own books), which I suspect turns some people away. However, since I'm aware that all one-author books inevitably share that person's biases and Raising Goats Naturallyknowledge gaps, I enjoyed the honesty of Niemann's book and definitely pulled out some interesting tidbits that weren't covered in the Storey guide. Specifically, I learned that you should always breed miniature or partially miniature goats with bucks that are as small as the doe or smaller so that you don't have to worry about extra-large kids causing problems coming out. This and other factoids probably seem obvious to many of you, but I sucked them up happily, glad to have someone else's experiences to help me avoid beginner mistakes.

By the time I finished Niemann's book, I was starting to feel more like an accomplished goatkeeper...but I still didn't have goats. Since I couldn't move up our goat-arrival date, I settled on getting another book instead, this time Natural Goat Care by Pat Coleby. I'll admit up front that our two spoiled darlings arrived when I was only a Natural Goat Carequarter of the way through Coleby's book and my attention quickly turned to real, live goats, so I've still got a lot left to read, but I think that this book makes a very good addition to the beginning goatkeeper's long as you take the contents with a grain of salt. Coleby veers a little too far toward the personal-experience/no-science side for my tastes in a few spots, but most of her book walks a more middle ground. And she presents intriguing suggestions about how the prehistory of goats impacts their current needs, explaining that goats' tendency to browse on tree leaves means that the animals can develop mineral deficiencies when dining primarily on short-rooted grasses in human-build pastures. In turn, Coleby asserts that those cravings are what spur goats to break out of our pastures...which may be wishful thinking, but is worth considering.

I'd be curious to hear from our readers. Which other goat books do you feel help beginners turn into permaculture goat herders? Did I miss an obvious introductory text from my lineup?

Anna (Anna and Mark: Waldeneffect)
Learning from garden failures
Roast brussels sprouts

We enjoyed our first and possibly only roast brussels sprouts of the season Tuesday, the combination of a new variety and an extremely wet fall meaning that the plants blighted instead of thrived. The experience made me think about how frequently home gardeners give up on a crop because of a single failure, when what they really should have gotten out of the experience was an impulse to figure out what made their plants refuse to grow.

For example, I often hear from folks who think carrots aren't worth growing, while for us the tasty roots are an easy crop. Well, an easy crop as long as I pay attention and make sure their seeds germinate during the summer heat. And as long as I locate the root vegetables in loose, humus-rich soil. So, not really an easy crop, but easy once you figure out what factors of your unique site are standing in the way of getting a stellar carrot crop.

Garden vegetables

Now that the cold weather has truly set in and most of you have nothing left to plant for the year, why not spend a few hours thinking back over your garden past? When you look at all of those luscious-looking pictures in the seed catalogs this winter, try to ignore the pretty photos and tantalizing descriptions. Instead, seek out the less sensational but more important notes on which blights each variety is resistant to and how well they do in other difficult situations that your garden will throw at them in the year to come.

And, as a reward, next year your garden will grow twice as well!


List of feeds:

  • Anna: last checked (25 posts)
  • Anna and Mark: Waldeneffect: last checked (2175 posts)
  • Anna and Mark: Clinch Trails: last checked (10 posts)
  • Joey: last checked (76 posts)
  • Joey chatter: last checked (830 posts)
  • Joey git-annex devblog: last checked (220 posts)
  • Joey: blog: last checked (12 posts)
  • Jay: last checked (25 posts)
  • Dani: last checked (21 posts)
  • Errol: last checked (28 posts)
  • Maggie: last checked (849 posts)
  • Tomoko: last checked (69 posts)
  • Jerry: last checked (28 posts)